Modular Architecture

In this chapter, we dive into the heart of the FANA LLM Framework's design philosophy—its modular architecture.

Modular Architecture

In the development of the FANA LLM Framework, we embrace a modular architecture approach to ensure our application is scalable, maintainable, and understandable. This architecture allows individual components or features to be developed, tested, and debugged independently, leading to a more robust and agile development process.

Why Modular Architecture?

• Scalability: Modules can be independently scaled depending on demand for specific functionalities, allowing for efficient use of resources.

• Maintainability: Changes in one module can be made with minimal impact on others, making it easier to manage updates and bug fixes.

• Reusability: Common functionalities are abstracted into modules that can be reused across different parts of the application, reducing code duplication.

• Collaboration: Teams can work on different modules simultaneously without significant conflicts, enhancing team productivity.

Backend Modularization

Our backend modules are organized under the src/ directory, ensuring that each logical component of our backend functionality is encapsulated within its own namespace. Key modules include:

• API Modules:

  • api_auth.py: Handles API authentication.

  • api_routes.py: Manages API endpoints routing.

  • api_validate_key.py: Validates API keys.

• Chat Modules:

  • chat_groq.py: Routes for chat API endpoints (Groq-based).

  • chat_history_supabase.py: Manages chat history using Supabase.

  • chat_openai.py: Routes for chat API endpoints (OpenAI-based).

• Content Modules:

  • content_fetcher.py: Finds the most similar content using cosine similarity.

  • db_supabase.py: Interacts with Supabase for retrieval-augmented generation (RAG).

• Embedding and Text Processing:

  • embedding.py: Generates and retrieves embeddings for RAG.

  • text_process.py: Manages text logic when no triggers are found.

• Image and Vision Modules:

  • image_diffusion_openai.py: Generates images using OpenAI.

  • image_process.py: Orchestrates all image processing logic.

  • image_vision_claude.py: Manages image vision API endpoints (Claude-based).

  • image_vision_openai.py: Manages image vision API endpoints (OpenAI-based).

• Logging:

  • logging_setup.py: Configures and manages logging for the system.

• Socket.IO:

  • socketio_event_handler.py: Handles Socket.IO events.

  • socketio_instance.py: Manages Socket.IO instances.

• Speech and Translation Modules:

  • speech_openai.py: Manages speech-to-text processing using OpenAI.

  • translate_openai.py: Manages multi-language translation using OpenAI.

  • translate_groq.py: Manages multi-language translation using Groq.

• Storage:

  • storage_azureblob.py: Uploads images to Azure Blob Storage.

• System and Task Management:

  • system_prompt.py: Generates and manages system-level prompts.

  • task_determination.py: Handles main LLM orchestration tasks.

• Trigger Modules:

  • triggers_check.py: Manages triggers API endpoints.

  • triggers_generate.py: Manages triggers generation endpoints.

  • triggers_greetings.py: Manages greeting triggers.

  • triggers_regenerate.py: Manages triggers regeneration endpoints.

Infrastructure as Code (IAC) for Scalability

We utilize Infrastructure as Code (IAC) practices to automate the provisioning and management of our infrastructure, ensuring that our environment is reproducible, scalable, and consistent. Our IAC setup includes:

• Terraform Scripts: Define and manage our cloud resources in a declarative manner.

• Docker Configurations: Ensure that our application can be consistently deployed across any environment.

Project Structure

Fana_LLM Project Structure/ 
├── src/  # Source code directory
│   ├── client/  # Front-end static files
│   │   ├── index.html  # Frontend HTML
│   │   ├── styles.css  # Frontend styling
│   │   └── frontend.js  # Frontend javascript
│   ├── data/  # Data directory
│   │   ├── docs/  # Documentation files
│   │   ├── env/  # Environment files
│   │   └── logs/  # Directory for log files
│   ├── api_auth.py  # API authentication
│   ├── api_routes.py  # API endpoints router
│   ├── api_validate_key.py  # API key validation
│   ├── chat_groq.py  # Chat API endpoints router
│   ├── chat_history_supabase.py  # Chat history API endpoints router
│   ├── chat_openai.py  # Chat API endpoints router
│   ├── content_fetcher.py  # Find the most similar content with cosine similarity
│   ├── db_supabase.py  # Supabase RAG Client
│   ├── embedding.py  # RAG Generate and Retrieve Emabeddings
│   ├── image_diffusion_openai.py  # Generate Image Module
│   ├── image_process.py  # Orchestrate all image-processing logic
│   ├── image_vision_claude.py  # Image vision API endpoints router
│   ├── image_vision_openai.py  # Image vision API endpoints router
│   ├── logging_setup.py  # Logs setup module
│   ├── socketio_event_handler.py  # Socket.IO event handler
│   ├── socketio_instance.py  # Socket.IO instance
│   ├── speech_openai.py  # Speech-to-text module
│   ├── storage_azureblob.py  # Upload an image to Azure Blob Storage
│   ├── system_prompt.py  # System Prompt
│   ├── task_determination.py  # Handle Main LLM Orchestration
│   ├── text_process.py  # Text logic once it doesn't find a trigger to generate
│   ├── translate_openai.py  # Multi-language translation logic
│   ├── translate_groq.py  # Multi-language translation logic
│   ├── triggers_check.py  # Triggers API endpoints router
│   ├── triggers_generate.py  # Triggers API endpoints router
│   ├── triggers_greetings.py  # Triggers API endpoints router
│   └── triggers_regenerate.py  # Triggers API endpoints router
├── main.py  # Main application entry point, set up servers
├── requirements.txt  # Requirements Libraries To Run Application
├── Dockerfile  # Docker configuration file
└── compose.yml  # Docker Compose configuration file

Updated on July 7th at 16:50 UTC

Summary

The modular architecture not only supports the rapid development and deployment of new features but also ensures that our system remains flexible and robust against the challenges of evolving user requirements and technological advancements. By documenting our architecture, we aim to provide a clear roadmap for current and future developers, fostering an environment of transparency and continuous improvement.