FANA LLM v0.2.7 - Enhanced Analysis Process, 16_ID, Image Upload Processing, Token Usage Tracking

Deployment coming on our next update, Monday, May 20th, 2024.

Enhanced Image Upload Processing

Unique 16-character ID Hash Generation:
- Feature: Generates a unique 16-character random ID for each image uploaded via API.
- Ensures precise metadata tracking with a timestamp indicating the upload date and UTC time.

New Modules:

generate_random_id:

Generates a random 8-character ID.

async def generate_random_id(length=8):
    return ''.join(random.choices(string.ascii_letters + string.digits, k=length))

sanitize_filename:

Replaces invalid characters in filenames with an underscore.

@observe()
async def sanitize_filename(filename):
    """Replace invalid characters in filenames with an underscore and replace spaces with underscores."""
    invalid_chars = r"[\\/?%*:|\"<> ]"
    sanitized_name = re.sub(invalid_chars, "_", filename)
    random_id = await generate_random_id()
    return f"{sanitized_name}_{random_id}"

Enhanced Computer Vision Processing

Bug Fixes:
- Image Analysis Duplication:
  - Issue: Images were analyzed twice when a URL or image was uploaded.
  - Solution: Refined logic in handle_llm_interaction and process_image_interaction functions.
  - Implemented a flag to prevent duplicate processing.
  - Added return statements to halt further processing once the analysis result is obtained.
- Analyze_image Module Error:
  - Issue: "Unable to create new images" error.
  - Solution: Resolved critical error, ensuring smooth frontend operations.

Langfuse Tracking Improvements:

Null Pointer Exception Bug Fix
- Issue: Null pointer exception bug in the Langfuse tracking module.
- Solution: Ensured all image metadata contains Langfuse tracking information.
Analyze_image 'Generation' Trace Integrated
Span Latency Enhanced

Token Usage, Logging and Its Importance

Token Logging Overview: Token logging is an essential aspect of managing interactions with large language models (LLMs). It involves keeping track of the number of tokens processed during each interaction. Tokens are the individual units of text that the model processes and they are a key factor in determining the cost and efficiency of using LLMs.

Why Token Logging is Crucial:

Cost Monitoring:
- Expense Tracking: Each token processed by an LLM incurs a cost. By logging tokens, organizations can accurately monitor their spending on LLM services.
- Budget Management: Keeping track of token usage helps in managing and forecasting budgets effectively. It prevents unexpected costs and allows for better financial planning.
Efficiency Optimization:
- Identifying Redundancies: Logging tokens helps identify unnecessary token usage, such as redundant prompts or excessively verbose responses. This ensures the LLM operates efficiently without wasting tokens.
- Prompt Engineering: Understanding token usage patterns aids in refining prompts to be more concise and effective, thus reducing the overall token consumption.
Performance Analysis:
- Response Quality: By analyzing token usage, developers can assess the quality and relevance of responses generated by the LLM. It helps in fine-tuning the model for better performance.
- User Experience: Monitoring token usage ensures that users receive concise and relevant responses, enhancing their overall experience.
Resource Allocation:
- Scaling Decisions: Token logs provide insights into the demand and usage patterns of the LLM. This information is vital for making informed decisions about scaling resources up or down.
- Load Balancing: By understanding token usage, organizations can optimize load balancing across different LLM instances, ensuring smoother operations.

Implementation Example

Here's a brief example of how token logging is implemented in our system:

Example Log for Token Usage:

2024-05-18 09:24:35 - INFO: The most similar response was rewritten. Tokens: 553
...
...
2024-05-18 09:24:35 - INFO: Tokens used for input in analyze image: 40
2024-05-18 09:24:35 - INFO: Tokens used for output in analyze image: 75
2024-05-18 09:24:35 - INFO: Total tokens used in analyze image: 115
...
...
2024-05-18 09:24:49 - INFO: Tokens used for input in generate image: 230
2024-05-18 09:24:49 - INFO: Tokens used for output in generate image: 108
2024-05-18 09:24:49 - INFO: Total tokens used in generate image: 338

Key Implementation Details:

Importing tiktoken and initializing the tokenizer:

import tiktoken
tokenizer = tiktoken.get_encoding("cl100k_base")

Counting Tokens in Messages:

async def count_tokens(messages):
    """Utility function to count tokens in a list of messages."""
    num_tokens = 0
    for message in messages:
        num_tokens += 3  # tokensPerMessage
        num_tokens += len(tokenizer.encode(message["content"]))
    return num_tokens + 1  # tokensPerName

Handling Text Interaction with Token Logging:

@observe()
async def handle_text_interaction(user_input, chat_history, client):
    logging.info("Received user input entering handle text interaction module")
....
            tokens_used = await count_tokens([
.....

Debug Logging Improvement:

Enhanced Log Format:

Added date, time, and seconds to track the duration of each process.

# modules.logging_setup.py
import logging
import os

log_dir = "./logs/"
log_file = "debug.log"

def setup_logging():
    if not os.path.exists(log_dir):
        os.makedirs(log_dir)
    log_file_path = os.path.join(log_dir, log_file)
    logging.basicConfig(
        level=logging.INFO,
        filename=log_file_path,
        filemode="a",
        format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S",
    )

Enhanced Excerpt:

Sample Log Output:

2024-05-17 18:50:09 - root INFO: Received text: None, file: Screenshot 2024-05-17 at 18.47.37.png
2024-05-17 18:50:09 - root INFO: Inside azure blob module. Uploading image to Azure Blob Storage
2024-05-17 18:50:09 - root INFO: Entering the sanitize filename module.
2024-05-17 18:50:09 - root INFO: Sending request to sanitize filename module to generate a random 16 key ID hash.
2024-05-17 18:50:09 - root INFO: Random 16-character ID hash generated successfully: LBtXiSoEXCJCh18h.
2024-05-17 18:50:09 - root INFO: Sanitize filename module generated a filename based on the current date and time successfully, followed by a 16-character ID hash: 2024-05-18_18.50.09_LBtXiSoEXCJCh18h.png.
2024-05-17 18:50:10 - root INFO: Leaving Azure Blob; image uploaded to Azure Blob storage successfully; URL: https://aimagine.blob.core.windows.net/aimagineuploads/2024-05-18_18.50.09_LBtXiSoEXCJCh18h.png
2024-05-17 18:50:10 - root INFO: Entering Handle LLM Interaction Module.
2024-05-17 18:50:10 - root INFO: Inside handle LLM interaction module. URL detected in user input, processing image interaction module.
2024-05-17 18:50:11 - root INFO: Entering process image interaction module.
2024-05-17 18:50:11 - root INFO: Inside process image interaction module. Searching for URL in the user-input
2024-05-17 18:50:11 - root INFO: URL found in user input.
2024-05-17 18:50:11 - root INFO: Sending a request to analyze image module.
2024-05-17 18:50:11 - root INFO: Entering analyze image module.
2024-05-17 18:50:11 - root INFO: Inside analyze image module. Request sent to computer vision.
2024-05-17 18:50:15 - root INFO: Received analyze image module response.
2024-05-17 18:50:15 - root INFO: Analyze image module processed the image successfully.
2024-05-17 18:50:15 - root INFO: Back to process image interaction module. Processed successfully.
2024-05-17 18:50:15 - root INFO: Leaving process image interaction module. No message along with URL found, returning analysis result.
2024-05-17 18:50:15 - root INFO: Image processed successfully with the process image interaction module; response generated and sent back to user.

This update ensures improved stability, performance, and functionality for the FANA LLM v0.2.6.1 release.

PreviousFANA LLM v0.2.8 - Telegram Integration - Text Formatting Enhancements NextFANA LLM v0.2.6 - GPT4o Integration, Enhanced API, URL Sanitizer, Additional Logging and Bugs Fixed

Last updated 6 months ago