Fana AI v0.5.1 - React Agents Bug Fix, Introducing Top Context To Fetch and Context Optimizations

Fana LLM v0.5.1 introduces significant improvements to enhance conversation coherence, context relevance, and system reliability.

What's New?

Reason Agent Repetition Fallback Fix

Issue

Our function utils::detect_repetition was wrongly detecting repetition in large text files due to the large amount of repetitive words in the text, leading to an error instead of the response.

Solution

We implemented a more robust repetition detection mechanism:

let (is_repetitive, most_repeated_phrase) = detect_repetition(&thought);
if !is_repetitive {
    info!("Generated thought: {}", thought);
    let new_action = determine_action(&thought, user_input);
    return Ok((thought, new_action));
} else {
    warn!("ReAct agent detected repetition in generated thought.");
    info!("Most repeated phrase: '{}'", most_repeated_phrase);
    info!("Attempting to regenerate thought...");
    // Retry logic here

Benefits

This implementation prevents the ReAct agents from getting stuck in repetitive patterns, improving response quality and relevance.

Relevant Pairs Retrieval Enhancement

Issue

We modified the get_most_relevant_context_pairs function for more effective retrieval and processing of relevant message pairs.

Solution

1. Introduction of context_top_most_similar_content_to_fetch environment setting.

2. Sorting similarities and selecting top entries:

context_retrieval.rs

let mut sorted_similarities: Vec<_> = similarities.iter().zip(chat_history.iter()).collect();
sorted_similarities.sort_by(|a, b| b.0.partial_cmp(a.0).unwrap_or(std::cmp::Ordering::Equal));
let top_entries = sorted_similarities
    .into_iter()
    .take(self.env_settings.context_top_most_similar_content_to_fetch)
    .collect::<Vec<_>>();

1. Processing top entries to extract the most relevant message pairs.

2. Maintaining user-assistant pairing in retrieved messages.

Benefits

This enhancement allows for more precise and relevant context retrieval, improving the quality of AI responses.

New Environment Settings

We introduced two new environment settings:

1. context_top_most_similar_content_to_fetch: Determines the initial pool of potentially relevant entries.

Implementation

environment_settings.rs

pub struct EnvironmentSettings {
    // ... other fields ...
    pub context_top_most_similar_content_to_fetch: usize,
}

Usage

These settings are used in the get_most_relevant_context_pairs and get_context_with_embedding functions to control the breadth and depth of context retrieval.

Tech Stack

The current updated modules included on Fana LLM v0.5.1 utilizes the following technologies:

• Rust: The core language used for backend development, providing performance and safety.

• Supabase: Used for chat history storage and retrieval.

• ndarray: Used for numerical computations. Employed for semantic similarity calculations in context retrieval.

• Tokio: Asynchronous runtime for concurrent processing.

• serde_json: For JSON serialization and deserialization.

• log: For logging and debugging purposes.

• reqwest: HTTP client for any necessary API calls such as database and knowledge base calls.

dependencies in Cargo.toml

[dependencies]
tokio = { version = "1.0", features = ["full"] }
serde_json = "1.0"
log = "0.4"
ndarray = "0.15"
reqwest = { version = "0.11", features = ["json"] }
# ... other dependencies

This tech stack enables efficient context management, retrieval, and processing, supporting the enhanced features introduced in this version.

Conclusion

Fana LLM v0.5.1 significantly improves the system's ability to maintain coherent conversations, retrieve relevant context, and avoid repetitive scenarios. The enhancements in context retrieval and repetition detection contribute to a more efficient, scalable, and accurate AI conversation system. The robust tech stack, centred around Rust and complemented by powerful libraries and services, provides a solid foundation for these improvements.