Update : clean and optimized retrieval from Chromadb

[haddybhaiya]

This PR improves context retrieval quality for intent extraction and test generation.

Changes:

• Increased Top-K retrieval for richer context
• Removed noisy and low-quality chunks
• Prioritized functions and classes for better relevance
• Eliminated duplicate chunks
• Added debug logging with chunk previews

Impact:

• Better context → improved intent detection → smarter test generation
• Reduced noise and redundancy in retrieved results
• Builds on Phase 3 improvements.

[github-actions]

Automated Code Review Comment:

Bugs

• Potential bug: In the query_embeddings function, if docs is an empty list, docs[0] will raise an IndexError.
• Potential bug: The MIN_CHUNK_LENGTH threshold is hardcoded, which might not be suitable for all use cases.

Improvements

• Code quality improvement: The new query_embeddings function is more robust, handling cases where no documents are retrieved and removing duplicates.
• Code quality improvement: The store_embeddings function now uses client.get_or_create_collection, which is safer than client.create_collection.

Suggestions

• Best practice: Consider making the MIN_CHUNK_LENGTH threshold a configurable parameter to improve flexibility.
• Best practice: Add error handling to the store_embeddings and query_embeddings functions to handle potential exceptions.
• Best practice: Consider adding type hints for function parameters and return types to improve code readability and maintainability.
• Code looks good overall ✅

---

Suggested Tests

import pytest
from your_module import store_embeddings, query_embeddings, client

def test_store_embeddings():
chunks = ["chunk1", "chunk2"]
embeddings = [[1, 2], [3, 4]]
store_embeddings(chunks, embeddings)

def test_query_embeddings():
query_embedding = [1, 2]
results = query_embeddings(query_embedding)
assert isinstance(results, list)

def test_query_embeddings_empty():
collection = client.get_or_create_collection(name="empty")
query_embedding = [1, 2]
results = collection.query(query_embeddings=[query_embedding], n_results=5)
assert results.get("documents", []) == [[]]

def test_query_embeddings_min_length():
query_embedding = [1, 2]
chunks = ["a" * 29, "b" * 31]
embeddings = [[1, 2], [3, 4]]
store_embeddings(chunks, embeddings)
results = query_embeddings(query_embedding)
assert len(results) == 1

def test_query_embeddings_k():
query_embedding = [1, 2]
chunks = ["a" * 31, "b" * 31, "c" * 31]
embeddings = [[1, 2], [3, 4], [5, 6]]
store_embeddings(chunks, embeddings)
results = query_embeddings(query_embedding, k=2)
assert len(results) == 2

[haddybhaiya]

will squash this bugs (hopefully)!