Commit Graph

11 Commits

Author SHA1 Message Date
pseco 1ee5f21c7c Add BEIR analysis notebooks and evaluation pipeline for embedding models
- Created `n00 Beir Analysis_cosqa.ipynb` for analyzing CoSQA dataset with BEIR.
- Created `n00 first Analysis.ipynb` for initial analysis with embeddings.
- Implemented `evaluate_embeddings_pipeline.py` to evaluate embedding models across CodexGlue, CoSQA, and SciFact benchmarks.
- Added adapters for Ollama and HuggingFace embeddings to ensure compatibility with BEIR.
- Enhanced error handling and data normalization in embedding processes.
- Included functionality to load datasets from local cache or download if not present.
2026-03-26 09:37:37 +01:00
pseco d7f895804c Refactor code structure for improved readability and maintainability 2026-03-25 10:53:38 +01:00
pseco 71eb85cc89 Merge branch 'mrh-online-dev' of github.com:BRUNIX-AI/assistance-engine into mrh-online-dev 2026-03-25 10:50:13 +01:00
acano b2e5d06d96 Merge branch 'mrh-online-dev' of github.com:BRUNIX-AI/assistance-engine into mrh-online-dev 2026-03-25 10:41:26 +01:00
acano 21bc6fc3f0 feat: add embedding evaluation results and task processing notebook 2026-03-25 10:40:49 +01:00
pseco 9b1a0e54d5 updated pipeline to only download files when missing 2026-03-25 10:06:07 +01:00
pseco 2a33f8eb06 bge-m3 and qwen3-emb comparison 2026-03-23 15:59:44 +01:00
pseco b574517340 working on ADR0005 2026-03-23 13:17:50 +01:00
pseco 185ea276b7 updated cosqa notebook 2026-03-23 10:36:10 +01:00
acano 27cfbaf257 Refactor code structure for improved readability and maintainability 2026-03-19 16:37:42 +01:00
acano dd3bde2ec9 Add BEIR analysis notebooks for different datasets and models
- Created `n00 Beir Analysis.ipynb` for analyzing BEIR dataset with Ollama embeddings.
- Added `n00 Beir Analysis_cosqa.ipynb` for evaluating the CosQA dataset using similar embedding techniques.
- Introduced `n00 first Analysis.ipynb` for initial analysis with Ragas embeddings and semantic similarity evaluation.
- Implemented data loading and processing for each notebook, including downloading datasets and saving results.
- Included evaluation metrics such as NDCG, MAP, Recall, and Precision for model performance assessment.
2026-03-19 16:24:34 +01:00