🔍Kapa Launches Company Knowledge Bench with 1,000 Eval Cases
Kapa's new benchmark for company knowledge retrieval
TL;DR
Kapa introduces the Company Knowledge Bench, a platform for indexing and searching company knowledge with 1,000 eval cases from real data. The best score is 0.65 by Kapa Deep, a specialized retriever.
Kapa just launched the Company Knowledge Bench, a platform for indexing and searching company knowledge with 1,000 eval cases from real production data. This is a big deal for teams that rely on company knowledge for their workflows, as it provides a benchmark for retrieval performance. The best score of 0.65 is achieved by Kapa Deep, a specialized retriever designed for the platform. Teams now have a clearer way to measure how well their retrieval systems perform on real data, which is crucial for optimizing internal knowledge management. The platform evaluates retrieval on completeness, minimality, and source preference, making it a more comprehensive benchmark than existing public benchmarks.
Key Points
Kapa's Company Knowledge Bench has 1,000 eval cases from real production data.
The best retrieval score is 0.65 by Kapa Deep, a specialized retriever.
The platform evaluates retrieval on completeness, minimality, and source preference.
Public benchmarks are too narrow or artificial for Kapa's use cases.
Kapa's product covers various use cases, including developers, sales, and support teams.
Why It Matters
If you're managing company knowledge for your team, the Company Knowledge Bench provides a new standard for measuring retrieval performance. Teams can now optimize their systems for real-world queries, improving efficiency and accuracy. The 0.65 score by Kapa Deep sets a benchmark for what's achievable with optimized retrieval.
Comments
Be the first to comment
Enjoyed this article?
Get it daily. 7am. Free. Reads in 5 minutes.
Join 3,567 builders reading daily.