Time
6 months
Location
US
Sector
IT startup
Description
The project’s objective was to implement a system capable of answering general questions based on online data, not only fetching relevant answers but doing so in an optimized time frame. The challenge involved collecting data from the internet, integrating it with large language models, and installing a scalable architecture for inference. The goal was to improve the system’s response time from 20 seconds per question to just 100 to 200 milliseconds.
Solution
To tackle this ambitious challenge, a comprehensive pipeline was established for data collection from the internet. This data was then seamlessly integrated with advanced language models. The team developed a proprietary technology based on a semantic index to enhance the relevancy of the extracted texts and implemented a scalable, GPU-based architecture to support efficient inference. Additionally, a system for dataset management, discovery, and methodologies to validate system results was created.
Results
The project resulted in the development of a high-performing question answering system that not only met but exceeded the performance of prominent virtual assistants like Google Assistant and Siri, according to internal testing. This outstanding success garnered an endorsement from Norman Wienarski, co-founder of Siri, and led to a substantial investment of $600,000 in the customer’s project for further development and research in this field