Time
1 year
Location
USA
Sector
Education
Description
The project aimed to develop an AI system capable of answering a wide range of general questions, including open-ended inquiries. The goal was to create a general question-answering engine capable of providing concise and accurate responses to various questions.
Solution
The team designed a system that leverages unstructured data sourced from web pages or a custom text corpus. It applies natural language processing (NLP) techniques to search for and extract relevant information. The system undergoes multiple processing steps, including tokenization and semantic text embedding into a vector search space. One of the existing question-answering models is then applied to find answers. For a given question, the system performs a web search and ranks the best candidate results based on the extracted question context.
Results
The technology demo represents a pioneering achievement, showcasing a general question-answering AI engine with the capability to respond to a wide array of questions at a web scale. The project utilizes a robust infrastructure powered by a cluster of 64 CPUs with 10 GPUs, achieving a processing throughput of 40 questions per second (equivalent to 3.5 million questions per day). The neural network models incorporated into the application comprise 2.2 GB of trained parameters. As of June 2019, the AI system achieves an accuracy rate of 62% in factual question-answering.