Google has developed a new tool called DataGemma that taps into a vast database called Data Commons, providing researchers with a powerful resource to enhance their AI capabilities. While currently available only to researchers, there are plans to expand access after more testing.
However, there are some limitations to keep in mind. The Data Commons acts more as a data repository than an encyclopedia, which means its usefulness depends on whether the relevant information is available. While it can provide data like the GDP of Iran, it may struggle with specific details like historical events or the release dates of songs. In fact, Google’s researchers found that the RIG method, used by DataGemma, was unable to retrieve usable data from the Data Commons for about 75% of test questions. And even when the data is available, the model may not always ask the right questions to extract it.
Accuracy is another key consideration. Testing revealed that the RAG method produced incorrect answers 6% to 20% of the time, while the RIG method accurately retrieved data from the Data Commons only around 58% of the time. Despite these limitations, it represents a significant improvement over Google’s large language models, which typically have an accuracy rate of 5% to 17% when not connected to Data Commons.
Google’s VP, Reena Ramaswami, is optimistic about DataGemma’s future accuracy, citing plans to train the model on millions of questions to further refine its capabilities. The current version has been trained on just 700 questions, with each fact meticulously verified by the team. This commitment to data refinement is expected to greatly enhance the tool’s accuracy and utility in the future.