Google to manually disable AI Overviews for specific searches

Social media is abuzz with examples of Google’s new AI Overview product providing bizarre and incorrect suggestions, from recommending users put glue on their pizza to suggesting they eat rocks. The messy rollout has led Google to manually disable AI Overviews for specific searches as various memes highlighting these errors spread across social networks. This frantic response is due to the significant public attention and criticism these mistakes have garnered.

The odd situation arises despite Google having tested AI Overviews for a year. The feature launched in beta in May 2023 as the Search Generative Experience, and according to CEO Sundar Pichai, the company served over a billion queries during this period. Pichai also mentioned that Google reduced the cost of delivering AI answers by 80 percent through hardware, engineering, and technical advancements. However, it appears that such optimization might have been premature, implemented before the technology was fully mature and ready for wide-scale deployment.

“A company once known for being at the cutting edge and shipping high-quality stuff is now known for low-quality output that’s getting meme’d,” an anonymous AI founder told The Verge, highlighting the reputational shift Google is experiencing due to these issues.

Google maintains that its AI Overview product generally delivers "high quality information" to users. “Many of the examples we’ve seen have been uncommon queries, and we’ve also seen examples that were doctored or that we couldn’t reproduce,” Google spokesperson Meghann Farnsworth said. She added that the company is "taking swift action" to remove AI Overviews on certain queries "where appropriate under our content policies" and is using these examples to develop broader improvements to their systems, some of which have already started to roll out.

Gary Marcus, an AI expert and an emeritus professor of neural science at New York University, commented that many AI companies are "selling dreams" that this technology will improve from being 80 percent correct to 100 percent. Achieving the initial 80 percent correctness is relatively straightforward, as it involves approximating a large amount of human data, Marcus explained. However, the final 20 percent, which involves ensuring the information is plausible and verifying sources as legitimate, is extremely challenging and might require artificial general intelligence (AGI). Both Marcus and Meta’s AI chief Yann LeCun agree that the large language models powering current AI systems, such as Google’s Gemini and OpenAI’s GPT-4, will not be what creates AGI.

Google is in a challenging position. Bing took the lead on AI integration with Satya Nadella’s famous “we made them dance” quote, OpenAI is reportedly working on its own search engine, and a fresh AI search startup is already valued at $1 billion. Moreover, a younger generation of users who prioritize the best experience is increasingly turning to platforms like TikTok for search needs. This competitive pressure contributes to the rushed and error-prone rollout of AI Overviews.

Marcus pointed out a similar situation in 2022 when Meta released an AI system called Galactica, which had to be taken down shortly after its launch due to errors, including advising people to eat glass. This underscores the recurring challenge within the industry where AI systems are prematurely released, leading to significant issues.

Google has ambitious plans for AI Overviews. The current features are just a fraction of what the company announced recently. Future enhancements include multistep reasoning for complex queries, the ability to generate AI-organized results pages, and video search capabilities in Google Lens. However, the company’s immediate challenge is to ensure the basic functionalities work correctly, as current errors significantly impact its reputation.

“These models are constitutionally incapable of doing sanity checking on their own work, and that’s what’s come to bite this industry in the behind,” Marcus noted, summarizing the fundamental challenge facing AI development and deployment. 

Ultimately, while Google’s vision for AI Overviews is grand and ambitious, the immediate priority must be on refining the product to avoid these glaring errors and restoring trust among its users. The tech giant's ability to effectively address these issues will determine its success in the highly competitive AI-driven search market.

Google to manually disable AI Overviews for specific searches as various memes highlighting these errors spread across social networks