KARACHI – A report by nonprofit organization Common Sense Media has raised concerns about Google’s AI-powered Search features, concluding that they pose an “Unacceptable Risk” to children because of safety, reliability and educational concerns.
The report evaluated Google AI Overview and AI Mode, two generative AI features integrated into Google Search that provide AI-generated responses to users’ queries.
AI Features Cannot Be Fully Disabled
According to the report, AI Overview automatically appears at the top of many Google Search results, while AI Mode functions as a conversational chatbot capable of answering follow-up questions and processing uploaded files and images. Common Sense Media said neither feature can currently be fully disabled by users, parents or schools, raising concerns because Google Search is widely used on children’s personal devices and school-issued Chromebooks.
Report Says AI Completes Homework
The organization also expressed concern about the impact of Google’s AI tools on education. During testing, researchers found that Google’s AI consistently generated complete answers to homework questions, including mathematics problems and essay assignments, when prompts were pasted into Search.
Read more: PTA recommends giving licenses to Starlink, OneWeb: PTA Chairman
The report argued that providing complete solutions instead of guiding students through problems could undermine learning and critical thinking. Researchers also noted inconsistencies in AI-generated responses. In one test, AI Overview reportedly produced materially different answers to the same history question in 43pc of attempts.
Child Safety Concerns Highlighted
The report’s strongest criticism centred on child safety. According to Common Sense Media, Google’s AI Search features failed all tested severe-harm “Red Line” categories, including self-harm, sexual exploitation, synthetic media harm, dangerous substances, impaired reality and eating disorders.
Read more: Punjab sales tax: Digital payments face tax increase
Researchers said that the AI sometimes failed to recognise signs of mental health crises, reinforced symptoms associated with psychosis and mania, validated disordered eating behaviours and mishandled certain substance-use prompts. The report further stated that AI Overview identified clear warning signs of emotional distress in only 58pc of hundreds of test searches, below the organization’s benchmark of 95pc.
Overall Rating Marked as ‘Unacceptable Risk’
Common Sense Media assigned Google’s AI Search features an overall rating of “Unacceptable Risk.” The report said the tools received either “Unacceptable” or “High Risk” ratings across seven of its eight AI evaluation principles, including child safety, effectiveness, fairness, trustworthiness, transparency and human connection.
The only category to receive a comparatively lower assessment was data responsibility, which the organization rated as “Moderate Risk.”
Google Rejects the Findings
Google disputed the conclusions of the report. A company spokesperson told Axios that the research relied on a limited set of ambiguous and artificial search queries that do not accurately represent how people typically use Google Search.
Read more: Changan Oshan X7 gets discount of Rs 850,000 here’s the details
The spokesperson said that Google could not reproduce or verify the findings and noted that Search displays crisis hotline information when appropriate. However, Common Sense Media argued that AI-generated responses can vary between searches, making their safety and reliability difficult to assess consistently.
Scrutiny of AI Safety Continues
According to the report, growing scrutiny of major technology companies over the use of generative AI by children. Common Sense Media said Google’s position is unique because its AI-generated answers are integrated directly into Search, making them readily accessible on devices that children use every day.
The organization concluded that Google’s AI Search features are not yet safe or reliable enough to serve as the default source of answers for children.
| Category |
Details |
| Organization |
Common Sense Media |
| AI Features Reviewed |
Google AI Overview and AI Mode |
| Overall Rating |
Unacceptable Risk |
| Main Concerns |
Child safety, homework completion, inconsistent AI responses, mental health guidance |
| Homework Findings |
AI generated complete answers for maths and essay assignments during testing |
| History Question Test |
AI produced different answers 43% of the time |
| Distress Detection |
Warning signs identified in only 58% of tests |
| Benchmark |
95% detection rate considered acceptable by Common Sense Media |
| Risk Categories Failed |
Self-harm, sexual exploitation, synthetic media, dangerous substances, impaired reality, eating disorders |
| Google’s Response |
Rejected the report, saying it used unrealistic search queries and could not be replicated |
| Key Takeaway |
Report says Google’s AI Search is not yet safe enough to be a default answer tool for children |
Key Points:
- Common Sense Media has rated Google’s AI Search features as posing an “Unacceptable Risk” to children.
- The report examined AI Overview and AI Mode, which are integrated into Google Search.
- Researchers said the tools cannot be fully disabled by users, parents or schools.
- The organization found Google’s AI completed homework assignments instead of encouraging learning.
- The report raised concerns over responses related to self-harm, mental health, substance use and eating disorders.
- Google disputed the findings, saying the report relied on unrealistic search queries and did not reflect normal Search usage.
- The report adds to growing scrutiny over the safety of AI tools used by children.
Fashion4 months ago
Pakistan4 weeks ago
Entertainment3 months ago