For 15 years, we have let children bring their cell phones into most of our classrooms. By the time states began ordering phones, a generation had spent their childhoods inundated with screens. We had protected everything in the arrangement except the children.
Similarly, during the rise of the commercial Internet, we wrote rules to protect the privacy of platforms and the people who profit from them. The protections we then built for children, like the rule that a website must be permission from a parent Before collecting a child’s data, we could never have made up the ground we had already given up.
As a subject matter expert on human trafficking and child labor exploitation at the U.S. Department of Education, I observed systems that were built to protect children, instead protecting the privacy of people who harmed them. This failure taught me that the safeguards we neglect to build at the start are protections we permanently lose.
We are now watching AI companies rush to put in place enforceable rules protecting their interests. We’ve seen it before: if the trend continues, the industry protections that set precedent will not be dislodged by the child protections that follow. This time, the protection of children must come first.
I believe that no AI tool that teaches, assesses, or tracks a child should be eligible for a school district contract until it passes a bias test built around the same students that digital tools have previously harmed.
Related: There’s a lot going on in classrooms from kindergarten through high school. Follow our free weekly K-12 education newsletter.
Last year, a study AI assistants used by teachers found that they recommended harsher approaches to struggling students whose names were black. A separate study found that an AI evaluator gave black students’ essays lower grades than Asian students’ essays, replicating a gap already present in human grading.
None of these prejudices should reach a child, because districts already have the perfect place to catch them. Most districts already do a privacy check before purchasing technology: They check whether a digital tool will retain a child’s data. private and secure; a tool that fails is not supposed to be purchased. Bias testing would work within this same control. To test a tool for bias, an evaluator gives it nearly identical student work, changing only the name of each sample as well as the background and speech patterns that the writing itself signals, and compares how the tool responds. An evaluator also examines what the tool teaches students about people: what stereotypes it repeats and what story it tells in its entirety.
A tool that fails either exam should lose its eligibility. What makes a tool safe for children most at risk, makes it safe for all children.
Nearly every state has moved to regulate AI in some way, and a few have enacted chatbot laws aimed at protecting children. California need a chatbot to tell a child that they are using AI and to remind that child to take regular breaks. New York State need chatbots to detect expressions of suicidal thoughts and direct the user to crisis services.
These are positive changes, but the child protected when using AI at home does not benefit from comparable protections at school. Only a few states have passed laws order their educational agencies and school districts to manage AI tools. In Oklahomafor example, a teacher must review any content generated by an AI tool before it reaches a classroom, and no AI tool can become the primary basis for a grade, promotion, or retention decision.
Oklahoma law goes further than most. Typically, when one of these laws is passed, it is just a preliminary directive directing agencies and districts to develop their own policies.
This month, New York City prohibits students up to eighth grade from using generative AI, and Los Angeles Unified blocked all students from using generative AI on district devices, including AI assistants built into Google Classroom, which nearly all U.S. schools use. They banned what they couldn’t test. New York City’s ban lasts for one year and the city will spend that year reviewing all technology tools used in its schools. This is the review that each district should be able to conduct before a tool reaches a student. A ban saves time, but does not tell the district which tools are safe, and it removes useful tools as well as harmful tools.
Even the most specific law will not be enough to protect a child. A law is only a promise before being applied. The school district must keep its promise by examining each tool before it is purchased and again while children are using it. Even without new laws, every district has a legal obligation to care for every child, and yet almost no district has the staff, training, or technology to fulfill this obligation.
The privacy checks most school districts conduct today don’t test tools for bias at all: No state need districts to do so, and there is no law requiring businesses to do so. disclose biases found in their AI.
But these checks, which only ask whether a tool protects a child’s data, cannot determine whether a tool built on a large language model is safe for a child. This underlying AI model can treat a child differently even when race is never named, because it can infer race from the child’s name, neighborhood, school, language spoken at home, or writing style. These are the most obvious clues; A standard checklist is unlikely to identify the more subtle ones. Facing public scrutiny, the companies that built these models made this bias harder to detect. The companies trained the models to stop saying explicitly biased things, but did not remove the biases themselves.
Related: AI gives more praise and less criticism to black students
When researchers gave AI models samples of writing in the dialect that many Black Americans speak at home, the models said nothing negative about Black people, but judged the writers themselves as less intelligent and associated them with less prestigious works. This is why no tool should retain its eligibility without being retested while students use it. These systems change after they are sold.
Even if a privacy check cannot detect this bias, until recently, a protection remained after the check: a biased tool could always be removed from schools if statistical evidence showed that it harmed one group of children more than another. The US Department of Education recently removed the rule who made this possible. Now its Office for Civil Rights will only act when someone can prove a school or district intended to discriminate. It will no longer be enough to prove that children of one race are harmed more than another, and the tool will remain in the classroom.
No one wants these tools to be biased: models absorb it on the Internet from which they learn. Harm only appears in patterns where children receive less teaching, are graded lower, and are seen as struggling.
Again, everything about the arrangement is protected except for the children.
Shauna DA Knox is the founder and CEO of The Emancipation Group, an innovation studio that combines research and development with digital product development to ensure Black children and families achieve authorship and ownership in the innovation economy.
Contact the opinion editor at opinion@hechingerreport.org.
This story about protecting children from AI was produced by The Hechinger reportan independent, nonprofit news organization focused on inequality and innovation in education. Register with Hechinger weekly newsletter.
The article OPINION: Our children need more protection from the AI that teaches them appeared first on The Hechinger Report.
