Age-inappropriate AI interaction

What it is

AI features are added to products that children already use, or are open to anyone who says they are old enough. A general-purpose assistant will discuss adult subjects, give confident guidance on health or relationships, and hold long personal conversations. Children are more trusting of such systems, less able to judge what they are told, and more open to forming attachments. UK law expects online products likely to be accessed by children to be designed with their interests in mind and to protect them from harmful content, whether or not children were the intended audience.

How it happens

• No assessment was made of whether children are likely to use the feature.
• Age is self-declared at sign-up and never checked.
• The same model and settings serve all users.
• Content filters were tuned for adult norms.

Why it matters

Exposure to unsuitable content, poor guidance or manipulative design can do real harm to a child. Regulators in data protection and online safety both treat children's use as a priority, and public reaction to failures is severe.

Warning signs

• The product has a young user base and no age-appropriate mode.
• The impact assessment does not mention children.
• Conversations show signs of school-age users.
• Parental complaints have been received.