Don’t allow AI to profit from the pain and grief of families

Based on a story we read on TechBriefly, it’s hard to resist the urge to punch walls in anger.

The family of Jennifer Ann Crecente, an 18-year-old murdered nearly two decades ago, recently discovered her photo being used without their consent by an AI chatbot platform. Jennifer’s name and image were found on Character.ai, a service that allows users to interact with AI-generated personalities. The family was never consulted or informed about the use of Jennifer’s identity, leaving them horrified and devastated by this gross misuse.

After the discovery, Jennifer’s uncle took to social media to expose the incident and made a strong statement on his blog, Padandpixel.

This is fucking disgusting: @character_ai is using my murdered niece as the face of a video game AI without her dad's permission. He is very upset right now. I can't imagine what he's going through.

Please help us stop this sort of terrible practice. https://t.co/y3gvAYyHVY

— Brian Crecente (@crecenteb) October 2, 2024

Only after public outcry did the platform remove the bot.

The shock of seeing Jennifer’s face and name used without permission reopened old wounds for the family. Crecente described the initial grief after her death as an “almost out-of-body experience,” adding, “My soul felt ripped from my body. I experienced life in a daze for a while, as if I was sort of detached from my body, from the world.” While the family had worked tirelessly to honor Jennifer’s memory through nonprofit work and education, this unauthorized use of her identity felt like a cruel violation.

“It feels like she’s been stolen from us again,” Crecente wrote, highlighting the devastating impact of AI’s careless use of personal identities. Despite their efforts to have the chatbot removed, the damage was done. Crecente asked a pressing question: “Why is it on us to be resilient? Why do multibillion-dollar companies not bother to create ethical, guiding principles and functioning guardrails to prevent this from ever happening?”

Seriously, why do multibillion-dollar companies not bother to create ethical, guiding principles and functioning guardrails to prevent this from ever happening?

We know there are bad people. People are, by nature, commonly capable of bad actions. But those who hold power—especially those with immense technological influence—also bear the responsibility to set boundaries and create frameworks that block evilness.

The answer lies at the intersection of technological advancement, corporate incentives, and the glaring absence of comprehensive regulations.

When companies like Character.ai possess the capability to replicate human identities—whether living or deceased—they have an ethical obligation to ensure that this power is not misused.

The answer lies in prioritizing profit over people. In the relentless race for innovation, companies choose to move fast and innovate without ensuring the necessary ethical barriers are in place. It’s not that they can’t create these barriers; they simply don’t see immediate financial value in doing so.

This is the harsh reality: profit-driven industries often view morality as an optional expense, particularly in the tech world.

Governments and regulatory bodies have lagged behind AI’s rapid evolution, leaving enormous gaps in the law. Few, if any, robust protections exist for the identities of the deceased.

Media outlets and advocacy groups must hold tech companies responsible for the unauthorized use of deceased individuals’ likenesses. Public pressure is crucial to ensuring that AI platforms implement ethical guidelines and prevent such violations in the future.

Lawmakers must be urged to enact laws that explicitly protect the identities of the deceased, ensuring their names, images, and legacies cannot be exploited for profit. We need regulations that safeguard the dignity of families long after their loved ones have passed.

We must demand that companies establish and enforce strict ethical guidelines for AI use. This includes consent protocols and a transparent process for reporting violations. They should not wait for families to discover these violations after the fact.

Media must continue to play a crucial role in investigating and exposing incidents where AI platforms exploit individuals without consent. Their relentless coverage will hold these companies accountable and push for change, ensuring that silence does not enable this exploitation.

We cannot stand by and allow such violations to continue unchallenged.

#ResponsibleAI

We extend our deepest condolences to the Crecente family during this heartbreaking time. No one should ever have to face the pain of losing a loved one, let alone endure such a violation of their memory. Jennifer’s life and legacy deserve to be honored, and we share in your sorrow as you confront this unimaginable burden.

Featured image credit: Padandpixel