As it becomes easier to create hyperrealistic digital characters using artificial intelligence, much of the conversation around these tools has focused on deceptive and potentially dangerous deepfake content. But technology can also be used for positive purposes – to revive Albert Einstein to teach a physics class, to talk about a career change with your older adult, or to anonymize people while preserving facial communication.
To encourage the positive possibilities of technology, researchers at MIT Media Lab and their collaborators at the University of California at Santa Barbara and the University of Osaka have compiled an open-source and easy-to-use character generation pipeline that combines AI models for facial gestures, voice, and movement and can be used to create a variety of audio and video outputs.
The pipeline also marks the resulting output with a traceable, human-readable watermark to distinguish it from genuine video content and to show how it was generated – an addition to help prevent its malicious use.
By making this pipeline easily accessible, the researchers hope to inspire teachers, students and healthcare workers to explore how these tools can help them in their respective fields. If more students, educators, healthcare workers and therapists are given the opportunity to create and use these characters, the results could improve health and well-being and contribute to personalized education, write the researchers. researchers in Nature Machine Intelligence.
“It will be a strange world when AIs and humans start to share identities. This article does an incredible job of thought leadership, mapping the space of what’s possible with AI-generated characters in areas ranging from education to health to close relationships, while also giving a leaf a tangible guide on how to avoid ethical challenges related to privacy and misrepresentation. “said Jeremy Bailenson, founding director of the Stanford Virtual Human Interaction Lab, which was not associated with the study.
Although the world is primarily familiar with deepfake technology, “we see its potential as a tool for creative expression,” says lead author Pat Pataranutaporn, doctoral student in the Fluid Interfaces research group of Pattie Maes, professor. of media technology.
Other authors on the paper include Maes; Valdemar Danry, master’s student Fluid Interfaces and Joanne Leong, doctoral student; Media Lab research scientist Dan Novy; Assistant Professor at Osaka University Parinya Punpongsanon; and the University of California at Santa Barbara, assistant professor Misha Sra.
Deeper truths and deeper learning
Generative Antagonist Networks, or GANs, a combination of two competing neural networks, have facilitated the creation of photorealistic images, voice clones, and animated faces. Pataranutaporn, along with Danry, first explored its possibilities in a project called Machinoia, where he generated multiple alternative representations of himself – as a child, as an old man, as a woman – to have a self-dialogue of life choices from different angles. The unusual experience of deepfaking made him aware of his “journey as a person,” he says. “It was a deep truth – to discover something about yourself that you had never thought of before, using your own data about yourself.”
Self-exploration is just one of the positive applications of AI-generated characters, the researchers say. Experiences show, for example, that these traits can make students more enthusiastic about learning and improve the performance of cognitive tasks. Technology provides a way for instruction to be “personalized to suit your interests, your idols, your background and can be changed over time,” says Pataranutaporn, in addition to traditional instruction.
For example, researchers at MIT used their pipeline to create a synthetic version of Johann Sebastian Bach, who had a live conversation with acclaimed cellist Yo Yo Ma in the Musical Interfaces class of Professor Tod Machover of the Media Lab – to the delight of the students and Ma.
Other applications could include characters who help deliver therapy, address a growing shortage of mental health professionals, and meet the estimate. 44 percent Americans with mental health issues who never receive counseling, or AI-generated content that offers exposure therapy to people with social anxiety. In a related use case, the technology can be used to anonymize faces in video while preserving facial expressions and emotions, which can be useful for sessions where people want to share sensitive personal information such as health and trauma experiences, or for whistleblowers and testimonials. .
But there are also more artistic and playful use cases. In this fall Experiences in Deepfakes Class, led by Maes and research affiliate Roy Shilkrot, students used technology to animate characters in a historical Chinese painting and to create a dating “break-up simulator”, among other projects.
Legal and ethical challenges
Many applications of AI-generated characters raise legal and ethical issues that need to be discussed as technology evolves, the researchers note in their article. For example, how will we decide who has the right to digitally recreate a historical figure? Who is legally responsible if an AI clone of a famous person encourages harmful behavior online? And is there a risk that we would prefer to interact with synthetic characters rather than humans?
“One of our goals with this research is to raise awareness of what is possible, to ask questions and to initiate public conversations about how this technology can be used ethically for societal benefit. What technical, legal, policy and educational actions can we take to promote positive use cases while reducing the risk of harm? Maes says.
By widely sharing the technology, while clearly calling it synthesized, Pataranutaporn says, “We hope to stimulate more creative and positive use cases, while educating people about the potential advantages and disadvantages of the technology.”