The future of Computational Intelligence is Social AI: systems that can sense, understand and respond to people, not just one user at a time but in groups, in real time and in shared physical space. Researchers Mathur, Liang and Morency describe it as building agents that can perceive and respond to the affect, behavior and cognition of other agents, human or artificial.
That definition covers a lot of ground. To make it concrete, we group Social AI by the type of interaction it has to master.
1. Presentation (1 → N)
One person addressing many: talks, lectures, panels and webinars. Here, the hard problems are reading an audience and handling questions. Today's lecture assistants and live captioning tools mostly track the speaker; the next generation will need to read the room as well. Data needed: recordings of presenters and their audiences together, including reactions and Q&A.
2. Conversation (N ↔ M)
Many people talking with each other. Examples include full-duplex voice models like Kyutai's Moshi, which can handle interruptions in about 200 milliseconds, and Meta's Seamless Interaction avatars, which learned to nod and smile while listening from more than 4,000 hours of recorded conversation. AI meeting assistants live here too, along with the consent questions now facing them in court. Data needed: multi-speaker recordings with separate tracks, overlaps and backchannels labeled.
3. Physical AI (human ↔ human, human ↔ robot)
Interaction in shared physical space. Diligent Robotics' Moxi navigates busy hospital corridors in more than 25 U.S. hospitals. Agility's Digit became the first humanoid with a formal commercial job, at GXO, though still kept apart from human workers. Waymo's robotaxis negotiate crosswalks with pedestrians every day. Data needed: synchronized, multi-viewpoint recordings of people coordinating physically with each other and with machines.
4. Social settings (groups in context)
People together in natural environments, where roles and norms shape everything. DeepMind's Habermas Machine helped groups of people find common ground in an experiment with more than 5,000 participants. ElliQ, an AI companion for older adults, was placed in hundreds of New York homes in a state program. Data needed: recordings of real gatherings, workshops and encounters, with group dynamics annotated.
What connects them
Every example above runs into the same constraint: data. The open web is full of text and solo video, but it holds very little consented, multi-party, multimodal recording of people actually interacting. The largest open research corpora together add up to under 10,000 hours.
That is why the next decade of Social AI will be paced by access to the right data, collected with consent from everyone in it.
Crosstalk is the data provider for Social AI, sourcing consented interaction data across all four types. Explore the catalog and read the full argument in our paper, The Future Economics of Social AI.
Sources: Mathur, Liang and Morency, EMNLP 2024, plus the sources in each linked article.