Scholars at Carnegie Mellon University recently offered what they call a “High-Modality Multimodal Transformer,” which combines not just text, image, video, and speech but also database table information and time series data. Lead author Paul Pu Liang and colleagues report that they observed “a crucial scaling behavior” of the 10-mode neural network. “Performance continues to improve with each modality added, and it transfers to entirely new modalities and tasks.”

Scholars Yiyuan Zhang and colleagues at the Multimedia Lab of The Chinese University of Hong Kong boosted the number of modalities to a dozen in their Meta-Transformer. Its point clouds model 3D vision, while its hyper-spectral sensing data represents electromagnetic energy reflected back from the ground to fly-over images of landscapes.

The immediate payoff of multi-modality will simply be to enrich the output of a thing such as ChatGPT in ways that go far beyond the “demo” mode. A children’s storybook, a book with text passages combined with pictures illustrating the text, is one immediate example. By combining the language and image attributes, the kinds of pictures created by the diffusion process can be more subtly controlled from picture to picture.

Following the successful launch of Spotify’s AI-powered DJ feature and, more recently, added support for AI-translated podcasts, Spotify now appears to be developing another means of using AI in its app: AI-powered playlists. References discovered in the app’s code indicate the company may be developing generative AI playlists users could create using prompts.

The additions were uncovered by tech veteran-turned-investor Chris Messina, who posted screenshots of code in Spotify’s app that refer to “AI playlists” and “playlists based on your prompts.” He theorized that creating these may be an option within the Blend genre, where the tastes of different users are mixed to create a playlist with songs that everyone likes.

Your photos and words trained Meta’s AI.

Once considered a forbidden topic in the AI community, discussions around the concept of AI consciousness are now taking center stage, marking a significant shift since the current AI resurgence began over a decade ago. For example, last year, Brad Lemoine, an engineer at Google, made headlines claiming the large language model he was developing had become sentient [1]. CEOs of tech companies are now openly asked in media interviews whether they think their AI systems will ever become conscious [2,3].

Unfortunately, missing from much of the public discussion is a clear understanding of prior work on consciousness. In particular, in media interviews, engineers, AI researchers, and tech executives often implicitly define consciousness in different ways and do not have a clear sense of the philosophical difficulties surrounding consciousness or their relevance for the AI consciousness debate. Others have a hard time understanding why the possibility of AI consciousness is at all interesting relative to other problems, like the AI alignment issue.

This brief introduction is aimed at those working within the AI community who are interested in AI consciousness, but may not know much about the philosophical and scientific work behind consciousness generally or the topic of AI consciousness in particular. The aim here is to highlight key definitions and ideas from philosophy and science relevant for the debates on AI consciousness in a concise way with minimal jargon.