Toggle light / dark theme

Introducing Ego-Exo4D: A foundational dataset for research on video learning and multimodal perception

📸 Watch this video on Facebook https://www.facebook.com/share/v/NNeZinMSuGPtQDXL/?mibextid=i5uVoL


Working together as a consortium, FAIR or university partners captured these perspectives with the help of more than 800 skilled participants in the United States, Japan, Colombia, Singapore, India, and Canada. In December, the consortium will open source the data (including more than 1,400 hours of video) and annotations for novel benchmark tasks. Additional details about the datasets can be found in our technical paper. Next year, we plan to host a first public benchmark challenge and release baseline models for ego-exo understanding. Each university partner followed their own formal review processes to establish the standards for collection, management, informed consent, and a license agreement prescribing proper use. Each member also followed the Project Aria Community Research Guidelines. With this release, we aim to provide the tools the broader research community needs to explore ego-exo video, multimodal activity recognition, and beyond.

How Ego-Exo4D works.

Ego-Exo4D focuses on skilled human activities, such as playing sports, music, cooking, dancing, and bike repair. Advances in AI understanding of human skill in video could facilitate many applications. For example, in future augmented reality (AR) systems, a person wearing smart glasses could quickly pick up new skills with a virtual AI coach that guides them through a how-to video; in robot learning, a robot watching people in its environment could acquire new dexterous manipulation skills with less physical experience; in social networks, new communities could form based on how people share their expertise and complementary skills in video.

DeepMind’s New AI can Learn Tasks Directly from HumansDeepMind’s New AI can Learn Tasks Directly from Humans

Google DeepMind has developed an AI agent system that can learn tasks from a human instructor. After enough time, the AI agent can not only imitate the actions of the human instructor but also recall the observed behavior.

In a paper published in Nature, researchers outline a process called cultural transmission to train the AI model without using any pre-collected human data.

This is a new form of imitative learning that the DeepMind researchers contend allows for more efficient transmission of skills to an AI. Think of it like watching a video tutorial – you watch, learn and apply the teachings as well as remember the video’s lessons later on.

AI Revolutionizes Neuron Tracking in Moving Animals

Summary: Researchers developed an AI-based method to track neurons in moving and deforming animals, a significant advancement in neuroscience research. This convolutional neural network (CNN) method overcomes the challenge of tracking brain activity in organisms like worms, whose bodies constantly change shape.

By employing ‘targeted augmentation’, the AI significantly reduces the need for manual image annotation, streamlining the neuron identification process. Tested on the roundworm Caenorhabditis elegans, this technology has not only increased analysis efficiency but also deepened insights into complex neuronal behaviors.

Google is finally launching Gemini, its biggest shot at OpenAI

After months of teasing us, Google is starting to roll out its generative artificial intelligence model, Gemini.

The new model, which will be launched in phases, is Google’s chance to thwart the narrative that it’s fallen behind rivals such as OpenAI.

But while users will have access to Gemini this month, the most advanced version of the model won’t arrive until early next year.

These astonishing biobots can help neurons regrow — but researchers have no idea how

Scientists have created tiny, self-assembling robots made from human cells that could one day repair damaged skin and tissue.

These tiny biological machines, called Anthrobots, are made from human tracheal cells without any genetic modification. Lab dish experiments revealed they can encourage neurons, or nerve cells, to grow in damaged tissue.

Risks of Artificial Intelligence & Shifting Goal Definitions

The development of artificial intelligence poses potential risks to society and requires a shift in goal definitions, consideration of the motivational landscape, and wisdom to prevent self-extinction and promote sustainability.

On this episode, Daniel Schmachtenberger returns to discuss a surprisingly overlooked risk to our global systems and planetary stability: artificial intelligence.

Through a systems perspective, Daniel and Nate piece together the biophysical history that has led humans to this point, heading towards (and beyond) numerous planetary boundaries and facing geopolitical risks all with existential consequences.

Meta AI develops a Non-invasive method to Decode Speech from Brain Activity

Recent technological advancements have opened invaluable opportunities for assisting people who are experiencing impairments or disabilities. For instance, they have enabled the creation of tools to support physical rehabilitation, to practice social skills, and to provide daily assistance with specific tasks.

Researchers at Meta AI recently developed a promising and non-invasive method to decode speech from a person’s brain activity, which could allow people who are unable to speak to relay their thoughts via a computer interface. Their proposed method, presented in Nature Machine Intelligence, merges the use of an imaging technique and machine learning.

“After a stroke, or a brain disease, many patients lose their ability to speak,” Jean Remi King, Research Scientist at Meta, told Medical Xpress. “In the past couple of years, major progress has been achieved to develop a neural prosthesis: a device, typically implanted on the motor cortex of the patients, which can be used, through AI, to control a computer interface. This possibility, however, still requires brain surgery, and is thus not without risks.”