At the beginning of my research career around 15 years ago, any suggestion that a bee, or any invertebrate, had a mind of its own or that it could experience the world in an intricate and multifaceted way would be met with ridicule. As Lars Chittka points out in the opening chapters of “The Mind of a Bee,” the attribution of human emotions and experiences was seen as naivety and ignorance; anthropomorphism was a dirty word.

Pet owners eagerly ascribe emotions to their animals, but the simple brain of a bee surely could not experience the rich tapestry that is our existence. They are far too simplistic and robotic, right?

Lars Chittka has been researching honeybees for the past 30 years. “The Mind of a Bee” is a collection of his research stories. It also covers the influential figures in bee research and provides a historical perspective on the research that much behavioral work is built on today.

Language is the most intuitive way for us to express how we feel and what we want. However, despite recent advancements in artificial intelligence, it is still very hard to control a robot using natural language instructions. Free-form commands such as “Robot, please go a little slower when you pass close to my TV” or “Stay far away from the swimming pool!” are hard to parse into actionable robot behaviors, and most human-robot interfaces today still rely on complex strategies such directly programming cost functions which define the desired behavior.

With our latest work, we attempt to change this reality through the introduction of “LaTTe: Language Trajectory Transformer”. LaTTe is a deep machine learning model that lets us send language commands to robots in an intuitive way with ease. When given an input sentence by the user, the model fuses it with camera images of objects that the robot observes in its surroundings, and outputs the desired robot behavior.

As an example, think of a user trying to control a robot barista that’s moving a wine bottle. Our method allows a non-technical user to control the robot’s behavior only using words, in a natural and simple interface. We will explain how we can achieve this in detail through this post.

face_with_colon_three circa 2020.

Watch any kid play with a tape measure and you’ll understand where researchers from the Seoul National University of Science and Technology found the inspiration for their new chameleon-like robot that can snag objects with its artificial tongue over 30 inches away in less than 600 milliseconds.

With the advent of generative image AI, a new discipline is emerging: prompt engineering. Search engines like Krea provide inspiration.

Prompts are the short text descriptions that an image AI like DALL-E 2, Midjourney or Stable Diffusion uses to generate an image.

Often it is difficult to put the idea you have in mind into fitting words. Moreover, the AI may execute instructions differently than you want it to. And there are plenty of modifiers, for example for image cropping or style, that can lead to very different results for the otherwise same prompt.

ERNIE-ViLG, the new text-to-image AI developed by Baidu can generate images that show Chinese objects and celebrities more accurately than existing AIs. But a built-in censorship mechanism will filter out politically sensitive words.

Advancing the frontiers of machine learning.