Lifeboat Foundation

The pretraining of BERT-type large language models — which can scale up to billions of parameters — is crucial for obtaining state-of-the-art performance on many natural language processing (NLP) tasks. This pretraining process however is expensive, and has become a bottleneck hindering the industrial application of such large language models.

In the new paper Token Dropping for Efficient BERT Pretraining, a research team from Google, New York University, and the University of Maryland proposes a simple but effective “token dropping” technique that significantly reduces the pretraining cost of transformer models such as BERT, without degrading performance on downstream fine-tuning tasks.

The team summarizes their main contributions as:

To effectively interact with humans in crowded social settings, such as malls, hospitals, and other public spaces, robots should be able to actively participate in both group and one-to-one interactions. Most existing robots, however, have been found to perform much better when communicating with individual users than with groups of conversing humans.

Hooman Hedayati and Daniel Szafir, two researchers at University of North Carolina at Chapel Hill, have recently developed a new data-driven technique that could improve how robots communicate with groups of humans. This method, presented in a paper presented at the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘22), allows robots to predict the positions of humans in conversational groups, so that they do not mistakenly ignore a person when their sensors are fully or partly obstructed.

Continue reading “A new approach that could improve how robots interact in conversational groups” »

O,.o! Circa 2020

The discovery expands the definition of what an “animal” can be, researchers say.

Circa 2004

Internal antioxidants may shield cells from radiation damage.

A new tool speeds up development of vaccines and other pharmaceutical products by more than 1 million times while minimizing costs.

In search of pharmaceutical agents such as new vaccines, industry will routinely scan thousands of related candidate molecules. A novel technique allows this to take place on the nano scale, minimizing use of materials and energy. The work is published in the journal Nature Chemistry.

More than 40,000 molecules can be synthesized and analyzed within an area smaller than a pinhead. The method, developed through a highly interdisciplinary research effort in Denmark, promises to drastically reduce the amounts of material, energy, and economic cost for pharmaceutical companies.

ab initio calculations

Classical computing is of very little help when the task to be accomplished pertains to ab initio calculations. With quantum computing in place, you have a quantum system simulating another quantum system. Furthermore, tasks such as modelling atomic bonding or estimating electron orbital overlaps can be done much more precisely.

Adrenalin drivers seemingly go haywire.

German publication Igor’s Lab investigated a potential abnormality surrounding AMD’s Adrenalin GPU software. The current speculation is that the integration of AMD’s Ryzen Master module into the Adrenalin version 22.3.1 software or later has inadvertently allowed the program to manipulate CPU PBO and Precision Boost settings without the user’s permission.

According to user feedback, the problem occurs only with AMD CPU and GPU combinations since that hardware supports the Ryzen Master SDK. So if you are running a system that has either an Intel CPU paired to a Radeon GPU, or a Ryzen CPU paired to an Nvidia GPU, you will not encounter this problem.

Axiom Space’s first mission plans a hefty science agenda during eight days on the International Space Station.

Houston company Axiom Space has a huge science haul planned for its debut mission, including robots and filters that could assist with future space exploration at the moon or Mars.

Starlink is a space internet project run by aerospace company SpaceX.