Lifeboat Foundation: Safeguarding Humanity
Toggle light / dark theme

Blog – Page 5396

We study the question of how to decompose Hilbert space into a preferred tensor-product factorization without any pre-existing structure other than a Hamiltonian operator, in particular the case of a bipartite decomposition into “system” and “environment.” Such a decomposition can be defined by looking for subsystems that exhibit quasi-classical behavior. The correct decomposition is one in which pointer states of the system are relatively robust against environmental monitoring (their entanglement with the environment does not continually and dramatically increase) and remain localized around approximately-classical trajectories. We present an in-principle algorithm for finding such a decomposition by minimizing a combination of entanglement growth and internal spreading of the system. Both of these properties are related to locality in different ways.

Read more | >

In recent years, large neural networks trained for language understanding and generation have achieved impressive results across a wide range of tasks. GPT-3 first showed that large language models (LLMs) can be used for few-shot learning and can achieve impressive results without large-scale task-specific data collection or model parameter updating. More recent LLMs, such as GLaM, LaMDA, Gopher, and Megatron-Turing NLG, achieved state-of-the-art few-shot results on many tasks by scaling model size, using sparsely activated modules, and training on larger datasets from more diverse sources. Yet much work remains in understanding the capabilities that emerge with few-shot learning as we push the limits of model scale.

Last year Google Research announced our vision for Pathways, a single model that could generalize across domains and tasks while being highly efficient. An important milestone toward realizing this vision was to develop the new Pathways system to orchestrate distributed computation for accelerators. In “PaLM: Scaling Language Modeling with Pathways”, we introduce the Pathways Language Model (PaLM), a 540-billion parameter, dense decoder-only Transformer model trained with the Pathways system, which enabled us to efficiently train a single model across multiple TPU v4 Pods. We evaluated PaLM on hundreds of language understanding and generation tasks, and found that it achieves state-of-the-art few-shot performance across most tasks, by significant margins in many cases.

Read more | >

The pretraining of BERT-type large language models — which can scale up to billions of parameters — is crucial for obtaining state-of-the-art performance on many natural language processing (NLP) tasks. This pretraining process however is expensive, and has become a bottleneck hindering the industrial application of such large language models.

In the new paper Token Dropping for Efficient BERT Pretraining, a research team from Google, New York University, and the University of Maryland proposes a simple but effective “token dropping” technique that significantly reduces the pretraining cost of transformer models such as BERT, without degrading performance on downstream fine-tuning tasks.

The team summarizes their main contributions as:

Read more | >

To effectively interact with humans in crowded social settings, such as malls, hospitals, and other public spaces, robots should be able to actively participate in both group and one-to-one interactions. Most existing robots, however, have been found to perform much better when communicating with individual users than with groups of conversing humans.

Hooman Hedayati and Daniel Szafir, two researchers at University of North Carolina at Chapel Hill, have recently developed a new data-driven technique that could improve how robots communicate with groups of humans. This method, presented in a paper presented at the 2022 ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘22), allows robots to predict the positions of humans in conversational groups, so that they do not mistakenly ignore a person when their sensors are fully or partly obstructed.

“Being in a conversational group is easy for humans but challenging for robots,” Hooman Hedayati, one of the researchers who carried out the study, told TechXplore. “Imagine that you are talking with a group of friends, and whenever one of your friends blinks, she stops talking and asks if you are still there. This potentially annoying scenario is roughly what can happen when a robot is in conversational groups.”

Read more | >

A new tool speeds up development of vaccines and other pharmaceutical products by more than 1 million times while minimizing costs.

In search of pharmaceutical agents such as new vaccines, industry will routinely scan thousands of related candidate molecules. A novel technique allows this to take place on the nano scale, minimizing use of materials and energy. The work is published in the journal Nature Chemistry.

More than 40,000 molecules can be synthesized and analyzed within an area smaller than a pinhead. The method, developed through a highly interdisciplinary research effort in Denmark, promises to drastically reduce the amounts of material, energy, and economic cost for .

Read more | >

ab initio calculations

Classical computing is of very little help when the task to be accomplished pertains to ab initio calculations. With quantum computing in place, you have a quantum system simulating another quantum system. Furthermore, tasks such as modelling atomic bonding or estimating electron orbital overlaps can be done much more precisely.

Read more | >