Lifeboat Foundation

Retention is an alternative to Attention in Transformers that can both be written in a parallel and in a recurrent fashion. This means the architecture achieves training parallelism while maintaining low-cost inference. Experiments in the paper look very promising.

OUTLINE:
0:00 — Intro.
2:40 — The impossible triangle.
6:55 — Parallel vs sequential.
15:35 — Retention mechanism.
21:00 — Chunkwise and multi-scale retention.
24:10 — Comparison to other architectures.
26:30 — Experimental evaluation.

Continue reading “Retentive Network: A Successor to Transformer for Large Language Models (Paper Explained)” »

Kevin Slagle, Quantum 7, 1113 (2023). Although tensor networks are powerful tools for simulating low-dimensional quantum physics, tensor network algorithms are very computationally costly in higher spatial dimensions. We introduce $\textit{quantum gauge networks}$: a different kind of tensor network ansatz for which the computation cost of simulations does not explicitly increase for larger spatial dimensions. We take inspiration from the gauge picture of quantum dynamics, which consists of a local wavefunction for each patch of space, with neighboring patches related by unitary connections. A quantum gauge network (QGN) has a similar structure, except the Hilbert space dimensions of the local wavefunctions and connections are truncated. We describe how a QGN can be obtained from a generic wavefunction or matrix product state (MPS). All $2k$-point correlation functions of any wavefunction for $M$ many operators can be encoded exactly by a QGN with bond dimension $O(M^k)$. In comparison, for just $k=1$, an exponentially larger bond dimension of $2^{M/6}$ is generically required for an MPS of qubits. We provide a simple QGN algorithm for approximate simulations of quantum dynamics in any spatial dimension. The approximate dynamics can achieve exact energy conservation for time-independent Hamiltonians, and spatial symmetries can also be maintained exactly. We benchmark the algorithm by simulating the quantum quench of fermionic Hamiltonians in up to three spatial dimensions.

A chinless jawbone from eastern China that displays both modern and archaic features could represent a new branch of the human family tree.

With the demise of Roche’s gantenerumab in November 2022, the Alzheimer’s disease space became a two-horse race between Eisai and Biogen’s Leqembi (lecanemab) and Eli Lilly’s donanemab. One, Leqembi, received full FDA approval in July; the other, donanemab, is widely expected to secure the agency’s approval before the end of 2023.

With a potential combined market value of $30 billion, BioSpace takes a deep dive into the Phase III data supporting Eisai and Biogen’s Leqembi and Eli Lilly’s investigational donanemab.

A multispectral, super-low-dose photoacoustic microscopy (SLD-PAM) system developed by City University of Hong Kong (CUHK) achieves significantly higher sensitivity than traditional optical resolution photoacoustic imaging.

By providing an exceptionally high level of sensitivity, SLD-PAM could help broaden the use of photoacoustic microscopy in biomedical applications. In the future, it could translate to clinical settings; for example, it could be used for ophthalmic exams where a low-power laser is preferred for the patient’s safety and comfort. Long-term monitoring of pharmacokinetics or blood flow also requires low-dose imaging to alleviate perturbation to tissue function.

General-purpose automation could radically improve society by vastly accelerating construction, manufacturing, and R&D. Just as the scale and complexity of today’s cities would have been unimaginable 200 years ago, we may see a similar factor of value growth over the next 50 years. Quality of life may dramatically increase as well. I envision that billions could be lifted out of poverty and the average person may live like today’s wealthiest top 1%. Space colonization might be made feasible. Keep in mind these projections are highly speculative. Nonetheless, it is worth considering the remarkable possibilities! #automation #tech #robotics #futurism

The rise of humanoid robots didn’t happen overnight, but a kind of perfect storm has accelerated the phenomenon over the past year and change. The foundation, of course, is decades of research.

Toiling away in research facilities and R&D departments laid the ground work for a new generation of technology. The necessary software and components have come a long way, driven by innovations in industrial robotics, autonomous driving and even the smartphone industry.

Continue reading “This is Apptronik’s humanoid robot, Apollo” »

One promising approach to treating type 1 diabetes is implanting pancreatic islet cells that can produce insulin when needed, which can free patients from giving themselves frequent insulin injections. However, one major obstacle to this approach is that once the cells are implanted, they eventually run out of oxygen and stop producing insulin.

To overcome that hurdle, MIT engineers have designed a new implantable device that not only carries hundreds of thousands of insulin-producing islet cells, but also has its own on-board oxygen factory, which generates oxygen by splitting water vapor found in the body.

The researchers showed that when implanted into diabetic mice, this device could keep the mice’s blood glucose levels stable for at least a month. The researchers now hope to create a larger version of the device, about the size of a stick of chewing gum, that could eventually be tested in people with type 1 diabetes.

Shared Access Signature (SAS) link exposed a storage bucket with 38TB of private data, including passwords, Teams messages, and the backups of two Microsoft AI research employees’ workstations.

Discover the evolution of Android trojans — ‘Hook’ inherits its powers from ‘ERMAC.’ How does it outperform its predecessor? Read on.