Lifeboat Foundation

MIT 9.40 Introduction to Neural Computation, Spring 2018
Instructor: Michale Fee.
View the complete course: https://ocw.mit.edu/9-40S18
YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP61I4aI5T6OaFfRK2gihjiMm.

Covers the dendrite circuit diagram, voltage plot, length diagram, dendritic radius, electronic length, and the circuit diagram of a two-compartment model.

Continue reading “6: Dendrites — Intro to Neural Computation” »

To what degree do qualitative dendritic properties impact ANN-based neuron models’ ability to perform ML benchmark tasks.

This could solve a conundrum that’s been plaguing astronomers for almost half a century.

Instead of a single Big Bang that brought the universe into existence billions of years ago, cosmologists are starting to suspect there may have been a second transformative event that could explain the vast abundance of dark matter in the universe.

As New Scientist reports, our recent glimpses into early moments of the universe, just millions of years after the Big Bang, could allow us to gain new insights into this “dark” Big Bang, which could solve a conundrum that’s been plaguing astronomers for almost half a century.

Continue reading “Scientists Say There May Have Been a Second Big Bang” »

StreamingLLM is an innovative framework that allows large language models to handle text of infinite length without the need for finetuning. This technique preserves attention sinks to maintain a near-normal attention score distribution. When the sequence of the conversation with the LLM surpasses the model’s context length, retains the KV cache for the attention sink tokens—four initial tokens are sufficient—and discards subsequent tokens to make room for the sliding window tokens. This approach enables the model to extend its context and stabilize its performance without having to recompute the entire KV values.

“The introduction of four initial tokens, as attention sinks, suffices to restore the LLM’s performance,” the researchers write. “In contrast, adding just one or two doesn’t achieve full recovery. We believe this pattern emerges because these models didn’t include a consistent starting token across all input samples during pre-training.”

Under the framework, the KV cache comprises the attention sinks and the rolling KV cache that retains the most recent tokens vital for language modeling. The researchers emphasize the versatility of, stating, design is versatile and can be seamlessly incorporated into any autoregressive language model that employs relative positional encoding.”

Google’s company-defining effort to catch up to ChatGPT creator OpenAI is turning out to be harder than expected.

Google representatives earlier this year told some cloud customers and business partners they would get access to the company’s new conversational AI, a large language model known as Gemini, by November. But the company recently told them not to expect it until the first quarter of next year, according to two people with direct knowledge. The delay comes at a bad time for Google, whose cloud sales growth has slowed while that of its bigger rival, Microsoft, has accelerated. Part of Microsoft’s success has come from selling OpenAI’s technology to its customers.

Fio Omenetto, Silklab, Tufts University.

Transistors were a pivotal innovation in the evolution of electronic technology, and they have played a critical part in the miniaturization and advancement of electronic equipment. However, they could be even better.

German researchers hoping to be the first to successfully measure quantum flickering directly in a completely empty vacuum are setting their sights on 2024.

If successful, the first-of-their-kind experiments are expected to either confirm the existence of quantum energy in the vacuum, a core concept of quantum electrodynamics (QED), or potentially result in the discovery of previously unknown laws of nature.

Quantum Flickering, Ghost Particles, and Energy in the Vacuum.

Physicists from the Eötvös Loránd University (ELTE) have been conducting research on the matter constituting the atomic nucleus utilizing the world’s three most powerful particle accelerators. Their focus has been on mapping the “primordial soup” that filled the universe in the first millionth of a second following its inception.

Intriguingly, their measurements showed that the movement of observed particles bears resemblance to the search for prey of marine predators, the patterns of climate change, and the fluctuations of stock market.

In the immediate aftermath of the Big Bang, temperatures were so extreme that atomic nuclei could not exists, nor could nucleons, their building blocks. Hence, in this first instance the universe was filled with a “primordial soup” of quarks and gluons.

To teach an AI agent a new task, like how to open a kitchen cabinet, researchers often use reinforcement learning—a trial-and-error process where the agent is rewarded for taking actions that get it closer to the goal.

In many instances, a human expert must carefully design a reward function, which is an incentive mechanism that gives the agent motivation to explore. The human expert must iteratively update that reward function as the agent explores and tries different actions. This can be time-consuming, inefficient, and difficult to scale up, especially when the task is complex and involves many steps.

Researchers from MIT, Harvard University, and the University of Washington have developed a new reinforcement learning approach that doesn’t rely on an expertly designed reward function. Instead, it leverages crowdsourced feedback, gathered from many non-expert users, to guide the agent as it learns to reach its goal. The work has been published on the pre-print server arXiv.