In this video I discuss Neuromorphic Computing and the Future of AI
#AI
Support me on Patreon: https://www.patreon.com/AnastasiInTech
Activity of neurons embedded in networks is an inseparable composition of evoked and intrinsic processes. Prevalence of either component depends on the neuron’s function and state (e.g. low/high conductance or depolarization states). Dominant intrinsic firing is thought functionally normal for the pacemaker neuron, but not for the sensory afferent neuron or spinal motoneuron serving to transmit rather than to originate signals. Activity of the multi-functional networked cell, depending on its intrinsic states, bears both cell-and network-defined features. Complex firing patterns of a neuron are conventionally attributed to complex spatial-temporal organization of inputs received from the network-mates via synapses, in vast majority dendritic. This attribution reflects widespread views of the within-cell job sharing, such that the main function of the dendrites is to receive signals and deliver them to the axo-somatic trigger zone, which actually generates the output pattern. However, these views require revisiting with account of active properties of the dendrites due to voltage-dependent channels found in the dendritic membrane of practically all types of explored neurons. Like soma and axon, the dendrites with active membrane are able to generate self-maintained, propagating depolarizations and thus share intrinsic pattern-forming role with the trigger zone. Unlike the trigger zone, the dendrites have complex geometry, which is subject to developmental, activity-dependent, or neurodegenerative changes. Structural features of the arborization inevitably impact on electrical states and cooperative behavior of its constituting parts at different levels of organization, from branches and sub-trees to voltage-and ligand-gated ion channels populating the membrane. Nearly two decades of studies have brought numerous phenomenological demonstrations of influence of the dendritic structure on firing patterns in neurons. A necessary step forward is to comprehend these findings and build a firm theoretical basis, including quantitative relationships between geometrical and electrical characteristics determining intrinsic firing of neurons. This Research Topic is aimed at bringing together contributions of researches from different domains of expertise and building a conceptual framework for deeper insight into the nature of dynamic intrinsic motifs in the firing patterns.
We welcome research and methodology papers, mini-reviews, conceptual generalizations and opinions on the following issues:
1. Electrical states of heterogeneous populations of ion channels: definition, life-times, meta-and multi-stability.
MIT 9.40 Introduction to Neural Computation, Spring 2018
Instructor: Michale Fee.
View the complete course: https://ocw.mit.edu/9-40S18
YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP61I4aI5T6OaFfRK2gihjiMm.
Covers extracellular spike waveforms, local field potentials, spike signals, threshold crossing, the peri-stimulus time histogram, and the firing rate of a neuron.
Continue reading “8: Spike Trains — Intro to Neural Computation” »
MIT 9.40 Introduction to Neural Computation, Spring 2018
Instructor: Michale Fee.
View the complete course: https://ocw.mit.edu/9-40S18
YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP61I4aI5T6OaFfRK2gihjiMm.
Covers the dendrite circuit diagram, voltage plot, length diagram, dendritic radius, electronic length, and the circuit diagram of a two-compartment model.
Continue reading “6: Dendrites — Intro to Neural Computation” »
To what degree do qualitative dendritic properties impact ANN-based neuron models’ ability to perform ML benchmark tasks.
This could solve a conundrum that’s been plaguing astronomers for almost half a century.
Instead of a single Big Bang that brought the universe into existence billions of years ago, cosmologists are starting to suspect there may have been a second transformative event that could explain the vast abundance of dark matter in the universe.
As New Scientist reports, our recent glimpses into early moments of the universe, just millions of years after the Big Bang, could allow us to gain new insights into this “dark” Big Bang, which could solve a conundrum that’s been plaguing astronomers for almost half a century.
Continue reading “Scientists Say There May Have Been a Second Big Bang” »
StreamingLLM is an innovative framework that allows large language models to handle text of infinite length without the need for finetuning. This technique preserves attention sinks to maintain a near-normal attention score distribution. When the sequence of the conversation with the LLM surpasses the model’s context length, retains the KV cache for the attention sink tokens—four initial tokens are sufficient—and discards subsequent tokens to make room for the sliding window tokens. This approach enables the model to extend its context and stabilize its performance without having to recompute the entire KV values.
“The introduction of four initial tokens, as attention sinks, suffices to restore the LLM’s performance,” the researchers write. “In contrast, adding just one or two doesn’t achieve full recovery. We believe this pattern emerges because these models didn’t include a consistent starting token across all input samples during pre-training.”
Under the framework, the KV cache comprises the attention sinks and the rolling KV cache that retains the most recent tokens vital for language modeling. The researchers emphasize the versatility of, stating, design is versatile and can be seamlessly incorporated into any autoregressive language model that employs relative positional encoding.”
Google’s company-defining effort to catch up to ChatGPT creator OpenAI is turning out to be harder than expected.
Google representatives earlier this year told some cloud customers and business partners they would get access to the company’s new conversational AI, a large language model known as Gemini, by November. But the company recently told them not to expect it until the first quarter of next year, according to two people with direct knowledge. The delay comes at a bad time for Google, whose cloud sales growth has slowed while that of its bigger rival, Microsoft, has accelerated. Part of Microsoft’s success has come from selling OpenAI’s technology to its customers.
Fio Omenetto, Silklab, Tufts University.
Transistors were a pivotal innovation in the evolution of electronic technology, and they have played a critical part in the miniaturization and advancement of electronic equipment. However, they could be even better.
German researchers hoping to be the first to successfully measure quantum flickering directly in a completely empty vacuum are setting their sights on 2024.
If successful, the first-of-their-kind experiments are expected to either confirm the existence of quantum energy in the vacuum, a core concept of quantum electrodynamics (QED), or potentially result in the discovery of previously unknown laws of nature.
Quantum Flickering, Ghost Particles, and Energy in the Vacuum.
