Recently, computation in memory becomes very hot due to the urgent needs of high computing efficiency in artificial intelligence applications. In contrast to von-neumann architecture, computation in memory technology avoids the data movement between CPU/GPU and memory which could greatly reduce the power consumption. Memristor is one ideal device which could not only store information with multi-bits, but also conduct computing using ohm’s law. To make the best use of the memristor in neuromorphic systems, a memristor-friendly architecture and the software-hardware collaborative design methods are essential, and the key problem is how to utilize the memristor’s analog behavior. We have designed a generic memristor crossbar based architecture for convolutional neural networks and perceptrons, which take full consideration of the analog characteristics of memristors. Furthermore, we have proposed an online learning algorithm for memristor based neuromorphic systems which overcomes the varation of memristor cells and endue the system the ability of reinforcement learning based on memristor’s analog behavior.

Full abstract and speaker details can be found here: https://nus.edu/3cSFD3e.

Register for free for all our upcoming webinars at https://nus.edu/3bcN9pS.

Algorithms like OpenAI’s GPT-4 are like brains frozen in time. A new study shows how future AIs could learn continuously in response to a changing world.

The CEO of Amazon Web Services suggested in leaked audio that human devs may not be coding in as little as two years.

The team has released the width-pruned version of the model on Hugging Face under the Nvidia Open Model License, which allows for commercial use. This makes it accessible to a wider range of users and developers who can benefit from its efficiency and performance.

“Pruning and classical knowledge distillation is a highly cost-effective method to progressively obtain LLMs [large language models] of smaller size, achieving superior accuracy compared to training from scratch across all domains,” the researchers wrote. “It serves as a more effective and data-efficient approach compared to either synthetic-data-style fine-tuning or pretraining from scratch.”

This work is a reminder of the value and importance of the open-source community to the progress of AI. Pruning and distillation are part of a wider body of research that is enabling companies to optimize and customize LLMs at a fraction of the normal cost. Other notable works in the field include Sakana AI’s evolutionary model-merging algorithm, which makes it possible to assemble parts of different models to combine their strengths without the need for expensive training resources.