Toggle light / dark theme

Calculus on Computational Graphs: Backpropagation

Backpropagation is the key algorithm that makes training deep models computationally tractable. For modern neural networks, it can make training with gradient descent as much as ten million times faster, relative to a naive implementation. That’s the difference between a model taking a week to train and taking 200,000 years.

Beyond its use in deep learning, backpropagation is a powerful computational tool in many other areas, ranging from weather forecasting to analyzing numerical stability – it just goes by different names. In fact, the algorithm has been reinvented at least dozens of times in different fields (see Griewank (2010)). The general, application independent, name is “reverse-mode differentiation.”

Fundamentally, it’s a technique for calculating derivatives quickly. And it’s an essential trick to have in your bag, not only in deep learning, but in a wide variety of numerical computing situations.

Pattern recognition in the nucleation kinetics of non-equilibrium self-assembly

Can the intrinsic physics of multicomponent systems show neural network like #Computation? A new study shows how molecules draw on the rules of #physics to perform computations similar to neural networks:


Examination of nucleation during self-assembly of multicomponent structures illustrates how ubiquitous molecular phenomena inherently classify high-dimensional patterns of concentrations in a manner similar to neural network computation.

Wolfram Alpha, Meet ChatGPT: Stephen Wolfram Talks Intelligence And Large Language Models

They’re two great tastes that taste great together. Or rather, they’re two technologies that, put together in collaborative ways, are becoming much more powerful!

Marvin Minsky famously said that the brain is not one computer, but several hundred computers working in tandem. If that’s true, ChatGPT’s cognitive power just got a boost with the creation of a Wolfram Alpha plug-in that allows for the two systems to send and receive natural language input, so that ChatGPT systems can utilize a different system of symbolic representation that had already been pioneered before the days when we could just ask a computer to write an essay.

We heard early this year that teams were working on this merge, and it’s been interesting to the AI community. Now it’s come to fruition.

Generative AI helps to explain human memory and imagination

Recent advances in generative AI help to explain how memories enable us to learn about the world, relive old experiences and construct totally new experiences for imagination and planning, according to a new study by UCL researchers.

The study, published in Nature Human Behaviour, uses an AI —known as a generative neural network—to simulate how in the brain learn from and remember a series of events (each one represented by a simple ).

The model featured networks representing the hippocampus and neocortex, to investigate how they interact. Both parts of the brain are known to work together during , imagination and planning.

Arizona State students will get their own ChatGPT-powered tutors as OpenAI partners with the university

On Thursday, OpenAI and ASU announced the first-of-its-kind partnership, which has reportedly been in the works for six months.

Students, professors, and researchers are set to get access to the tech in February. The university plans to build personalized AI tutors and avatars for students and expand its prompt engineering course.

In a press release, Arizona State University said the partnership would set a new precedent for how universities “enhance learning, creativity and student outcomes.”

Mini-robots modeled on insects may be smallest, lightest, fastest ever developed

Two insect-like robots, a mini-bug and a water strider, developed at Washington State University, are the smallest, lightest and fastest fully functional micro-robots ever known to be created.

Such miniature robots could someday be used for work in areas such as artificial pollination, search and rescue, , micro-fabrication or robotic-assisted surgery. Reporting on their work in the proceedings of the IEEE Robotics and Automation Society’s International Conference on Intelligent Robots and Systems, the mini-bug weighs in at eight milligrams while the weighs 55 milligrams. Both can move at about six millimeters a second.

“That is fast compared to other micro-robots at this scale, although it still lags behind their biological relatives,” said Conor Trygstad, a Ph.D. student in the School of Mechanical and Materials Engineering and lead author on the work. An ant typically weighs up to five milligrams and can move at almost a meter per second.

‘Matrix here we come’: Scientists say AI now able to reproduce new AI on its own for the first time

Artificial intelligence models can now replicate independently new AI systems without any human intervention, according to scientists of the project.

A scientific collaboration between Aizip Inc., the Massachusetts Institute of Technology, and several University of California campuses has made a breakthrough in using larger AI models that can create smaller AI models.

OATH OF OFFICE: DID BIDEN FULFILL HIS PROMISE TO RESTORE THE ECONOMY?

/* */