Kaggle notebook with all the code: https://www.kaggle.com/wwsalmon/simple-mnist-nn-from-scratch-numpy-no-tf-keras.
Blog article with more/clearer math explanation: https://www.samsonzhang.com/2020/11/24/understanding-the-mat…numpy.html
An introductory lecture for MIT course 6.S094 on the basics of deep learning including a few key ideas, subfields, and the big picture of why neural networks have inspired and energized an entire new generation of researchers. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo.
INFO:
Website: https://deeplearning.mit.edu.
GitHub: https://github.com/lexfridman/mit-deep-learning.
Slides: http://bit.ly/deep-learning-basics-slides.
Playlist: http://bit.ly/deep-learning-playlist.
Blog post: https://link.medium.com/TkE476jw2T
OUTLINE:
0:00 — Introduction.
0:53 — Deep learning in one slide.
4:55 — History of ideas and tools.
9:43 — Simple example in TensorFlow.
11:36 — TensorFlow in one slide.
13:32 — Deep learning is representation learning.
16:02 — Why deep learning (and why not)
22:00 — Challenges for supervised learning.
38:27 — Key low-level concepts.
46:15 — Higher-level methods.
1:06:00 — Toward artificial general intelligence.
CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: https://twitter.com/lexfridman.
- LinkedIn: https://www.linkedin.com/in/lexfridman.
- Facebook: https://www.facebook.com/lexfridman.
- Instagram: https://www.instagram.com/lexfridman
What are the neurons, why are there layers, and what is the math underlying it?
Help fund future projects: https://www.patreon.com/3blue1brown.
Written/interactive form of this series: https://www.3blue1brown.com/topics/neural-networks.
Additional funding for this project provided by Amplify Partners.
Typo correction: At 14 minutes 45 seconds, the last index on the bias vector is n, when it’s supposed to in fact be a k. Thanks for the sharp eyes that caught that!
For those who want to learn more, I highly recommend the book by Michael Nielsen introducing neural networks and deep learning: https://goo.gl/Zmczdy.
There are two neat things about this book. First, it’s available for free, so consider joining me in making a donation Nielsen’s way if you get something out of it. And second, it’s centered around walking through some code and data which you can download yourself, and which covers the same example that I introduce in this video. Yay for active learning!
https://github.com/mnielsen/neural-networks-and-deep-learning.
I also highly recommend Chris Olah’s blog: http://colah.github.io/
Fossils of a 70 million-year-old platypus relative called Patagorhynchus pascuali found in South America show that egg-laying mammals evolved on more than one continent.
A person who is lucidly aware of the miracles that surround him, who has learned to bear up under the loneliness, has made quite a bit of progress on the road to wisdom.
Quantum consciousness microtubules.
Share your videos with friends, family, and the world.
Research suggests inflammation may be just as important as cholesterol as a cause of heart attacks, suggesting different treatments should be considered for prevention.
Analysis By Clare Wilson
Researchers at the School of Cyber Security at Korea University, Seoul, have presented a new covert channel attack named CASPER can leak data from air-gapped computers to a nearby smartphone at a rate of 20bits/sec.
The CASPER attack leverages the internal speakers inside the target computer as the data transmission channel to transmit high-frequency audio that the human ear cannot hear and convey binary or Morse code to a microphone up to 1.5m away.
The receiving microphone can be in a smartphone recording sound inside the attacker’s pocket or a laptop in the same room.
Deep learning has made significant strides in text generation, translation, and completion in recent years. Algorithms trained to predict words from their surrounding context have been instrumental in achieving these advancements. However, despite access to vast amounts of training data, deep language models still need help to perform tasks like long story generation, summarization, coherent dialogue, and information retrieval. These models have been shown to need help capturing syntax and semantic properties, and their linguistic understanding needs to be more superficial. Predictive coding theory suggests that the brain of a human makes predictions over multiple timescales and levels of representation across the cortical hierarchy. Although studies have previously shown evidence of speech predictions in the brain, the nature of predicted representations and their temporal scope remain largely unknown. Recently, researchers analyzed the brain signals of 304 individuals listening to short stories and found that enhancing deep language models with long-range and multi-level predictions improved brain mapping.
The results of this study revealed a hierarchical organization of language predictions in the cortex. These findings align with predictive coding theory, which suggests that the brain makes predictions over multiple levels and timescales of expression. Researchers can bridge the gap between human language processing and deep learning algorithms by incorporating these ideas into deep language models.
The current study evaluated specific hypotheses of predictive coding theory by examining whether cortical hierarchy predicts several levels of representations, spanning multiple timescales, beyond the neighborhood and word-level predictions usually learned in deep language algorithms. Modern deep language models and the brain activity of 304 people listening to spoken tales were compared. It was discovered that the activations of deep language algorithms supplemented with long-range and high-level predictions best describe brain activity.
There’s a new sight to see in the skies over the eastern seaboard of the United States courtesy of Rocket Lab and NASA.
The space startup is beginning to make a habit of launching its Electron rockets from NASA’s Wallops Flight Facility in Virginia. Unlike the majority of space launches in the US that blast off from the far southeastern corner of the country in Florida, some of the nation’s largest population centers have a view of launches from Wallops.
The “Stronger Together” mission is the second launch of the space startup’s Electron rocket from Virginia. Before adding a second launch facility, all of the company’s previous launches were conducted from its primary launch pads in New Zealand over the past couple years.