Menu

Blog

Latest posts

Feb 21, 2024

Let’s build the GPT Tokenizer

Posted by in categories: health, information science, robotics/AI

W/ Andrej Karpathy


The Tokenizer is a necessary and pervasive component of Large Language Models (LLMs), where it translates between strings and tokens (text chunks). Tokenizers are a completely separate stage of the LLM pipeline: they have their own training sets, training algorithms (Byte Pair Encoding), and after training implement two fundamental functions: encode() from strings to tokens, and decode() back from tokens to strings. In this lecture we build from scratch the Tokenizer used in the GPT series from OpenAI. In the process, we will see that a lot of weird behaviors and problems of LLMs actually trace back to tokenization. We’ll go through a number of these issues, discuss why tokenization is at fault, and why someone out there ideally finds a way to delete this stage entirely.

Continue reading “Let’s build the GPT Tokenizer” »

Feb 21, 2024

Inexpensive device that can harvest energy from a light breeze and store it as electricity

Posted by in categories: computing, mobile phones

Researchers from Nanyang Technological University in Singapore (NTU Singapore) have created a low-cost tool that can capture power from wind energy as moderate as a light breeze.

The gadget can create a voltage of three volts and energy power of up to 290 microwatts when exposed to winds with speeds as low as 2 meters per second (m/s). This is enough to power a commercial sensor device and allow it to transfer data to a smartphone or computer.

Feb 21, 2024

Finding a food-cancer connection: How what you eat might affect your cancer risk

Posted by in category: biotech/medical

This study reveals a mechanism by which the environment, diet in this case, can influence cancer risk.

Feb 21, 2024

The Brain Of Colonel Barham

Posted by in categories: robotics/AI, space

“With the world growing more crowded, the great powers strive to conquer other planets. The race is on. The interplanetary sea has been charted; the first caravelle of space is being constructed. Who will get there first? Who will be the new Columbus?” A robot probe is being readied to explore the secrets of the red planet, Mars. The only component lacking: a human brain. No body. Just the brain. It is needed to deal with unexpected crises in the cold, dark depths of space. The perfect volunteer is found in Colonel Barham, a brilliant but hot-tempered astronaut dying of leukemia. But all goes awry as, stripped of his mortal flesh, Barham — or rather his disembodied brain — is consumed with a newly-found power to control…or destroy. Project psychiatrist Major McKinnon (Grant Williams) diagnoses the brain as having delusions of grandeur…but, just perhaps, Col. Barham has achieved grandeur.

Feb 21, 2024

New ‘supermolecule’ demonstrated for the first time at record-breaking ultracold temperature

Posted by in categories: chemistry, quantum physics

Largest polyatomic molecule created and stabilised at just above absolute zero provides new insights into quantum chemistry.

Feb 21, 2024

Self-propelling nanobots shrink bladder tumours in mice by 90%

Posted by in categories: bioengineering, biotech/medical, nanotechnology

Imagine an army of self-propelling, radioisotope-covered particles 2,500 to 10,000 times smaller than a speck of dust that, upon injection into the body, search for and attach themselves to cancerous tumours, destroying them. Sounds like science fiction? Not so for mice with bladder cancer.

Researchers in Spain report that nanoparticles containing radioactive iodine and which propel themselves upon reaction with urea have the ability to distinguish cancerous bladder tumours from healthy tissue. These “nanobots” penetrate the tumour’s extracellular matrix and accumulate within it, enabling the radionuclide therapy to reach its precise target. In a study conducted at the Institute for Bioengineering of Catalonia (IBEC) in Barcelona, mice receiving a single dose of this treatment had a 90% reduction in the size of bladder tumours compared with untreated animals.

This novel approach may one day revolutionize the treatment of bladder cancer. Bladder cancer is the tenth most common cancer in the world, with over 600,000 new cases diagnosed in 2022 and more than 220,000 deaths globally, according to the World Health Organization’s Global Cancer Observatory.

Feb 21, 2024

Sweeping chronic fatigue study brings clues but not clarity to mysterious syndrome

Posted by in categories: biotech/medical, neuroscience

“Megaworkup” revealed brain and immune differences in a small patient group.

Feb 21, 2024

Down syndrome identified in 2600-year-old infants through their DNA

Posted by in category: biotech/medical

Study shows power of paleogenomics to explore incidence of disability in the past.

Feb 21, 2024

A new AI model called Morpheus-1 claims to induce lucid dreaming

Posted by in categories: education, robotics/AI

‘Inception’ is about to become a documentary — or are we all just dreaming?

Feb 21, 2024

Discovery may enable an Effective Long-Term Lupus Treatment

Posted by in category: biotech/medical

Australian researchers have worked out how to fix a defect that causes lupus, and hope their world-first discovery will offer effective long-term treatment.

Published in Nature Communications, the Monash University-led study found a way to reprogram the defective cells of lupus patients with protective molecules from healthy people.

Using human cells, the new treatment restores the protective side of the immune system that prevents autoimmunity, which is when the immune system attacks its own cells.

Page 1 of 10,62712345678Last