Lifeboat Foundation: Safeguarding Humanity
Toggle light / dark theme

Blog

Category: computing – Page 382

face_with_colon_three circa 2016.

Two basic types of encryption schemes are used on the internet today. One, known as symmetric-key cryptography, follows the same pattern that people have been using to send secret messages for thousands of years. If Alice wants to send Bob a secret message, they start by getting together somewhere they can’t be overheard and agree on a secret key; later, when they are separated, they can use this key to send messages that Eve the eavesdropper can’t understand even if she overhears them. This is the sort of encryption used when you set up an online account with your neighborhood bank; you and your bank already know private information about each other, and use that information to set up a secret password to protect your messages.

The second scheme is called public-key cryptography, and it was invented only in the 1970s. As the name suggests, these are systems where Alice and Bob agree on their key, or part of it, by exchanging only public information. This is incredibly useful in modern electronic commerce: if you want to send your credit card number safely over the internet to Amazon, for instance, you don’t want to have to drive to their headquarters to have a secret meeting first. Public-key systems rely on the fact that some mathematical processes seem to be easy to do, but difficult to undo. For example, for Alice to take two large whole numbers and multiply them is relatively easy; for Eve to take the result and recover the original numbers seems much harder.

Public-key cryptography was invented by researchers at the Government Communications Headquarters (GCHQ) — the British equivalent (more or less) of the US National Security Agency (NSA) — who wanted to protect communications between a large number of people in a security organization. Their work was classified, and the British government neither used it nor allowed it to be released to the public. The idea of electronic commerce apparently never occurred to them. A few years later, academic researchers at Stanford and MIT rediscovered public-key systems. This time they were thinking about the benefits that widespread cryptography could bring to everyday people, not least the ability to do business over computers.

Read more | >

Electronics engineers worldwide are trying to improve the performance of devices, while also lowering their power consumption. Tunnel field-effect transistors (TFETs), an experimental class of transistors with a unique switching mechanism, could be a particularly promising solution for developing low-power electronics.

Despite their potential, most TFETs based on silicon and III-V heterojunctions exhibit low on-current densities and on/off current ratios in some modes of operation. Fabricating these transistors using 2D materials could help to improve electrostatic control, potentially increasing their on-current densities and on/off ratios.

Researchers at University of Pennsylvania, the Chinese Academy of Sciences, the National Institute of Standards and Technology, and the Air Force Research Laboratory have recently developed new heterojunction tunnel triodes based on van der Waals heterostructures formed from 2D metal selenide and 3D silicon. These triodes, presented in a paper published in Nature Electronics, could outperform other TFETs presented in the past in terms of on-current densities and on/off ratios.

Read more | >

Turing’s machine should sound familiar for another reason. It’s similar to the way ribosomes read genetic code on ribbons of RNA to construct proteins.

Cellular factories are a kind of natural Turing machine. What Leigh’s team is after would work the same way but go beyond biochemistry. These microscopic Turing machines, or molecular computers, would allow engineers to write code for some physical output onto a synthetic molecular ribbon. Another molecule would travel along the ribbon, read (and one day write) the code, and output some specified action, like catalyzing a chemical reaction.

Now, Leigh’s team says they’ve built the first components of a molecular computer: A coded molecular ribbon and a mobile molecular reader of the code.

Read more | >

Their quantum computing processors can store information up to two milliseconds.

Researchers from the University of New South Wales have broken new ground in quantum computing by demonstrating that ‘spin qubits’- qubits where the information is stored in the spin momentum of an electron-can store data for up to two milliseconds, 100 times longer than previous benchmarks in the same quantum processor.

Classical computers work with bits—consisting of ones and zeroes—but a quantum computer uses quantum bits or qubits, which, on top of the ones and zeroes, also has a superposition where it can be a one and a zero at the same time.

Hh5800/iStock.

The time that qubits can be manipulated in increasingly complex calculations is known as ‘coherence time.’

Read more | >

Solar cells that are stretchable, flexible and wearable won the day and the best poster award from a pool of 215 at Research Expo 2016 April 14 at the University of California San Diego. The winning nanoengineering researchers aim to manufacture small, flexible devices that can power watches, LEDs and wearable sensors. The ultimate goal is to design and build much bigger flexible solar cells that could be used as power sources and shelter in natural disasters and other emergencies.

Research Expo is an annual showcase of top graduate research projects for the Jacobs School of Engineering at UC San Diego. During the poster session, graduate students are judged on the quality of their work and how well they articulate the significance of their research to society. Judges from industry, who often are alumni, pick the winners for each department. A group of faculty judges picks the overall winner from the six department winners.

This year, in addition to solar cells, judges recognized efforts to develop 3D skeletal muscle on a chip; a better way to alleviate congestion in data center networks; a nano-scale all-optical sensor; fiber optic strain sensors for structural health monitoring; and a way to predict earthquake damage in freestanding structural systems.

Read more | >

Large Language Models have the ability to store vast amounts of facts about the world. But little is known, how these models actually do this. This paper aims at discovering the mechanism and location of storage and recall of factual associations in GPT models, and then proposes a mechanism for the targeted editing of such facts, in form of a simple rank-one update to a single MLP layer. This has wide implications both for how we understand such models’ inner workings, and for our ability to gain greater control over such models in the future.

OUTLINE:
0:00 — Introduction.
1:40 — What are the main questions in this subfield?
6:55 — How causal tracing reveals where facts are stored.
18:40 — Clever experiments show the importance of MLPs.
24:30 — How do MLPs store information?
29:10 — How to edit language model knowledge with precision?
36:45 — What does it mean to know something?
39:00 — Experimental Evaluation & the CounterFact benchmark.
45:40 — How to obtain the required latent representations?
51:15 — Where is the best location in the model to perform edits?
58:00 — What do these models understand about language?
1:02:00 — Questions for the community.

Paper: https://arxiv.org/abs/2202.05262
Follow-up paper on Mass-Editing Memory in a Transformer: https://arxiv.org/abs/2210.

Abstract:
We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model’s factual predictions. This reveals a distinct set of steps in middle-layer feed-forward modules that mediate factual predictions while processing subject tokens. To test our hypothesis that these computations correspond to factual association recall, we modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another. Our results confirm an important role for mid-layer feed-forward modules in storing factual associations and suggest that direct manipulation of computational mechanisms may be a feasible approach for model editing. The code, dataset, visualizations, and an interactive demo notebook are available at this https URL

Authors: Kevin Meng, David Bau, Alex Andonian, Yonatan Belinkov.

Links:

Continue reading “ROME: Locating and Editing Factual Associations in GPT (Paper Explained & Author Interview)” | >