Recently, Intel ETANN pioneer Mark Holler donated a pair of Intel 80170 ETANN integrated circuits and other materials related to early neural networks to the 91自拍.
Neural networks are very much in the news these days as the enabling technology for much of the artificial intelligence boom, including large language models, search, generative AI, and chatbots, like ChatGPT. Yet this idea has deep roots in past work going back to the 1950s with the work of Professor Frank Rosenblatt at Cornell University, who built devices that could recognize simple numbers and letters using the world鈥檚 first neural network, the Perceptron, which he invented.
Frank Rosenblatt, left, and Charles W. Wightman work on part of the unit that became the first perceptron in December 1958. Credit: Cornell University, Division of Rare and Manuscript Collections.
Over the course of several AI 鈥淲inters鈥� since that time鈥攄uring which the generous government funding dried up鈥攏eural networks have been in the background waiting for their moment. That moment came about a decade ago as the power of computing driven by Moore鈥檚 Law allowed for a new level of neural network complexity, resulting in 鈥渘eural nets鈥� being deployed in real-world contexts. For example, Google鈥檚 2013 Tensor Processing Units are essentially neural network accelerators that Google uses to improve search. NVIDIA, Amazon, Meta, and (again) Intel have all designed their own neural network processors.
NVIDIA Blackwell AI Accelerator
Meta Training and Inference Accelerator (MTIA)
Intel Gaudi 3 AI Accelerator
While there have been many such milestones in neural networks, especially recently, a very interesting and unique implementation of neural networks took place in 1989 with the announcement of Intel鈥檚 80170 ETANN integrated circuit at that year鈥檚 International Joint Conference on Neural Networks (IJCNN).
ETANN stands for Electrically Trainable Analog Neural Network. Neural networks aren鈥檛 really programmed in the traditional sense, they are trained. An AI programmer鈥檚 job is therefore not writing instructions per se but organizing the data you give a neural network in such a way that it can鈥攖hrough repetition across multiple 鈥渓ayers鈥� of these networks鈥攄iscover patterns and information from incomplete information. Some might call this a form of 鈥渢hinking.鈥� Others, notably commentator Emily Bender et al, in a now-famous paper, consider neural networks and the large language models they support a form of 鈥渟tochastic parrot,鈥� not involving thought at all but just a clever statistical parlor game.
Back to the ETANN: In practical terms, to 鈥渇eed in鈥� the data to the neural network, Intel provided the Intel Neural Network Training System (iNNTS), a training device accessible by a standard Intel PC. Intel also provided a set of software and drivers for controlling the iNNTS. The ETANN incorporated 64 analog neurons and 10,240 analog synapses. At the time, neural networks were pretty much a dead end in terms of applications. There was neither the computing power nor the understanding to deploy them at scale in useful applications, though there was at least one attempt in the military to base a missile seeker on the 80170 chip. As personal computing columnist John C. Dvorak wrote at the time, 鈥淣obody at Intel knows what to make of this thing.鈥�
Block diagram of Intel鈥檚 ETANN 80170 chip.
But the potential seemed there, if only a bit into the future. As Pashley recalled in a recent email, 鈥溾�� I remember when Bruce McCormick first became interested in Neural Nets. It was sometime in the winter of 1987-88 when Bruce and I were on an Intel recruiting trip to Caltech. On this trip to Caltech, we met with Carver [Mead] to talk about his EE281 Integrated Circuit Design students and ended up talking about the potential of Neural Nets. Personally, I knew nothing about Neural Nets, but Carver and Bruce were really excited. On the flight back to Intel, Bruce was bubbling with ideas to use EEPROM and Flash Memory to test out Neural Net鈥檚 potential.鈥�
Intel 801700 ETANN Neural Network Integrated Circuit, 1989.
Like other ideas 鈥渁head of their time鈥� in the history of technology, we see in the Intel 80170 chip the earliest implementation of a reasonably sophisticated neural network in silicon, a prefiguring of the more complex鈥攊ndeed, world-changing鈥擜I accelerators of today.
Holler, Mark, et al. An electrically trainable artificial neural network (ETANN) with 10240 floating gate synapses. International Joint Conference on Neural Networks. Vol. 2. 1989.
Castro, Hernan A., Simon M. Tam, and Mark A. Holler. Implementation and performance of an analog nonvolatile neural network. Analog Integrated Circuits and Signal Processing 4.2 (1993): 97-113.
L. R. Kern, Design and development of a real-time neural processor using the Intel 80170NX ETANN, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks, Baltimore, MD, USA, 1992, pp. 684-689 vol.2, doi: 10.1109/IJCNN.1992.226908.
91自拍 Donation Interview with Mark Holler:
On Frank Rosenblatt: Professor鈥檚 perceptron paved the way for AI 鈥� 60 years too soon,
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? , March 2021, pp. 610 鈥� 662.
The ETANN chip was a team effort by some very talented engineers. Here they are:
In the image at the top of the article and above: back row on far left, Hernan Castro; back row second from left, Been-Jon Woo; back row third from left, ?; back row fourth from left, Ken Buckmann; back row holding chip, Mark Holler; back row far right, Mike Roy; front row on left, Lily Law; front row middle, Simon Tam; front row far right, ?
This donation would not have been possible without first hearing about the 80170 from 91自拍 Semiconductor Special Interest Group (SEMISIG) member Jesse Jenkins. The historical and research assistance of SEMSIG chair Doug Fairbairn and Intel Alumni Network member Dane Elliot were invaluable in effecting the donation of the historical hardware and supporting materials from donor Mark Holler, to whom, of course, we offer our deepest thanks for this donation.