The silicon chip, or integrated circuit (IC), is one of humankind鈥檚 most magnificent, complex, and transformative creations.
The IC itself is a silicon sandwich made up of many transistors (little switches) wired together into electrical circuits. Since Robert Noyce and Jack Kilby鈥檚 coinvention of the IC in 1958-59, relentless improvements in transistor density have taken place. You could see this change in spacecraft, aviation, communications, and, most obviously for most of us, in the consumer products of their times. In the early days of ICs, for example, one of the earliest products was the basic four-function 1973 Sears/Bowmar handheld calculator. Sixty years later, the hot consumer product is a voice activated smartphone connected to the "world brain" of the internet鈥攕cience fiction only a generation ago.
It took a long time to get there, but the progress was steady: since Fairchild Semiconductor chemical engineer Gordon Moore first wrote about ICs in 1965, roughly every 18鈥�24 months the number of transistors on an integrated circuit has doubled. The basic process that has allowed this is photolithography, which, if you break it down into its Greek origins means 鈥渨riting on stone with light.鈥� Figure 1 below shows the general idea.
Source:
Light (often ultraviolet) is blasted through a stencil with the patterns IC designers want burned into the chip. Special chemicals are applied before and after to allow this and the process is repeated until a sandwich, made up of many layers, has been built up and the IC is finished and ready for packaging.
Computer designer Gordon Bell once noted that getting the highest speed possible in computers鈥攚hich rely entirely on ICs鈥攊s often about "plumbing and packaging." Since integrated circuits get hotter the faster they run, fast computers have to worry about keeping their chips cool. We can see this limitation echoed this in the observation that several of legendary supercomputer designer Seymour Cray鈥檚 patents were related to ways of keeping his world-beating computer systems cool. For Cray, who always pushed the limits of the possible, keeping the system from burning up was just as important as designing fast circuits.
Cerebras, a well-funded Silicon Valley startup, has developed one of the most ingenious and technically elegant solutions to the problem of both "cramming" (to paraphrase Gordon Moore) more transistors onto a chip while keeping it cool. How far could things be pushed? The answer is astounding: on a huge, single square piece of silicon, about 8.5鈥� on a side, the Cerebras Wafer-Scale Engine (WSE) has 2.6 trillion transistors, which make up 850,000 AI-optimized processing units. (Figure 2.) The size of a transistor in the WSE is 7 nanometers. Human DNA is 2.5 nanometers. There are more transistors鈥攂y far鈥攊n this one Cerebras chip than in all 100,000 computing objects combined in the Museum鈥檚 permanent collection.
Figure 2. The Cerebras Wafer-Scale Engine
What鈥檚 clever about the WSE is that its design and manufacturing largely follow existing industry processes and methods. Their unique genius was in providing a state-of-the-art ecosystem for the WSE to live in鈥攐ne which supplied it with power, cooling, and communications.
To pull this off Cerebras relies not on some exotic new method or technology but on existing semiconductor fabrication processes鈥攋ust modifying them slightly. (The 8.5鈥� on a side footprint, for example, is what鈥檚 left when you take a standard 300 mm wafer and cut it into a square).
Taking these threads鈥攑ackaging, cooling, and transistor density鈥攚hat does this amazing chip look like when packaged up? (Figure 3) The WSE itself is in the center, looking rather delicate among some pretty hard-core hardware. This surrounding technology though is key to the WSE functioning since it both connects the WSE to the outside world through some very serious connector systems, pumps water through cold plates that it touches to keep it cool (Bell and Cray again), and provides a stunning 20 kW of electrical power to supply those trillions of transistors. That, by the way, is enough power to supply about five US homes. That sounds like a lot, but it鈥檚 actually only a few billionths of a watt per transistor. These are the kind of engineering problems you have to solve when you shrink racks of ordinary computers down to a single chip.
The scale, ambition and performance of the Cerebras Wafer-Scale Engine is almost beyond imagining.
Figure 3: Exploded view of the 鈥淓ngine Block鈥� at the heart of the Cerebras CS-2 system. Source: Cerebras Systems.
With all this attention to massive integration (putting everything on one giant chip and packaging it efficiently) a typical Cerebras system is about 1/10th the size of an equivalent computer system using the "ancient ways" of wiring up hundreds of thousands of chips together on separate circuit boards, in separate cabinets, and then wiring those together. When asked why he did not favor using multiple small processors in his supercomputers, legendary computer designer Seymour Cray quipped, 鈥淲hat would you rather have pulling your plow? One large bull or 64 chickens?鈥�
So what are these trillions of transistors in the WSE used for, you're wondering?
The killer app of the WSE for AI is in reducing neural network training time鈥攖he networks that make AI "intelligent." The WSE can take something that takes other computers weeks and get results in minutes.
The result of this gigantic reduction in training time means answers to AI-powered experiments can come in minutes or hours, not days. This speeds up , can model potential , do and work in dozens of new AI-inspired applications that our world will need to face the challenges of the future.
Since so much of the world to come will be shaped by AI-based technologies as time goes by, the WSE is a great milestone to collect, which is why the Museum was particularly happy to bring an example of this amazing device into its permanent collection. The WSE was on display for a limited time at 91自拍.
Blogs like these would not be possible without the generous support of people like you who care deeply about decoding technology. Please consider聽.