Advances can enable artificial intelligence in household appliances, improving data security and energy efficiency.
Deep learning is everywhere. This branch of artificial intelligence targets your social networks and provides Google search results. Soon, in-depth studies can also check your vitality or set a thermostat. WITH ONE researchers have developed a system that can bring deep-learning neural networks to new – and much smaller – places, such as small computer chips in portable medical devices, appliances, and the other 250 billion objects that make up the “Internet of Things” (IoT). ).
A system called MCUNet designs unprecedented speeds and dense neural networks accuracy For in-depth learning on IoT devices, despite limited memory and processing power. Technology can easily expand the IoT universe to save energy and improve data security.
The research will be presented at the Neuronal Information Processing Systems Conference next month. The lead author is Ji Lin, who holds a PhD in the Song Han Laboratory from MIT’s Department of Electrical and Computer Engineering. Authors include Han and Yujun Lin of MIT, Wei-Ming Chen of MIT and Taiwan National University, and John Cohn and Chuang Gan of MIT-IBM Watson AI Lab.
The Internet of Things
IoT was born in the early 1980s. Carnegie Mellon University graduate students, including Mike Kazar ’78, connected the Cola-Cola machine to the Internet. The team’s motivation was simple: laziness. They wanted to use computers to confirm that the machine was equipped before making a purchase from the office. It was the world’s first Internet-connected device. “This was considered quite a joke,” says Kazar, now a Microsoft engineer. “No one expected billions of devices on the Internet.”
Since that coke machine, everyday objects have become more and more networked as IoT grows. It goes from portable heart monitors to smart refrigerators that say you have little milk. IoT devices often run on microcontrollers: simple computer chips without an operating system, low processing power, and less than a thousandth of the memory of an ordinary phone. So getting to know models like deep learning makes it difficult to run locally on IoT devices. For complex analyzes, the data collected by the IoT is often sent to the cloud, making it vulnerable to hacking.
“How do we propagate neural networks directly in these small devices? It’s a new area of research that’s getting really hot, ”says Han.“ Companies like Google and ARM are working in that direction. ”There, too.
With MCUNet, the Han team coded two components needed for “very in-depth learning” – the functioning of neural networks in microcontrollers. One component is TinyEngine, an inference engine that directs resource management similar to the operating system. TinyEngine is optimized to run a specific neural network structure, which is selected by other components of MCUNet: TinyNAS, a neural architecture search algorithm.
Signature of the system algorithm code
Designing a deep network for microcontrollers is not easy. Techniques for searching for existing neural architecture include a number of possible network structures based on a predefined template, which are then found to be of high accuracy and low cost. While the method works, it is not the most effective. “It can work pretty well for GPUs or smartphones,” says Line. “But it’s been difficult to apply these techniques directly to small microcontrollers because they’re too small.”
So Lin developed TinyNAS, a method for searching for neural architecture that creates bespoke networks. “The line said,‘ We have a lot of microcontrollers that come in different power and memory sizes. “So we developed the algorithm [TinyNAS] to optimize the search space of different microcontrollers. ”The custom nature of TinyNAS means it can create dense neural networks for a particular microcontroller without unnecessary parameters.“ Then we will deliver the latest efficient model to the microcontroller, ”says Line.
To run this small neural network, a microcontroller also needs a curved inference motor. A typical inference engine has dead weight – instructions for tasks it can rarely perform. The additional code is no problem for a laptop or smartphone, but it can easily overwhelm the microcontroller. “It doesn’t have off-chip memory, and it doesn’t have a disk,” Han says. “Everything put together is just a megabyte of flash, so we need to manage that small resource carefully.” Cue TinyEngine.
Researchers have developed an inference engine in conjunction with TinyNAS. TinyEngine generates the key code needed to run TinyNAS ‘custom neural network. Any dead code is discarded, which reduces compilation time. “We only keep what we need,” Han says. “And since we designed the neural network, we know exactly what we need. That’s the advantage of signing a system algorithm code. ” a depth-to-depth curve that reduces the memory usage point by almost half.After signing the TinyNAS and TinyEngine code, the Han team tested MCUNet.
MCUNet’s first challenge was image classification. The researchers used the ImageNet database to train the system with tagged images, which then tested its ability to classify new ones. In a commercial microcontroller tested, 70.7% of new images were successfully classified by MCUNet – the most advanced neural network in the past and the combination of inference motors was only 54%. “A 1 percent improvement is also considered important,” Line says. “So it’s a huge leap forward for microcontroller settings.”
The team found similar results in the ImageNet tests of three other microcontrollers. In terms of speed and accuracy, MCUNet outperformed the competition for audio and visual “wake-word” tasks, where the user begins to interact with the computer using voice signals (think, “Hey, Siri”) or simply entering a room. The experiments highlight the adaptability to a wide range of MCUNet applications.
The results of promising tests are expected to turn Han into a new industry standard for microcontrollers. “It has great potential,” he says.
Advances “extend the limit of deep neural network design to the computational domain of small energy-efficient microcontrollers,” says Kurt Keutzer, a computer scientist at the University of California, Berkeley who was not involved in the work. MCUNet added that “smart computer visual skills can lead to even the simplest kitchen appliances or enable smarter motion sensors.”
MCUNet can also make IoT devices more secure. “A key advantage is maintaining privacy,” Han says. “You don’t need to transmit data to the cloud.”
Analyzing data locally reduces the risk of personal information being stolen – including personal health data. Han predicts smart watches with MCUNet that not only help users sense heart rate, blood pressure, and oxygen levels, but also help them analyze and understand that information. MCUNet can bring in-depth learning to IoT devices in vehicles with limited Internet access and in rural areas.
In addition, the thin computer footprint of MCUNet becomes a light carbon footprint. “Our big dream is for green AI,” says Han, adding that training large neural networks can burn carbon equal to the lifetime emissions of five cars. An MCUNet microcontroller would require a small portion of that energy. “Our ultimate goal is to enable efficient and low-level AI, fewer computing resources, fewer human resources, and less data,” says Han.
Reference: Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan and Song Han, July 20, 2020, on “MCUNet: Tiny Deep Learning on IoT Devices” devices. Computer science> Computer vision and knowledge of models.