Compiling KB-Sized Machine Learning Models to Tiny IoT Devices
Recent advances in machine learning (ML) have produced KiloByte-size models that can directly run on constrained IoT devices. This approach avoids expensive communication between IoT devices and the cloud, thereby enabling energy-efficient real-time analytics. However, ML models are expressed typically in floating-point, and IoT hardware typically does not support floating-point. Therefore, running these models on IoT devices requires simulating IEEE-754 floating-point using software, which is very inefficient.
We present SeeDot, a domain-specific language to express ML inference algorithms and a compiler that compiles SeeDot programs to fixed-point code that can efficiently run on constrained IoT devices. We propose 1)~a novel compilation strategy that reduces the search space for some key parameters used in the fixed-point code, and 2)~new efficient implementations of expensive operations. SeeDot compiles state-of-the-art KB-sized models to various microcontrollers and low-end FPGAs. We show that SeeDot outperforms 1) software emulation of floating-point (Arduino), 2) high-bitwidth fixed-point (MATLAB), 3) post-training quantization (TensorFlow-Lite), and 4) floating- and fixed-point FPGA implementations generated using high-level synthesis tools.
Mon 24 Jun
|08:45 - 09:05|
Michael VollmerIndiana University, USA, Chaitanya KoparkarIndiana University, Mike RaineyIndiana University, USA, Laith SakkaPurdue University, Milind KulkarniPurdue University, Ryan R. NewtonIndiana UniversityDOI Authorizer link Pre-print Media Attached
|09:05 - 09:25|
Daniel J. FremontUniversity of California at Berkeley, USA, Tommaso DreossiUniversity of California at Berkeley, Shromona GhoshUniversity of California at Berkeley, USA, Xiangyu YueUniversity of California at Berkeley, USA, Alberto L. Sangiovanni-VincentelliUniversity of California at Berkeley, USA, Sanjit SeshiaUC BerkeleyMedia Attached
|09:25 - 09:45|
Sridhar GopinathMicrosoft Research, India, Nikhil GhanatheMicrosoft Research, India, Vivek SeshadriMicrosoft Research, India, Rahul SharmaMicrosoft ResearchLink to publication DOI Media Attached