Tue 25 Jun 2019 14:40 - 15:00 at 224AB - Reasoning and Optimizing ML Models Chair(s): Martin Maas

Convolutional Neural Networks (CNN) are widely used for Deep Learning tasks. CNN pruning is an important method to adapt a large CNN model trained on general datasets to fit a more specialized task or a smaller device. The key challenge is on deciding which filters to remove in order to maximize the quality of the pruned networks while satisfying the constraints. It is time-consuming due to the enormous configuration space and the slowness of CNN training.

The problem has drawn many efforts from the machine learning field, which try to reduce the set of network configurations to explore. This work tackles the problem distinctively from a programming systems perspective, trying to speed up the evaluations of the remaining configurations through computation reuse via a compiler-based framework. We empirically uncover the existence of composability in the training of a collection of pruned CNN models, and point out the opportunities for computation reuse. We then propose composability-based CNN pruning, and design a compression-based algorithm to efficiently identify the set of CNN layers to pre-train for maximizing their reuse benefits in CNN pruning. We further develop a compiler-based framework named Wootz, which, for an arbitrary CNN, automatically generates code that builds a Teacher-Student scheme to materialize composability-based pruning. Experiments show that network pruning enabled by Wootz shortens the state-of-art pruning process by up to 186X while producing significantly better pruning results.

Tue 25 Jun

Displayed time zone: Tijuana, Baja California change

14:00 - 15:30
Reasoning and Optimizing ML ModelsPLDI Research Papers at 224AB
Chair(s): Martin Maas Google
14:00
20m
Talk
An Inductive Synthesis Framework for Verifiable Reinforcement Learning
PLDI Research Papers
He Zhu Rutgers University, USA, Zikang Xiong Purdue University, Stephen Magill , Suresh Jagannathan Purdue University
Media Attached
14:20
20m
Talk
Programming Support for Autonomizing Software
PLDI Research Papers
Wen-Chuan Lee Purdue University, Peng Liu Purdue University, Yingqi Liu Purdue University, USA, Shiqing Ma Purdue University, USA, Xiangyu Zhang Purdue University
14:40
20m
Talk
Wootz: A Compiler-Based Framework for Fast CNN Pruning via Composability
PLDI Research Papers
Hui Guan North Carolina State University, Xipeng Shen North Carolina State University, Seung-Hwan Lim Oak Ridge National Laboratory, USA
Media Attached File Attached
15:00
20m
Talk
Optimization and Abstraction: A Synergistic Approach for Analyzing Neural Network Robustness
PLDI Research Papers
Greg Anderson University of Texas at Austin, USA, Shankara Pailoor University of Texas at Austin, USA, Isil Dillig UT Austin, Swarat Chaudhuri Rice University
Media Attached