Tue 25 Jun 2019 14:40 - 15:00 at 224AB - Reasoning and Optimizing ML Models Chair(s): Martin Maas

Convolutional Neural Networks (CNN) are widely used for Deep Learning tasks. CNN pruning is an important method to adapt a large CNN model trained on general datasets to fit a more specialized task or a smaller device. The key challenge is on deciding which filters to remove in order to maximize the quality of the pruned networks while satisfying the constraints. It is time-consuming due to the enormous configuration space and the slowness of CNN training.

The problem has drawn many efforts from the machine learning field, which try to reduce the set of network configurations to explore. This work tackles the problem distinctively from a programming systems perspective, trying to speed up the evaluations of the remaining configurations through computation reuse via a compiler-based framework. We empirically uncover the existence of composability in the training of a collection of pruned CNN models, and point out the opportunities for computation reuse. We then propose composability-based CNN pruning, and design a compression-based algorithm to efficiently identify the set of CNN layers to pre-train for maximizing their reuse benefits in CNN pruning. We further develop a compiler-based framework named Wootz, which, for an arbitrary CNN, automatically generates code that builds a Teacher-Student scheme to materialize composability-based pruning. Experiments show that network pruning enabled by Wootz shortens the state-of-art pruning process by up to 186X while producing significantly better pruning results.

Tue 25 Jun

pldi-2019-papers
14:00 - 15:30: PLDI Research Papers - Reasoning and Optimizing ML Models at 224AB
Chair(s): Martin MaasGoogle
pldi-2019-papers14:00 - 14:20
Talk
He ZhuGalois, Inc., Zikang XiongPurdue University, Stephen Magill, Suresh JagannathanPurdue University
Media Attached
pldi-2019-papers14:20 - 14:40
Talk
Wen-Chuan LeePurdue University, Peng LiuPurdue University, Yingqi LiuPurdue University, USA, Shiqing MaPurdue University, USA, Xiangyu ZhangPurdue University
pldi-2019-papers14:40 - 15:00
Talk
Hui GuanNorth Carolina State University, Xipeng ShenNorth Carolina State University, Seung-Hwan LimOak Ridge National Laboratory, USA
Media Attached File Attached
pldi-2019-papers15:00 - 15:20
Talk
Greg AndersonUniversity of Texas at Austin, USA, Shankara PailoorUniversity of Texas at Austin, USA, Isil DilligUT Austin, Swarat ChaudhuriRice University
Media Attached