研究生学术周末学术报告

【题    目】Towards Understanding Residual and Dilated Dense Neural Networks

【主讲人】张世华 教授，中国科学院数学与系统科学研究院

【时   间】2019-12-07（周六） 14:00-15：40

【地   点】南开大学津南校区人工智能学院南楼327

Abstract

Convolutional Neural Network (CNN) and its variants have led to many state-of-art results in various fields. However, a clear theoretical understanding about them is still lacking. Recently, Multi-Layer Convolutional Sparse Coding (ML-CSC) has been proposed and proved to equal such simply stacked networks (plain networks). Inspired by this scheme, we propose the Residual Convolutional Sparse Coding (Res-CSC) model and Mixed-Scale Dense Convolutional Sparse Coding (MSD-CSC) model, which have close relationship with the Residual neural network (ResNet) and Mixed-Scale (Dilated) Dense neural network (MSDNet), respectively. Mathematically, both the ResNet and MSDNet are special cases of Res-CSC and MSD-CSC, respectively. Moreover, we also find a theoretical interpretation of the dilated convolution and dense connection operations in the MSDNet by analyzing the MSD-CSC model, which gives a clear mathematical understanding about them. We implement the Iterative Soft Thresholding Algorithm (ISTA) and its fast version to solve Res-CSC and MSD-CSC without adding extra parameters. At last, extensive numerical experiments and comparison with competing methods demonstrate their effectiveness.

张世华，中国科学院数学与系统科学研究院研究员、中国科学院随机复杂结构与数据科学重点实验室副主任、中国科学院大学岗位教授。主要从事优化、统计、机器学习与生物信息学交叉研究，主要成果发表在Advanced Science、Nature Communications、Nucleic Acids Research、Bioinformatics、IEEE TPAIM、IEEE TKDE、IEEE TFS、AoAS等杂志。目前担任BMC Genomics等杂志编委。曾荣获中国青年科技奖、国家自然科学基金优秀青年基金。