|
|
| Online learning for high dimensional data processing: Exact dynamics and phase transitions |
| 2017-02-13
Text Size:
A A A |
| Institute of Theoretical Physics | | Chinese Academy of Sciences | | Key Laboratory of Theoretical Physics | | Seminar | | Title 题目 | Online learning for high dimensional data processing: Exact dynamics and phase transitions | | Speaker 报告人 | Dr. Chuang Wang | | Affiliation 所在单位 | Harvard University, USA | | Date 日期 | 13 February 2017, Monday: 10:30--11:30 | | Venue 地点 | ITP NEW BUILDING 6420 | | Abstract 摘要 | We study the dynamics of an online algorithm for learning a sparse leading eigenvector from samples generated from a spiked covariance model. This algorithm combines the classical Oja's method for online principal component analysis with an element-wise nonlinearity at each iteration to promote sparsity. In the high-dimensional limit, the joint empirical measure of the underlying sparse eigenvector and its estimate provided by the algorithm is shown to converge weakly to a deterministic, measure-valued process. This scaling limit is characterized as the unique solution of a nonlinear PDE, and it provides exact information regarding the asymptotic performance of the algorithm. For example, performance metrics such as the cosine similarity and the misclassification rate in sparse support recovery can be obtained by examining the limiting dynamics. A steady-state analysis of the nonlinear PDE also reveals an interesting phase transition phenomenon. Although our analysis is asymptotic in nature, numerical simulations show that the theoretical predictions are accurate for moderate signal dimensions. Moreover, such analysis framework can be applied to more complicated situations, for example, low-rank subspace tracking problem using partially observations. Similar PDEs/ODEs and phase transition phenomenon are observed. | | Contact Person 联系人 | Hai-Jun Zhou |
 |
 |
|
|
