Non-Convex Optimization: RMSProp Based Optimization for Long Short-Term Memory Network

Yan, Jianzhi

Publication

Non-Convex Optimization: RMSProp Based Optimization for Long Short-Term Memory Network

Yan, Jianzhi

Keywords

Nonconvex programming
Long Short-Term Memory (LSTM)
Back propagation (Artificial intelligence)
RMSProp optimization

Publication Date

2020-05

Abstract

This project would give a comprehensive picture of non-convex optimization for deep learning, explain in details about Long Short-Term Memory (LSTM) and RMSProp. We start by illustrating the internal mechanisms of LSTM, like the network structure and backpropagation through time (BPTT). Then introducing RMSProp optimization, some relevant mathematical theorems and proofs in those sections, which give a clear picture of how RMSProp algorithm is helpful to escape the saddle point. After all the above, we apply it with LSTM with RMSProp for the experiment; the result would present the efficiency and accuracy, especially how our method beat traditional strategy in non-convex optimization.

Non-Convex Optimization: RMSProp Based Optimization for Long Short-Term Memory Network

Journal Title

Keywords

Readers/Advisors

Journal Title

Term and Year

Publication Date

Book Title

Publication Volume

Publication Issue

Publication Begin

Publication End

Number of pages

Collections

Files

Research Projects

Organizational Units

Journal Issue

URI

Abstract

Citation

DOI

Description

Accessibility Statement

Embedded videos