ADAM

  ADAM （Adaptive Moment Estimation）是另外一种自适应学习率算法，它结合动量梯度 下降法，在不同参数方向上采用不同学习率，保留前几次迭代的梯度，能够很好 的适应于稀疏数据。(ADAM (Adaptive Moment Estimation) is another adaptive learning rate algorithm, which combines momentum gradient. The descent method, which uses different learning rates in different parameter directions and retains the gradients of previous iterations, is very good. It is suitable for sparse data.)

ADAM.py, 2708 , 2018-12-13