Supervisor: Prof. Ketan Rejawat, IIT Kanpur.
In this term report we eproduced and extended the empirical results of “On the Insufficiency of Existing Momentum Schemes for Stochastic Optimization” and ”Accelerating Stochastic Gradient Descent For Least Squares Regression” by Kidambi et al.