“Shift your Expectations – Covariate Shift in Reinforcement Learning“ – A Talk by Prof. Dr. Alexander Smola, Director at Amazon Web Services, USA
On 20th of January, Prof. Dr. Alexander Smola, Director at Amazon Web Services, gave a talk called “Shift your Expectations – Covariate Shift in Reinforcement Learning“.
In the lecture, he informed about covariate shift, its correction and how it applies to Reinforcement Learning algorithms. Reinforcement Learning is particularly sensitive to this phenomenon since a change in distribution is the direct result of an updated policy. That is, any update step in Reinforcement Learning immediately results in a shift and it requires careful correcting. In his talk, Prof. Dr. Smola has provided an example of how this can lead to significant improvement in the performance of policy gradient algorithms, simply by making better use of the off-policy replay buffer. A second application illustrates how this applies to meta-learning, yielding a principled alternative to Meta - Reinforcement Learning heuristics.
His talk was based on joint work with Rasool Fakoor, Pratik Chaudhari, and Stefano Soatto.
Prof. Dr. Alexander Smola is a digital pioneer in the field of Machine Learning and professor at Carnegie Mellon University. He earned his M.S. degree in physics Technical University Munich and his Ph.D. from Technical University Berlin.