TalkRL: Reinforcement Learning Interviews

Robert Tjarko Lange is a PhD student working at the Technical University Berlin.Featured ReferencesLearning not to learn: Nature versus nurture in silicoLange, R. T., & Sprekeler, H. (2020)On Lottery Tickets and Minimal Task Representations in Deep Reinforcement LearningVischer, M. A., Lange, R. T., & Sprekeler, H. (2021). Semantic RL with Action Grammars: Data-Efficient Learning of Hierarchical Task AbstractionsLange, R. T., & Faisal, A. (2019).MLE-Infrastructure on GithubAdditional References RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning, Duan et al 2016 Learning to reinforcement learn, Wang et al 2016 Decision Transformer: Reinforcement Learning via Sequence Modeling, Chen et al 2021

We hear about the idea of PERLS and why its important to talk about. Political Economy of Reinforcement Learning (PERLS) Workshop at NeurIPS 2021 on Tues Dec 14th  NeurIPS 2021

Amy Zhang is a postdoctoral scholar at UC Berkeley and a research scientist at Facebook AI Research. She will be starting as an assistant professor at UT Austin in Spring 2023. Featured References Invariant Causal Prediction for Block MDPs Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, Doina Precup Multi-Task Reinforcement Learning with Context-based Representations Shagun Sodhani, Amy Zhang, Joelle Pineau MBRL-Lib: A Modular Library for Model-based Reinforcement Learning Luis Pineda, Brandon Amos, Amy Zhang, Nathan O. Lambert, Roberto Calandra Additional References  Amy Zhang - Exploring Context for Better Generalization in Reinforcement Learning @ UCL DARK  ICML 2020 Poster session: Invariant Causal Prediction for Block MDPs  Clare Lyle - Invariant Prediction for Generalization in Reinforcement Learning @ Simons Institute 

Xianyuan Zhan is currently a research assistant professor at the Institute for AI Industry Research (AIR), Tsinghua University.  He received his Ph.D. degree at Purdue University. Before joining Tsinghua University, Dr. Zhan worked as a researcher at Microsoft Research Asia (MSRA) and a data scientist at JD Technology.  At JD Technology, he led the research that uses offline RL to optimize real-world industrial systems. Featured ReferencesDeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement LearningXianyuan Zhan, Haoran Xu, Yue Zhang, Yusen Huo, Xiangyu Zhu, Honglei Yin, Yu Zheng

Eugene Vinitsky is a PhD student at UC Berkeley advised by Alexandre Bayen. He has interned at Tesla and Deepmind. Featured ReferencesA learning agent that acquires social norms from public sanctions in decentralized multi-agent settingsEugene Vinitsky, Raphael Köster, John P. Agapiou, Edgar Duéñez-Guzmán, Alexander Sasha Vezhnevets, Joel Z. LeiboOptimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RLEugene Vinitsky, Nathan Lichtle, Kanaad Parvate, Alexandre BayenLagrangian Control through Deep-RL: Applications to Bottleneck DecongestionEugene Vinitsky; Kanaad Parvate; Aboudy Kreidieh; Cathy Wu; Alexandre Bayen 2018The Surprising Effectiveness of PPO in Cooperative Multi-Agent GamesChao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, Yi WuAdditional ReferencesSUMO: Simulation of Urban MObility

Dr. Jess Whittlestone is a Senior Research Fellow at the Centre for the Study of Existential Risk and the Leverhulme Centre for the Future of Intelligence, both at the University of Cambridge.Featured ReferencesThe Societal Implications of Deep Reinforcement LearningJess Whittlestone, Kai Arulkumaran, Matthew CrosbyArtificial Canaries: Early Warning Signs for Anticipatory and Democratic Governance of AICarla Zoe Cremer, Jess WhittlestoneAdditional References CogX: Cutting Edge: Understanding AI systems for a better AI policy, featuring Jack Clark and Jess Whittlestone

Dr Aleksandra Faust is a Staff Research Scientist and Reinforcement Learning research team co-founder at Google Brain Research.Featured ReferencesReinforcement Learning and Planning for Preference Balancing Tasks, Faust 2014Learning Navigation Behaviors End-to-End with AutoRLHao-Tien Lewis Chiang, Aleksandra Faust, Marek Fiser, Anthony FrancisEvolving Rewards to Automate Reinforcement LearningAleksandra Faust, Anthony Francis, Dar MehtaEvolving Reinforcement Learning Algorithms John D Co-Reyes, Yingjie Miao, Daiyi Peng, Esteban Real, Quoc V Le, Sergey Levine, Honglak Lee, Aleksandra FaustAdversarial Environment Generation for Learning to Navigate the WebIzzeddin Gur, Natasha Jaques, Kevin Malta, Manoj Tiwari, Honglak Lee, Aleksandra FaustAdditional References AutoML-Zero: Evolving Machine Learning Algorithms From Scratch, Esteban Real, Chen Liang, David R. So, Quoc V. Le 

Sam Ritter is a Research Scientist on the neuroscience team at DeepMind.Featured ReferencesUnsupervised Predictive Memory in a Goal-Directed Agent (MERLIN)Greg Wayne, Chia-Chun Hung, David Amos, Mehdi Mirza, Arun Ahuja, Agnieszka Grabska-Barwinska, Jack Rae, Piotr Mirowski, Joel Z. Leibo, Adam Santoro, Mevlana Gemici, Malcolm Reynolds, Tim Harley, Josh Abramson, Shakir Mohamed, Danilo Rezende, David Saxton, Adam Cain, Chloe Hillier, David Silver, Koray Kavukcuoglu, Matt Botvinick, Demis Hassabis, Timothy LillicrapMeta-RL without forgetting:  Been There, Done That: Meta-Learning with Episodic RecallSamuel Ritter, Jane X. Wang, Zeb Kurth-Nelson, Siddhant M. Jayakumar, Charles Blundell, Razvan Pascanu, Matthew BotvinickMeta-Reinforcement Learning with Episodic Recall: An Integrative Theory of Reward-Driven Learning, Samuel Ritter 2019Meta-RL exploration and planning: Rapid Task-Solving in Novel EnvironmentsSam Ritter, Ryan Faulkner, Laurent Sartran, Adam Santoro, Matt Botvinick, David RaposoSynthetic Returns for

Thomas Krendl Gilbert is a PhD student at UC Berkeley’s Center for Human-Compatible AI, specializing in Machine Ethics and Epistemology.Featured ReferencesHard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical CommitmentsRoel Dobbe, Thomas Krendl Gilbert, Yonatan MintzMapping the Political Economy of Reinforcement Learning Systems: The Case of Autonomous VehiclesThomas Krendl GilbertAI Development for the Public Interest: From Abstraction Traps to Sociotechnical RisksMcKane Andrus, Sarah Dean, Thomas Krendl Gilbert, Nathan Lambert and Tom ZickAdditional References Political Economy of Reinforcement Learning Systems (PERLS) The Law and Political Economy (LPE) Project The Societal Implications of Deep Reinforcement Learning, Jess Whittlestone, Kai Arulkumaran, Matthew Crosby Robot Brains Podcast: Yann LeCun explains why Facebook would crumble without AI

Professor Marc G. Bellemare is a Research Scientist at Google Research (Brain team), An Adjunct Professor at McGill University, and a Canada CIFAR AI Chair.Featured ReferencesThe Arcade Learning Environment: An Evaluation Platform for General AgentsMarc G. Bellemare, Yavar Naddaf, Joel Veness, Michael BowlingHuman-level control through deep reinforcement learningVolodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg & Demis HassabisAutonomous navigation of stratospheric balloons using reinforcement learningMarc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda & Ziyu WangAdditional References CAIDA Talk: A tour of distributional reinforcement learning November 18, 2020 - Marc G. Bellemare Amii AI Seminar

Robert Osazuwa Ness is an adjunct professor of computer science at Northeastern University, an ML Research Engineer at Gamalon, and the founder of AltDeep School of AI.  He holds a PhD in statistics.  He studied at Johns Hopkins SAIS and then Purdue University.References Altdeep School of AI, Altdeep on Twitch, Substack, Robert Ness Altdeep Causal Generative Machine Learning Minicourse, Free course  Robert Osazuwa Ness on Google Scholar Gamalon Inc Causal Reinforcement Learning talks, Elias Bareinboim The Bitter Lesson, Rich Sutton 2019 The Need for Biases in Learning Generalizations, Tom Mitchell 1980 Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics, Kansky et al 2017

Marlos C. Machado on Arcade Learning Environment Evaluation, Generalization and Exploration in RL, Eigenoptions, Autonomous navigation of stratospheric balloons with RL, and more!

Nathan Lambert on Model-based RL, Trajectory-based models, Quadrotor control, Hyperparameter Optimization for MBRL, RL vs PID control, and more!

Kai Arulkumaran on AlphaStar and Evolutionary Computation, Domain Randomisation, Upside-Down Reinforcement Learning, Araya, NNAISENSE, and more!

Michael Dennis on Human-Compatible AI, Game Theory, PAIRED, ARCTIC, EPIC, and lots more!

Roman Ring discusses the Research Engineer role at DeepMind, StarCraft II, AlphaStar, his bachelor's thesis, JAX, Julia, IMPALA and more!

Shimon Whiteson on his WhiRL lab, his work at Waymo UK, variBAD, QMIX, co-operative multi-agent RL, StarCraft Multi-Agent Challenge, advice to grad students, and much more!

Aravind Srinivas on his work including CPC v2, RAD, CURL, and SUNRISE, unsupervised learning, teaching a Berkeley course, and more!

Taylor Killian on the latest in RL for Health, including Hidden Parameter MDPs, Mimic III and Sepsis, Counterfactually Guided Policy Transfer and lots more!

Nan Jiang takes us deep into Model-based vs Model-free RL, Sim vs Real, Evaluation & Overfitting, RL Theory vs Practice and much more!