A collection of papers, codes and resources of learning from video in robotics.

Recent Advances in Imitation Learning from Observation IJCAI 2019

Recent advances in robot learning from demonstration Annual Review of Control, Robotics, and Autonomous Systems 2020

What Matters in Learning from Offline Human Demonstrations for Robot Manipulation arxiv2021 | website |code

RoboTube: Learning Household Manipulation from Human Videos with Simulated Twin Environments RSS2022 | video

Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations RSS 2022

R3M: A Universal Visual Representation for Robot Manipulation arXiv 2022 | code

Adversarial Skill Networks: Unsupervised Robot Skill Learning from Videos ICRA 2020 | website

Reinforcement Learning with Action-Free Pre-Training from Videos ICML2022 | code

Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos arXiv 2022

BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning CORL 2022 | website

DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video CORL 2021

DexMV: Imitation Learning for Dexterous Manipulation from Human Videos arxiv2022 | website | code

Learning Agile Robotic Locomotion Skills by Imitating Animals RSS 2020 | website | code

SFV: Reinforcement Learning of Physical Skills from Videos ACM Transactions on Graphics 2018 | code

Graph-Structured Visual Imitation CoRL 2019

Learning to Manipulate Tools by Aligning Simulation to Video Demonstration RAL 2022

Learning Periodic Tasks from Human Demonstrations arxiv2021

Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on YouTube RSS 2022

You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration arxiv2022

Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller NeurIPS 2019| website | code

Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation ICML 2016 | code

Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos RSS 2021 | website

One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks arxiv 2018

One-shot imitation from observing humans via domain-adaptive meta-learning RSS 2018

Visual Imitation Learning: Generalization, Perceptual Grounding, and Abstraction RSS Workshop 2020

Imitation Learning Tutorial ICML 2018

Model-Based Visual Imitation Learning CVPR 2021 - Franziska Meier

Workshop on Learning from Diverse, Offline Data RSS2022