Cornell's RHyME Enables Robots to Learn from Videos

Cornell University researchers have developed a new robotic framework called RHyME—short for Retrieval for Hybrid Imitation under Mismatched Execution—that allows robots to learn tasks by watching a single how-to video. The system uses artificial intelligence to translate human demonstrations into actions that robots can follow. By reducing the need for extensive, step-by-step task instructions, the framework could decrease the training time, energy, and money needed to deploy robotic systems.

The research team, led by doctoral student Kushal Kedia and assistant professor Sanjiban Choudhury, is using what is known as imitation learning. This approach enables robots to learn a sequence of tasks more quickly and helps the machines adapt to real-world environments. The team compared the AI translation process to converting between languages, saying it is similar to translating French to English by converting human demonstrations into robotic actions.

The development comes as part of efforts to improve home robot assistants and other robotic applications, addressing challenges faced when robots lose tools or screws during tasks. Further details on the framework and its implementation are available through Cornell University's publications.



Source: Cornell University

Source Type: College

Source Location: Ithaca

Apr 23, 2025 at 8:30 AM College

Cornell Unveils Soft Robot for Precision Agriculture

Cornell researchers create soft robot for precise plant monitoring and treatment.

Jun 13, 2025 at 2:30 PM College

DoD Halts Novel Tick-Borne Disease Test

Cornell's tick-disease test stopped early by a U.S. Department of Defense order.

Jun 13, 2025 at 8:31 AM College

Cornell to Host Free Summer Outdoor Concerts, Lectures

Cornell offers free outdoor summer concerts and lectures with engaging community events.

Jun 12, 2025 at 6:20 PM College

Cornell Model Predicts Wildfire Impact on Solar Farms

Cornell model predicts wildfire smoke impacts on solar farm energy production.

Jun 12, 2025 at 2:03 PM College

Cornell Study: Trees near White Hall Distract Visitors

Cornell's White Hall trees create a distracting, immersive natural environment.

Jun 11, 2025 at 3:16 PM College

Buffalo Youth Empowered by 4-H Community Program

Youth CAN empowers underprivileged teens in Buffalo through meaningful community projects.

Jun 11, 2025 at 12:00 PM College

Cornell President Outlines Core Values Amid Campus Challenges

Kotlikoff outlined Cornell's values, safety, and legal measures amid challenges.

Jun 10, 2025 at 12:00 PM College

Cornell Major Joins Storied Chimesmasters Tradition

Cornell computer science major selected as new chimesmaster after rigorous audition.

Jun 9, 2025 at 2:30 PM College

Ithaca Vineyard Celebrates Canine Companions

Six Mile Creek Vineyard celebrated a festive pup event with its beloved staff dogs.

Jun 9, 2025 at 11:54 AM winery

Cornell Reunion Welcomes Alumni, Families, and Students

Cornell reunion welcomes alumni, families, and students to reconnect and explore campus.

Jun 9, 2025 at 8:30 AM College