Signal RAPID: Robot Agentic Programming from Demonstrations

Summary

A preprint introduces RAPID, Robot Agentic Programming from Demonstrations, which generates, verifies and refines robot programs from a single visual human demonstration. The system infers a testable task specification, action primitives and an interactive execution environment from the demonstration. It uses an object-centric relational program representation so the result can be reused beyond the demonstrated setting. Action primitives are expressed as trajectory-optimization programs and composed through relational constraints. RAPID was evaluated in simulation on eight contact-rich nonprehensile tasks and on prehensile tasks in the LIBERO-Pro benchmark. It was also deployed on a real Franka arm for all eight nonprehensile tasks.