Imagine a world where you could train a robot to perform a task, upload it to the internet, and have anyone download and run it on a robot in their own home. This vision is becoming closer to reality thanks to a groundbreaking project that is not only pushing the boundaries of robotics but also making it more accessible to the average person.

Researchers recently deployed advanced models on a robot named Stretch, equipped with an iPhone and a camera, to test their abilities in new environments without additional tweaks. While the initial completion rate was a respectable 74.4%, they were able to boost it to an impressive 90% success rate by leveraging OpenAI’s cutting-edge GPT-4o LLM model. By feeding images from the iPhone and the robot’s camera to GPT-4o and resetting the robot if the task was deemed unsuccessful, the researchers achieved remarkable results.

One of the challenges faced by roboticists is the discrepancy between lab-controlled environments and real-world scenarios. Mohit Shridhar, a research scientist specializing in robotic manipulation, stresses the importance of research that helps robots behave reliably in diverse settings. “The true goal of robotics is to get a robot to work seamlessly in a random house,” he asserts.

This project not only demonstrated the capabilities of robots in varied environments but also laid the foundation for future utility robotics models. Shafiullah, one of the researchers involved, envisions a future where teaching robots new skills requires minimal effort. “The dream we’re pursuing is to enable anyone to train a robot online and deploy it in their home effortlessly,” he explains.