RoMA: Robotic 3D Printing and Augmented Reality Combine in Interactive Fabrication Platform

Huaishu Peng

We often see robotics and 3D printing combined, as well as 3D printing and augmented reality (AR). But Cornell University researcher Huaishu Peng, whose 3D printing work has made headlines multiple times, has been working on a project that combines all three. We see a wide variety of really interesting and innovative research and projects in this field, but I can honestly say that the Robotic Modeling Assistant (RoMA) created by Peng and his team is one of the coolest I’ve come across.

According to Peng’s website, he is interested in the technical aspects of human-computer interaction (HCI), and designs software and hardware systems to enable 3D modeling with interactive experiences, as well as making functional objects using custom fabrication machines.

Peng wrote, “I envision that in the future (1) people will design both the form and the function of everyday objects and (2) a personal fabrication machine will construct not only the 3D appearance, but also the interactivity of its prints.“

Talk about interactive – the RoMA is a fabrication system that gives users a hands-on, in-situ 3D modeling experience, using a robotic arm 3D printer and an AR CAD editor.

Peng, together with fellow Cornell researchers Jimmy Briggs, Cheng-Yao Wang, and Kevin Guo; Joseph Kider with the University of Central Florida; Stefanie Mueller with MIT CSAIL; Patrick Baudisch from the Hasso-Plattner Institute; and Cornell’s François Guimbretière, wrote a paper on the RoMA.

The abstract reads, “We present the Robotic Modeling Assistant (RoMA), an interactive fabrication system providing a fast, precise, hands-on and in-situ modeling experience. As a designer creates a new model using RoMA AR CAD editor, features are constructed concurrently by a 3D printing robotic arm sharing the same design volume. The partially printed physical model then serves as a tangible reference for the designer as she adds new elements to her design. RoMA’s proxemics-inspired handshake mechanism between the designer and the 3D printing robotic arm allows the designer to quickly interrupt printing to access a printed area or to indicate that the robot can take full control of the model to finish printing. RoMA lets users integrate real-world constraints into a design rapidly, allowing them to create well-proportioned tangible artifacts or to extend existing objects. We conclude by presenting the strengths and limitations of our current design.”

Basically, as a designer is using RoMA’s AR CAD editor to draw a new 3D model in the air, a 3D printing robotic arm is building features to augment the model at the same time, in the same design volume.

Then, the partially 3D printed model can act as the designer’s physical point of reference while they continue to add elements to the design.

According to the paper, “To use the RoMA system, a designer wears an Augmented Reality (AR) headset and starts designing inside the print volume using a pair of AR controllers. As soon as a design feature is completed, the RoMA robotic arm prints the new feature onsite, starting in the back half of the design volume. At any time, the designer can bring printed features into the front half of the design volume for use as a physical reference. As she does so, the robot updates its schedule and prints another available part of the model. Once she finishes a design, the designer steps back, allowing the robotic system to take full control of the build platform to finish printing.”

So while it may appear to an onlooker that the designer is pointing the AR controller at nothing, they are really designing a 3D model on the rotating platform below the robotic arm. Then, the arm will 3D print each completed design feature in what appears to be mid-air, around the model that only the designer wearing the headset can see.

Want to build a stand for your model jet, a garage for your LEGO vehicle, or a teapot with a finger hole perfectly designed to fit your finger? RoMA can get the job done. It’s almost like a 3D printing pen, but on a much larger scale, with AR technology and a robotic arm controlling the 3D printing process.

Augmented reality interaction

RoMA users are able to, according to the project page, “integrate real-world constraints into a design rapidly, allowing them to create well-proportioned tangible artifacts,” and even extend an object through in-situ fabrication.

The system includes a ceiling-mounted Adept S850 6DOF robotic arm 3D printer, a rotating platform, and an AR headset with cutter and indicator controllers. In terms of software, RoMA has:

• End-to-end pipeline which integrates AR and robot control
• Custom AR CAD editor
• Proxemics-inspired handshake mechanism, which supports human-robot interaction

The custom AR modeling tool emphasizes interactive design, similar to SketchUp, and is deeply integrated with Rhino CAD modeling software.

To begin the process, the designer needs to stay close to the rotating build platform, which is kept immobile by the 3D printing system. The system then 3D prints the part of the model that’s located in the back half of the platform.

To bring the model forward, all the designer has to do is touch the platform’s handle and rotate it.

The robot arm will automatically park away from the user, until the designer steps away. Then, the robotic fabricator is free to “take the full control of the platform” and complete the build.

Any strings left behind from the robotic arm’s 3D printing job can be easily removed with the system’s cutter controller.

Discuss this and other 3D printing topics at 3DPrintBoard.com or share your thoughts below.