URSim + ROS + Unity

ROS
1

I am new to Universal Robots and robotics in general. My background is in software development, not robotics.

I’ve been asked to develop an application to teach students how to program a UR3 in a virtual environment. Similar to the UR Academy courses but with additional ‘real world’ tasks and within a self-contained application.

I am trying to wrap my head around the ROS implementation for UR and if I can leverage the ROS capabilities of the Unity engine to control a virtual representation of a UR3.

Unity has built in ROS capabilities as described here - Robotics simulation in Unity is as easy as 1, 2, 3! | Unity Blog

I’m trying to understand if I can use UR ROS Driver in URSim to communicate with a virtual UR3 in Unity.

I’ve read through the ReadMe in Git for the UR ROS Driver but my lack of knowledge in robotics and UR tools is resulting in an unclear understanding of how all the pieces fit together.

My end goal is have a student program a robot in Polyscope and have it perform the programmed movements in a Unity scene. I thought the ROS implementation of UR might allow for that.

Can anyone point me in the direction of how URSim/Polyscope can send ROS commands?

Appreciate the help and patience.

Andy

2

If I understand correctly, you want to control the robot completely in polyscope and just visualize the robot in unity?

If that’s correct, you will achieve the full simulation, programming and execution using URSim e.g. inside a VirtualBox.

The ROS driver than would help you visualize the robot inside of ROS. If you don’t want to control it through ROS, you won’t have to use the external_control URCap as described in the driver’s README. Instead, as soon as you start the driver, the robot can be visualized using RViz as the default visualization tool inside of ROS.

For visualizing the robot in Unity; I personally don’t have any experience. The link you posted is (as far as I understand it from quickly skimming over it) about simulating the complete robot inside Unity instead of using URSim. This way, you won’t be able to use any of the polyscope surface, as it is simply not running.

Can anyone point me in the direction of how URSim/Polyscope can send ROS commands?

What kind of ROS commands would you like to send? If I understand your OP correctly, you want to do the full simulation and control inside URSim, so ROS will only be the bridge for visualizing the robot in unity.

3

Thanks for the quick response.

My goal is to teach students the IDE/Programming environment of Polyscope - creating waypoints, stops, timers, etc. It is critical to their learning that they are familiar with and competent in how to program a UR3 via Polyscope.

The intention of introducing Unity is to put a virtual robot in a contextual environment - factory/warehouse - where it could manipulate other objects like palletizing boxes. Very similar to the UR Academy course. I understand Polyscope will show the robot moving based on programmed waypoints but it doesn’t allow the introduction of other objects.

I need a bridge between Polyscope and Unity. I could interpret URScript from Polyscope and use the URscript commands to move a Unity version of the robot but this seems difficult. I thought ROS could be the bridge between a set of Polyscope commands/movements and Unity.

Am I way off base?

4

Hi Andy.

At the last place I worked we used ROS with RVIS (http://wiki.ros.org/rviz) together with the URSIM you can download on the UR+ site to simulate the robot in its working envioment.

I don’t think its a powerful as unity, but it might be easier to get to work as it already a part of ROS.

best regards Casper

EDIT: I forgot to say that you will properly need to use MoveIt also. (https://moveit.ros.org/)

5

Thanks Casper, I’ll look into RViz.

One piece I’m unclear on, will Polyscope (URSim) + the external_control URCap publish ROS messages/commands? Or am I thinking about that all wrong?

6

It might, but I’ve personally only use their Python/cpp based ROS driver.
(GitHub - UniversalRobots/Universal_Robots_ROS_Driver: Universal Robots ROS driver supporting CB3 and e-Series)

Best Regards Casper

7

Even without the URCap, the driver will publish robot data (joint positions, velocities, IO states, etc)

8

For clarity sake, when you say publish are you talking about publish ROS from the Polyscope in URSim?

9

Hey Andy,

I think ROS is overkill for your application. Consider this project from Roman (GitHub - rparak/Unity3D_Robotics_UR: A digital-twin of the Universal Robots UR3 integrated into the Unity3D development platform.). He uses TCP to communicate directly between the controller and Unity (No ROS). The solution will also be more familiar to you since it is based on TCP communication.

Note that his current implementation uses a single loop for receiving and sending messages leading to some limitation. Let me know if you decide to use Romans solution because I have some ideas on how to improve it.

Alex.

10

For visualizing the robot movement, you can use RoboDk:

11

There is an URSIM interface included in realvirtual.io Professional (which is an Asset for Unity).

Here is the documentation: