I’m working with a MiR250 and a UR5e robotic arm, which is equipped with a gripper. I want to mount a ZED X One camera on the end-effector to perform manipulation tasks. The whole system is intended to be controlled via ROS2 (Humble) over WiFi. Additionally, I’m considering integrating a ZED Box Orin for processing. I know there is the URCap , should it works for these camera?
My main question is:
How can I connect the ZED X One camera mounted on the robot’s end-effector to my computer via WiFi and exchange data using ROS2?
Any advice or guidance would be greatly appreciated!
probably the easiest would be to add a raspberry pi or some other form of Edge computer near the robot, with a rod node on it to handle this.
The Zed box one seems to be just an NVIDIA Jetson I don’t see why you shouldn’t be able to run a ROS node on it.
the robots control PC is just a normal linux debian computer, so you could also just install a ros node directly on the controller. This is not really recommend as it could potentially interfere with the function of the robot.
use the urcap. BUT URCaps and ROS don’t work together very well at least not directly, the reason being that URCaps generate a bunch of urscript code in the start of any programs created on the robot through the teach pendant. And when you control the robot remotely like trough ROS this ur script code simply won’t get generated.
There are ways around this, but they are all requires knowledge about a lot of different parts of UR and it’s interfacs.
(the easiest but very inflexible)
create any robot program that might be necessary with the teach pendant and the urcap.
The use the “dashboard interface” to load and start each program when they are needed.
(very hard, but alot more flexible)
Create a single robot program that starts interpreter_mode. load and start this using the “dashboard interface”. Because of how the interpreter works you will have access to any script functions created by the urcap. How well these functions are documented and understanderble you’ll have to find out your self.
(I would do this by creating a program on the robot using the urcap and then study the script code this generates.)
Annother crazy ideas
3) (hardest) creating your own URCap exactly for this purpose. You could create URCap background process (daemon) that does this and nothing else.
But I think that the ZED Box can be highly useful for processing data from the ZED X One, significantly reducing bandwidth load, especially in environments without a high-speed, low-latency network connection. By handling data processing locally, it minimizes the need for continuous high-throughput streaming, making it a practical solution.
This approach aligns well with the project’s ultimate goal of creating a digital twin, as it ensures efficient data handling and real-time processing without overloading the network.
I appreciate the various options for integrating URCaps with ROS. I’m particularly interested in exploring the second and third approaches to determine which would be the best fit for my project. I’ll take the time to study both in detail to understand their feasibility and implementation.
Let me know if you are going to discover new things! Thanks a lot!
The ZED X One uses GMSL2 comm i believe. It can only be connected to devices with the CSI port such as the Orin and, depending on the exact model, will need a capture card. Stereolabs does make usb cameras as well. I personally would either go with the onboard urscrypt and urcaps method OR go for the remote controlled ROS2 Humble, in which case all you need is external control urcap to hand over control of ur5e to your orin running ROS2.