Real-time camera feed

Hi,

I am trying to use URCaps to process some data coming from a MS Kinect camera to move UR5 in real-time, but I am unsure if URCaps is the right option. My understanding of what URCaps let me do is to create these “blocks” that can do different things, essentially Java that writes URScript to the robot.

How can I use these blocks to achieve my goals? I thought of having a while(true) loop that first reads data from the Kinect and the internal status(where the robot is and where it is moving) and then write it a new command through URscript. The data from the camera will be sent over a localhost port.

And how do I send data between two different “blocks”? Using the dataModel somehow? Also: except for the “generateScript(ScriptWriter writer)”, is there any other way to write commands to the UR5?
Finally: how can I get what the ScriptWriter outputs? Like using the command:
writer.appendLine(“get_actual_joint_positions()”);

I know this is a lot of questions, but I am unsure of the capabilities of URCaps and if it is the right tool to use.

I haven’t done this before. So just a possible solution I think. Create a daemon to receive MS Kinect data and transfer to script program by TCP/IP or RPC protocol. Then what script program does is fetching data from the daemon and move accordingly.

Hi Egil

Using a URCap to implement a daemon would be the right approach.

The daemon could receive the data from the Kinect module, and convert it to the date needed by the UR.
You might implement a simple program node that starts your deamon, and implements a while(True) wherein you use XML-RPC to receive the online data for next motion. Alternatively a Thread that syncs this data, and stores this in a variable, that the main loop reads from. Both could be implemented using the ScriptWriter for a programnode.

The DataModel is not meant for online data. It is meant for saving settings between program changes/saves/loads etc.
Generally the respective Contribution and Service classes should not handle live data, as this affects GUI functionality.
Check out the UR3 Screw sample in this forum (search for “samples”). This generates a longer script code, based on the users configuration of the program node.

Hi,

Thanks for your comments, these were really useful and I think we are moving towards right diretion :slight_smile:
Anyhow, I am facing an exception while deploying a simple C++ daemon within urcap.
I am using sample project povided in the urcap sdk and deployed the .urcap file on UR3 (ursim-3.3.3.292). it was deployed successfully. But, when I see the MyDeamon project, it says “Deamon failed”, the program outputs following messages on terminal:
ERROR [Thread-1] 16:55:16 15/11/16 ControllerSocket.writeBytes(line:1053): Not connected to controller, trying to send string:
restart scb
WARNING [AWT-EventQueue-0] 16:55:21 15/11/16 HTML.wrapTag(line:176): The
text “

The Robot Cannot Proceed with Normal
Operation:


  • Robot: NO
    CONTROLLER
” was already wrapped with tags that
should probably be removed from the translation.

ERROR [ControllerConnectionThread] 16:55:22 15/11/16
HelpLoader.loadHelpSetAccordingToRobotConfiguration(line:136): Problem
finding HelpSet: “gui:on_screen_keyboard is not a part of the loaded
help set”

Could not set the title in the daemon process.
Could not set the title in the daemon process.
Could not set the title in the daemon process.
Could not set the title in the daemon process.

Any hint to solve this issue???

Background:

  1. I am using 64 ubuntu 16
  2. using openjdk-7 (not able to install openjdk-6)
  3. Was NOT able to successfully run the ./install.sh under the “ursim-3.3.3.292/ursim-3.3.3” folder,
    4)However, I can start the UR3 by running script "./start-ursim.sh "
  4. Can run the “Hello world” project

Please let me know if something is not clear.

Regards,

Kashif Dar (On behalf of Egil Utheim)