# Developing ball balancing with UR3e in Simulation? **URL:** \ **Category:** Technical Questions\ **Created:** [September 25, 2025, 6:59pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276 "2025-09-25T18:59:52Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![fuzail-hamid.banday](https://avatars.discourse-cdn.com/v4/letter/f/3d9bf3/32.png) [@fuzail-hamid.banday](https://forum.universal-robots.com/u/fuzail-hamid.banday)\ **Post date:** [September 25, 2025, 6:59pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/1 "2025-09-25T18:59:52Z")
Hi everyone, been using the UR3E for the past couple months and working on a simple ball balancing platform. There’s a camera which detects the ball and an image processing pipeline gets the XY coordinates and sends them to a PID controller which then controls the robot with the RTDE library. I want to know if it’s possible to do this development in the UR sim? Basically I need to attach a square platform 30 x 30 cm and have a ball on it and then I need real time access to the coordinates of that ball XY. So for this to work, the simulator needs to take physics into account and gravity. I want to know if it’s possible with any simulator? I want to be able to control the robot with a script on the polyscope, so like have a simulated version of a polyscope which I already have in a docker, but that doesn’t really have a physical simulator for the robot. How shall I go about this? Appreciate any suggestions thank you! ---
**Author:** ![2383120](https://avatars.discourse-cdn.com/v4/letter/2/a4c791/32.png) [@2383120](https://forum.universal-robots.com/u/2383120)\ **Post date:** [September 25, 2025, 7:29pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/2 "2025-09-25T19:29:51Z")
i am trying to do something similar which platform are you using. I am using unity for real time simulation ---
**Author:** ![fuzail-hamid.banday](https://avatars.discourse-cdn.com/v4/letter/f/3d9bf3/32.png) [@fuzail-hamid.banday](https://forum.universal-robots.com/u/fuzail-hamid.banday)\ **Post date:** [September 25, 2025, 8:09pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/3 "2025-09-25T20:09:56Z")
Let me see! What do you mean? I dont understand the question ---
**Author:** ![2383120](https://avatars.discourse-cdn.com/v4/letter/2/a4c791/32.png) [@2383120](https://forum.universal-robots.com/u/2383120)\ **Post date:** [September 25, 2025, 8:22pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/4 "2025-09-25T20:22:26Z")
i am using unity to simulate real time positiion of object using smart camera the camera send coordinates to robot and robot send sto unity to simulate its position ---
**Author:** ![fuzail-hamid.banday](https://avatars.discourse-cdn.com/v4/letter/f/3d9bf3/32.png) [@fuzail-hamid.banday](https://forum.universal-robots.com/u/fuzail-hamid.banday)\ **Post date:** [September 26, 2025, 9:30pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/5 "2025-09-26T21:30:47Z")
How did you connect unity with the robot ursim? Are you using the official docker ursim? How did you link the polyscope with unity exactly? ---
**Author:** ![2383120](https://avatars.discourse-cdn.com/v4/letter/2/a4c791/32.png) [@2383120](https://forum.universal-robots.com/u/2383120)\ **Post date:** [September 29, 2025, 9:05am UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/6 "2025-09-29T09:05:36Z")
Using remote connection and urscript to communicate with robot ---
**Author:** ![2383120](https://avatars.discourse-cdn.com/v4/letter/2/a4c791/32.png) [@2383120](https://forum.universal-robots.com/u/2383120)\ **Post date:** [September 29, 2025, 10:12am UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/7 "2025-09-29T10:12:05Z")
I’ve been tinkering with UR5 control and just wrapped up a: a Unity-based VR setup that lets you teleoperate the UR5 arm using a Meta Quest controller. It handles real-time pose tracking, gripper actions for picking/placing objects, and smooth joint interpolation via TCP to a Python RTDE server. Super intuitive for immersive robot ops! If you’re hiring for roles in robotics software, VR/AR integration, or automation engineering, I’d love to chat – hit me up with leads or openings. Open to remote/full-time gigs! ---
**Author:** ![fuzail-hamid.banday](https://avatars.discourse-cdn.com/v4/letter/f/3d9bf3/32.png) [@fuzail-hamid.banday](https://forum.universal-robots.com/u/fuzail-hamid.banday)\ **Post date:** [September 29, 2025, 8:20pm UTC](https://forum.universal-robots.com/t/developing-ball-balancing-with-ur3e-in-simulation/41276/8 "2025-09-29T20:20:37Z")
I do not understand how you set up Unity Simulation env. Is there any guide you followed? I am talking about a fully simulated set up so the polyscope which comes with the URSIM docker which you view with VNC in a browser. How do you connect that with your unity and what kind of set do you have in unity?