6-DOF Robotic Arm + RL Digital Twin
Physical 6-DOF arm with reinforcement learning trained in a digital twin simulation
A full-stack robotics project: design and build a physical 6 degree-of-freedom robotic arm, then train a controller using reinforcement learning in a digital twin simulation before deploying to hardware.
Hardware
The arm was designed and assembled from scratch with:
- Joints: rotating base, shoulder, elbow, wrist pitch/yaw, gripper
- Actuators: servo motors (high-torque for shoulder/elbow) + stepper motors for precision joints
- Controller: ESP32 microcontroller with custom firmware for joint position control and sensor integration
- Perception: OpenCV-based camera system for object localization
Digital Twin & RL Training
A physics-accurate simulation of the arm is used for RL training, eliminating the need to run thousands of training episodes on physical hardware (which causes wear and is slow).
The RL agent learns:
- Object localization — given an image, determine where the target object is
- Grasping policy — plan a joint trajectory to reach and grasp the object
- Task execution — complete manipulation tasks (pick-and-place, sorting)
Training uses image observations + proprioceptive joint state as input, with reward shaped around grasp success and trajectory smoothness.
Sim-to-Real Transfer
The digital twin is kept faithful to the physical system (matched inertia, joint limits, actuator dynamics) to minimize the sim-to-real gap.
Stack: Python · PyTorch · StableBaselines3 · OpenCV · ESP32 (C++) · Isaac Sim (simulation)
Status: Active (Jan 2024 – Present)