RoboticsAutosterea.comTalk to us
Robotics
Autosterea.comTalk to us

R&D journal

Code meets the real world.

After each day in the lab, we write down what we did and what we learned. The photos and videos are real.

Latest day: RSS feed

Week of Nov 2 – 8, 2026

The eye learns to handle the real, messy view

The robot's 'eye' had only ever practised on tidy simulated pictures, so we rebuilt the practice world to be harder than real life — an angled, roving camera, several cubes at once, and random clutter to ignore — and retrained it overnight on our own PC. It now reads the real camera's full picture, with nothing cropped out, finds every cube and steps over almost all the clutter.

1 day

  1. The retrained finder on pictures it had never seen — now with several cubes per picture and clutter around them. The crosses are the model's guesses, the circles the true spots; they still land on top of each other.Photo: Claude (MuJoCo simulation)

    Teaching the cube-finder to read the real, messy camera view

    Our self-taught cube-finder worked beautifully on tidy simulated pictures — but the real camera looks down at an angle, sees several cubes at once, and a cluttered workbench all around them. So we rebuilt the practice world to be harder than real life: a camera that roves and tilts, up to several cubes per picture, and random clutter objects thrown in to ignore. We retrained the model overnight on our own PC. It now reads the real camera's full picture — nothing cropped out — finds every cube, and steps over almost all the clutter.

    • simulation
    • machine vision
    • sim-to-real
    • domain randomisation
    • clutter

Week of Oct 26 – Nov 1, 2026

A website, and the move into simulation

We put robotics.autosterea.com online — built from this journal — and cleared the first cubes with the real arm. Then we gave the arm a twin inside a physics simulator and used it to train the robot's first 'eye': a small model, taught entirely on practice images the simulator labelled itself, that now finds the cubes in a picture to within a few pixels.

3 days

  1. 0:08 video
    Our arm, rebuilt in a physics simulator, running a reach-and-approach over the cubes — tested entirely in software, where a mistake costs nothing.Video: Claude (MuJoCo simulation)

    Teaching the arm in a simulator, and a cube-finder that trained itself

    Training a real arm by trial and error is slow and risky, so we gave the arm a twin inside a physics simulator — the same arm, down to the lengths of its links — where it can practise for free. Then we used that simulator to teach the robot to see: hundreds of practice pictures that label themselves, and a small model, trained on our own PC, that now finds the coloured cubes in a photo to within a few pixels. No cloud, no outside service.

    • simulation
    • digital twin
    • machine vision
    • training
    • sim-to-real
  2. 1:15 video
    The arm clears the row nearest to it on its own: each cube off its cell and onto a spot beside the board, greens to its left, oranges to its right. Rings, numbers and crosses are the live page drawing what the arm sees and aims at.Video: Claude (GO 3S desk camera, live page recording)

    A tic-tac-toe board, a taught camera map, and the first cubes moved on their own

    The arm moved to a new spot under the small GO 3S camera, with a taped 3 x 3 board, a yellow dot in each cell and small green and orange cubes. After a long run of missed grabs, Ravi taught the camera where the arm is by touching each dot with the fingertips. From then on the arm cleared the row nearest to it on its own, 6 cubes out of 6. The two rows further out still fail: stretched out, the arm bends more than its motors report. Next is a small camera on the gripper itself.

    • tic-tac-toe
    • pick and place
    • calibration
    • teaching
    • live view
    • firmware
  3. 1:39 video
    Ravi drives the real arm from the command center: twist the wrist, lower, grab the black marker, lift, carry and put it down, while the 3D twin follows on the right.Video: Ravi

    A lab journal, a camera that finds the markers, and a command center

    We opened a team channel for the project's photos and videos and started this lab journal. Then the camera learned to find three whiteboard markers on the desk, and we built a command center: a page on the PC that shows the live camera and a 3D twin of the arm and lets a person drive it with an on-screen joystick. Late in the day the command center learned click actions: click a marker and choose pick up, put down, move over or tap. They work in simulation; the real arm waits for the camera to be calibrated in its new spot.

    • journal
    • team
    • website
    • vision
    • teleoperation
    • digital twin
    • click actions

Week of Oct 19 – 25, 2026

The arm arrives and picks up a pen

The Hiwonder LeArm AI arrived on Saturday, and by evening it was taking commands from our code over USB, with safety checks so a bad reading can never turn into a wild move. On Sunday it picked up a marker, spun it, drew a circle in the air and put it back, in a 23-second routine it runs by itself.

2 days

  1. 0:27 video
    The robot camera's view of the filmed third run of our first pick-and-place routine: grip, lift, spin, circle in the air, put back, rest.Video: Robot camera (EMEET 4K)

    The arm picks up and spins a pen

    The arm picked up an object for the first time. We lined the gripper up on a pen with camera checks and our own inverse kinematics, then built a 23-second routine that lifts the pen, spins it, draws a circle and puts it back.

    • arm
    • kinematics
    • pick-and-place
    • camera
    • gripper
    • demo
  2. The assembled Hiwonder LeArm AI with its ESP32 controller powered on: six bus servos that report their own positions to the PC.Photo: Ravi (iPhone)

    The arm arrives and makes its first moves

    The Hiwonder LeArm AI arrived and moved under PC control for the first time, and a 4K camera joined the setup. Each joint stops 1-2.5 degrees short of its target, which shapes how the arm will tap a phone screen.

    • arm
    • servos
    • demo
    • camera
    • safety

Week of Oct 12 – 18, 2026

Day one: a camera that reads a phone

We started with the camera. It streams into our Python code, finds a phone lying on the desk and straightens its screen so small text can be read. The arm was still on its way.

1 day

  1. Our code found the phone by itself: the green outline is the screen it detected and the red dot is the centre point an arm would aim for. The screen is blurred here.Photo: Robot camera + OpenCV, screen blurred

    The camera finds and straightens a phone screen

    First day: the Insta360 GO 3S camera came online, and our code found a phone on the desk, measured where it lay and straightened its screen to read it. This is the vision side of an arm that will tap a phone with a stylus.

    • camera
    • vision
    • phone
    • arm
    • planning