Autosterea Robotics journal

Teaching the cube-finder to read the real, messy camera view

Wed, 04 Nov 2026 12:00:00 GMT

The retrained finder on pictures it had never seen — now with several cubes per picture and clutter around them. The crosses are the model's guesses, the circles the true spots; they still land on top of each other.
Photo: Claude (MuJoCo simulation)

Our self-taught cube-finder worked beautifully on tidy simulated pictures — but the real camera looks down at an angle, sees several cubes at once, and a cluttered workbench all around them. So we rebuilt the practice world to be harder than real life: a camera that roves and tilts, up to several cubes per picture, and random clutter objects thrown in to ignore. We retrained the model overnight on our own PC. It now reads the real camera's full picture — nothing cropped out — finds every cube, and steps over almost all the clutter.

Read the full day

Teaching the arm in a simulator, and a cube-finder that trained itself

Sun, 01 Nov 2026 12:00:00 GMT

Our arm, rebuilt in a physics simulator, running a reach-and-approach over the cubes — tested entirely in software, where a mistake costs nothing.
Video: Claude (MuJoCo simulation)

Training a real arm by trial and error is slow and risky, so we gave the arm a twin inside a physics simulator — the same arm, down to the lengths of its links — where it can practise for free. Then we used that simulator to teach the robot to see: hundreds of practice pictures that label themselves, and a small model, trained on our own PC, that now finds the coloured cubes in a photo to within a few pixels. No cloud, no outside service.

Read the full day

A tic-tac-toe board, a taught camera map, and the first cubes moved on their own

Tue, 27 Oct 2026 12:00:00 GMT

The arm clears the row nearest to it on its own: each cube off its cell and onto a spot beside the board, greens to its left, oranges to its right. Rings, numbers and crosses are the live page drawing what the arm sees and aims at.
Video: Claude (GO 3S desk camera, live page recording)

The arm moved to a new spot under the small GO 3S camera, with a taped 3 x 3 board, a yellow dot in each cell and small green and orange cubes. After a long run of missed grabs, Ravi taught the camera where the arm is by touching each dot with the fingertips. From then on the arm cleared the row nearest to it on its own, 6 cubes out of 6. The two rows further out still fail: stretched out, the arm bends more than its motors report. Next is a small camera on the gripper itself.

Read the full day

A lab journal, a camera that finds the markers, and a command center

Mon, 26 Oct 2026 12:00:00 GMT

Ravi drives the real arm from the command center: twist the wrist, lower, grab the black marker, lift, carry and put it down, while the 3D twin follows on the right.
Video: Ravi

We opened a team channel for the project's photos and videos and started this lab journal. Then the camera learned to find three whiteboard markers on the desk, and we built a command center: a page on the PC that shows the live camera and a 3D twin of the arm and lets a person drive it with an on-screen joystick. Late in the day the command center learned click actions: click a marker and choose pick up, put down, move over or tap. They work in simulation; the real arm waits for the camera to be calibrated in its new spot.

Read the full day

The arm picks up and spins a pen

Sun, 25 Oct 2026 12:00:00 GMT

The robot camera's view of the filmed third run of our first pick-and-place routine: grip, lift, spin, circle in the air, put back, rest.
Video: Robot camera (EMEET 4K)

The arm picked up an object for the first time. We lined the gripper up on a pen with camera checks and our own inverse kinematics, then built a 23-second routine that lifts the pen, spins it, draws a circle and puts it back.

Read the full day

The arm arrives and makes its first moves

Sat, 24 Oct 2026 12:00:00 GMT

The assembled Hiwonder LeArm AI with its ESP32 controller powered on: six bus servos that report their own positions to the PC.
Photo: Ravi (iPhone)

The Hiwonder LeArm AI arrived and moved under PC control for the first time, and a 4K camera joined the setup. Each joint stops 1-2.5 degrees short of its target, which shapes how the arm will tap a phone screen.

Read the full day

The camera finds and straightens a phone screen

Sat, 17 Oct 2026 12:00:00 GMT

Our code found the phone by itself: the green outline is the screen it detected and the red dot is the centre point an arm would aim for. The screen is blurred here.
Photo: Robot camera + OpenCV, screen blurred

First day: the Insta360 GO 3S camera came online, and our code found a phone on the desk, measured where it lay and straightened its screen to read it. This is the vision side of an arm that will tap a phone with a stylus.

Read the full day