RoboticsAutosterea.comTalk to us
Robotics
Autosterea.comTalk to us
Journal

The arm picks up and spins a pen

The arm picked up an object for the first time. We lined the gripper up on a pen with camera checks and our own inverse kinematics, then built a 23-second routine that lifts the pen, spins it, draws a circle and puts it back.

  • arm
  • kinematics
  • pick-and-place
  • camera
  • gripper
  • demo

The robot camera's view of the filmed third run of our first pick-and-place routine: grip, lift, spin, circle in the air, put back, rest.

Video: Robot camera (EMEET 4K)0:27No sound

What we did

  1. The arm picked up an object for the first time: it gripped a whiteboard marker (the pen) lying flat about 16 cm in front of its base, lifted it, spun it with the wrist and put it back where it was.
  2. tools/pen_demo.py turns the pick into a 23-second routine: grip, lift, spin the pen about 85 degrees each way, draw a circle of 3 cm radius in the air, put the pen back and return to rest. It checks the grip before lifting, stops if a joint is blocked, and ran three times without a miss.
  3. We lined the gripper up over the pen by taking a 4K picture after each small move. Then we found the desk by feel, lowering 3-4 mm at a time until the joints stopped following: z = -7.9 cm in the arm's own coordinates.
  4. We wrote our own forward and inverse kinematics in Python (arm/kinematics.py), which keeps 3 degrees clear of each joint's end of travel, plus tools/arm_goto.py, which puts the gripper tip at a point given in cm and refuses points below a set floor.
  5. tools/record_run.py films the robot camera while a command runs and saves a 1080p clip trimmed to the action. A lead time (10 s for this run) lets a person start their own recording. We filmed the third run with it.
  6. We checked whether the arm could play chess against a person with no cloud service. The open-source Stockfish engine can pick moves on the PC, so the hard part is the hand: reach, fingers and placement accuracy.

What we learned

Inverse kinematics
Inverse kinematics turns a target point for the gripper tip into angles for the base, shoulder, elbow and wrist; forward kinematics does the reverse. We wrote both from the link lengths in Hiwonder's firmware source and checked them on the rest pose: 14.55 cm ahead and 1.63 cm up, matching the firmware.
Visual servoing
Visual servoing means steering a robot by what its camera sees. We moved the gripper in small steps and compared the fingertips with the pen in a 4K picture after each step, until the pen sat between the fingers. Side to side was the tight direction: the part-open jaws left only about 1 cm each side.
Parallel-linkage gripper
The fingers swing down and inward on an arc as they close, so the fingertips drop about 2.5 cm. The arm therefore part-closes the gripper in the air, lowers it around the pen, then closes it. A gripper that stops well short of its target is holding something: aimed at 640, it stopped at 604 on the pen, and the routine counts any reading under 620 as a catch.
Sensing contact through the joints
Bus servos report where they really are, so the arm can feel contact without a separate sensor. Lowering 3-4 mm at a time, we watched for the joints to stop following their commands, which put the desk at z = -7.9 cm. Contact showed first as a stall in the position reading, then as growing shoulder and wrist errors, so it pays to watch both.
Workspace limits on gripper tilt
A point the tip can reach may still be out of reach at a given tilt. At 16 cm from the base, the gripper can point almost straight down (about 87 degrees below horizontal) only near the desk; with the tip about 12 cm above the desk, the steepest is about 53 degrees. So the routine draws its circle at a fixed 56 degrees below horizontal, a tilt that reaches every point on the circle.

Numbers

Pick, spin, circle and place routine23.3-23.4 s per run (3 runs)
Desk height in the arm's coordinates (fingertips)z = -7.9 cm
Fingertip drop as the gripper closesabout 2.5 cm
Gripper reading when holding the 1.7 cm-thick pen597 (target 620) on the first pick, 604 (target 640) in the routine
Base deadbandabout 7 units (1.7°), 0.5 cm sideways at 16 cm
Wrist spin with the penabout 85° each way
Pen drift after scripted runsabout 3 mm after one run, about 5 mm after two
Grip range at chess-piece height (calculated)about 12-26 cm from the base

Problems and fixes

  • The camera sometimes sent all-black frames at 4K for several seconds after opening, so snapshots came out black.

    Fix: We first kept the camera open in a background helper, then changed grab_frame in vision/camera.py to skip all-black frames for up to 5 s after warm-up.

  • The base stops about 7 units (1.7°) short of its target and ignores small corrections. That is 0.5 cm sideways at the pen, where the jaws had only about 1 cm to spare on each side.

    Fix: We aimed the base past the target so it stopped in the right place, and checked the result in the camera picture.

  • When we felt for the desk, our stop rule missed the contact: the read-back position kept creeping down after the fingertips touched, so the arm kept pressing for several more 3 mm steps.

    Fix: Contact showed as a stall at -7.92 cm followed by growing shoulder and wrist errors, so we backed off right after the probe and set the desk at z = -7.9 cm. The lesson: watch how far each joint lags its target, not only its position.

  • pen_demo.py replays one measured spot and does not look for the pen, and the pen crept toward the arm's right: about 3 mm after one run, about 5 mm after two.

    Fix: That is still within what the gripper tolerates. Next we map camera pixels to desk positions so the arm can find the pen wherever it lies.

Next

  • Find the pen anywhere on the desk: touch 4 or more marked points with the fingertips to map camera pixels to desk positions, then detect the pen in the picture.
  • Proposed, not yet agreed: an arm that plays chess against a person, fully offline. The open-source Stockfish engine chooses the moves, the camera spots which squares changed, and our kinematics moves the pieces. Step one (one session): the arm points at its move and the person moves the piece for it.
  • Then two to three sessions on picking and placing pieces. The arm only grips between about 12 and 26 cm from its base, so it needs a small board (about 16 cm, 2 cm squares, pieces under about 3.5 cm tall). The stock fingers spread about 4 cm, so we need thin 3D-printed fingers or a suction cup, and the camera must move higher to look straight down at the board.
  • Chess needs placement within about 3 mm, and we estimate the arm lands within about 5 mm now. So we would measure each square once with the camera and always approach from the same side, and the arm must never move while a hand is over the board.

Photos and videos

  • The starting view from the robot camera: the gripper at its rest pose and the pen lying straight out in front of it.Open the photo large

    The starting view from the robot camera: the gripper at its rest pose and the pen lying straight out in front of it.

    Photo: Robot camera (EMEET 4K)

  • The gripper, part-closed in the air, hovers centred over the pen: the first camera check before lowering step by step.Open the photo large

    The gripper, part-closed in the air, hovers centred over the pen: the first camera check before lowering step by step.

    Photo: Robot camera (EMEET 4K)

  • The fingers straddle the pen just above the desk, right after the arm felt for the desk and backed off.Open the photo large

    The fingers straddle the pen just above the desk, right after the arm felt for the desk and backed off.

    Photo: Robot camera (EMEET 4K)

  • The gripper closed on the pen at the desk; it stopped at 597 of a 620 target, a sign that it was holding something.Open the photo large

    The gripper closed on the pen at the desk; it stopped at 597 of a 620 target, a sign that it was holding something.

    Photo: Robot camera (EMEET 4K)

  • The pen lifted about 3 cm off the desk, the check that the grip had really caught it.Open the photo large

    The pen lifted about 3 cm off the desk, the check that the grip had really caught it.

    Photo: Robot camera (EMEET 4K)

  • The pen held high during the first step-by-step pick, after the gripper target was raised to 640 (it stopped at 604), showing that the grip holds while the arm moves.Open the photo large

    The pen held high during the first step-by-step pick, after the gripper target was raised to 640 (it stopped at 604), showing that the grip holds while the arm moves.

    Photo: Robot camera (EMEET 4K)

  • The arm lowers the pen back onto the spot it came from, just before letting go.Open the photo large

    The arm lowers the pen back onto the spot it came from, just before letting go.

    Photo: Robot camera (EMEET 4K)

  • After the second scripted run, the arm is back at rest and the pen is on the desk, having crept about 5 mm to the arm's right over two runs.Open the photo large

    After the second scripted run, the arm is back at rest and the pen is on the desk, having crept about 5 mm to the arm's right over two runs.

    Photo: Robot camera (EMEET 4K)