Intro to CMUcam5 Pixy Vision Camera Sensor Part 2 – Creating a Ball Balance Beam

Intro to CMUcam5 Pixy Vision Camera Sensor Part 2 – Creating a Ball Balance Beam

camera sensor

Click here to read Part 1 of this article >

camera sensor

This is part 2 of the introduction to CMUcam5 Pixy. Please refer back to Intro to: CMUcam5 Pixy Vision Camera Sensor if you’re not already familiar with PixyMon. In Part 1, I covered the basics of Pixy, explained the hello_world code and created a simple servo-driven application. In this tutorial, I will go a step further and build a ball balancing beam. A servo will set the angle of the beam so the ball stays in the middle, and of course, the ball will be tracked by the Pixy camera sensor.


  • Arduino Uno (You can use any Arduino)
  • CMUcam5 Pixy Camera
  • Servo motor (S06NF)
  • Wood pieces and screws
  • Cables (for the camera USB MINI and for the Uno USB B)
  • External 5V for the servo (!WARNING! if you connect the servo to your Arduino and power it through USB it will fry your Arduino)



Brief Introduction to Processing

camera sensor

Processing is a very useful, flexible piece of software. It is mostly used for visual arts and visual literacy within technology. It has over 100 libraries and support for various projects. It’s very well documented and there are guide books available covering topics from programming basics to visualization. It supports all operating systems (GNU/Linux, Mac OS X, and Windows).  The design of the software is almost identical to the Arduino IDE. 

Today, we’ll use Processing to communicate with the Arduino via serial communication.

camera sensor

Figure 1: GUI of Processing

Project Overview

For this project I will make a ball balancing beam. A “tunnel” made of wood will be balancing the ball like a scale (Figure 2). The Beam is 44cm in width and 3cm in height. I made it narrow like a tunnel so the ball that we’ll be tracking can’t fall out.

To move this entire beam I’m using a S06NF servo motor, which will be controlled by the Arduino. We’ll go over the code later in the tutorial. I’ve placed the servo ¼ of the way from the left end of the beam.

camera sensor

Figure 2: S06NF STD Servo Motor / ©RobotShop inc.

The servo will move the beam up and down and the ball will follow this path.

camera sensor

Figure 3: Beam moving up and down

The pixy camera will sit above the beam. I’ll set it up so the visual field of the camera is only limited to the beam. It will only track the ball and nothing else (Figure 14).

Construction of the Beam

First we will need supplies for the construction. I’ll be using a simple XXMM wood (20 cm x 27 cm). I cut the wood using a circular saw, but you can use whatever type of saw you have available that will allow you to make straight, even cuts.

camera sensor

Figure 4: XXMM wood

Remember, only the right tools will allow you to make this beam perfect! I’m using a hammer, a ruler, nails, sandpaper, hot glue, a drill, and a saw.

camera sensor

Figure 5: Tools

First, I’ll make a tunnel where the ball will move left or right. The tunnel will be comprised of 4 wood panels (21cm x 3cm each) for the sides. Two pieces of wood (4cm x 3cm) in height will be attached to each end of the tunnel. The dimensions of the base are 42cm x 3cm x 1cm.

I’m using 15mm clout nails to connect the parts.

camera sensor

Figure 6: Close up of the tunnel

There are many ways to make a tilt point in the middle of the tunnel. I used a very simple method mainly because it is the cheapest and the easiest method. I used a long nail and two small tubes that act like bearings. I marked the center of the tunnel first, then hot glued the bearings to it and inserted the nail.

camera sensor

Figure 7: Nail and tubes for tilt point construction

To make the tilt point, we will also need to make holders for the nail. I used 2 pieces of 8cm x 2cm wood as shown in Figure 8. I also made a small platform that will hold everything together. The dimensions are 12cm x 4.5cm.

camera sensor

Figure 8: Tilt point holders

I used a small piece of wood to mount the servo and to raise it.

camera sensor

Figure 9: Servo mounted on the wood

In this tutorial, I’ve used Arduino UNO but feel free to use any other Arduino that has an SPI connector for connecting to the Pixy Camera.

Connecting Everything Together

Once the construction is done, it’s time to move on to connecting the Pixy camera to the Arduino and then to the servo. The schematic is the same as in Intro to: CMUcam5 Pixy Vision Camera Sensor. I’m still using an external 5V supply for the servo.

!WARNING! Don’t forget to connect the GND’s together. If you don’t connect the Power supply, Servo, and Arduino GND together, the servo will freak out!

camera sensor

Figure 10: The wiring diagram

Next, I need to set up the Pixy somewhere above the beam construction so it can detect the ball at any time. Adjust the settings so it only detects the ball. Please refer back to Part 1 to set the settings.

camera sensor

Figure 11: Pixy vision

Now let’s look at some code. To test if the servo works correctly, I modified the middle, far-left and far-right angles, so it fits my construction:

Of course, you can adjust the variables to your preference.

Earlier, I introduced a software called Processing. I’ll be using this to communicate with the Arduino via serial communication. 

Arduino code

Simple serial communication:

Processing code

It creates a 200x200px window and initilizes the serial port. Void draw() function checks if a mouse is pressed on the window (1 if it was pressed or otherwise it’ll write a 0).

Now, let’s test the code. Click run and try clicking anywhere on the window. Your LED should flash. That means it’s working correctly!

camera sensor

Figure 12: Basic Processing and Arduino code testing

Programming with Processing

I got values of the servo and processed them in Processing so it could make a small graph like the one shown below:

camera sensor

Figure 13 The graph example

To create a graph, use the following code:

Arduino code:

Code explanation

I convert the x position from the Pixy to 0-100% and with that I know exactly where the ball is. By knowing where the ball is, I adjust the intensity of the servo swings. If the ball is <=10% then the servo will swing very fast to balance; if it’s around ~40% then the servo will only swing slightly to balance. Balancing is very tricky. We can certainly improve this algorithm to make it more precise.

Here are some suggestions for imporvement:

  • Try out various algorithms
    • There are various types of mathmatical algorithms to calculate this. I tried at least 2 or 3, but in the end I decided to go with this one. I suggest writing your own algorithm to get a better grasp of this balance method.
  • Better hardware
    • Nothing is perfect, especially wood, for something like this. If I had the resources readily available, I would have constructed this out of metal. The whole project would be more stable and precise.
  • Even faster
    • How can we make it so it balances faster? I’m using a simple servo here. We can replace it with servos like UART or AX-12 servos. They are much stronger and faster. This also has to do with the algorithm. Again, I recommend trying out different algorithms to find the one that suits your purpose.

There are many projects out there that use a similar concept to balance an object. In additional to Pixy, you can use OpenCV with any web camera to detect objects and colors. Next to Processing, there is also Max/MSP version 5. You can utilize distance sensors, pressure sensors, etc. So there are many ways you can improve this project to make it sturdier, steadier, and faster.