The Sense HAT Add-On Board For Raspberry Pi – Joystick

sense hat joystick

In continuation of our Sense HAT series — Today we’ll be playing with the built-in joystick!

Please feel free to check back the previous articles:

The Sense HAT Add-On Board For Raspberry Pi – Operating The LED Display

The Sense HAT Add-On Board For Raspberry Pi – 6 Types of Sensors

Sense HAT, made especially for the Astro Pi mission, comes with built-in joystick as an input device.

sense hat joystick

The joystick is located next to the Raspberry Pi logo on the Sense HAT, as you can see in the above picture. The joystick handle is bit small (especially when you have a big hand) but it’s quite easy to control the handle. It’s just like other joysticks on game console controllers (but small).

This time we’ll show how to receive input from this joystick on Python. We’ll also design games using the LED display.


Input values of joystick

THE JOYSTICK – Raspberry Pi Learning Resources

The Sense HAT joystick is mapped to the four keyboard cursor keys, with the joystick middle-click being mapped to the Return key. This means that moving the joystick has exactly the same effect as pressing those keys on the Remember that the down direction is with the HDMI port facing downwards.


The orientation of the joystick is as shown in the picture below. The HDMI port faces downward.

sense hat joystick

Figure 1

In addition to the four directions (i.e. up, down, left and right), you can also press the joystick straight in, so you can say there are total of 5 keys.

Let’s start IDEL and run the first sample code.



sense hat joystick

Figure 2

When executed, a black screen called “pygame window” will be launched. This is the screen of the pygame library which we called in lines 6-7.

Pygame – Wikipedia

Pygame is a cross-platform set of Python modules designed for writing video games. It includes computer graphics and sound libraries designed to be used with the Python programming language.


event – Pygame

This will get all the messages and remove them from the queue. If a type or sequence of types is given only those messages will be removed from the queue.

The event is checked by “pygame.event.get ()” in line 14 of the program, and the event contents are printed in line 15.

If you move the mouse over “pygame window” or press the key on the keyboard, the contents of the “event” will be displayed on the console screen. Similarly, input events from the Sense HAT joystick are also detected.

sense hat joystick

Figure 3

I tried to operate in a clockwise fashion, alternating between the joystick and the keyboard (e.g. Joystick – up, keyboard – up, joystick – right, keyboard – right, etc.). You can see that the same event content is displayed for input methods (Figure 3).

The joystick not only can be tilted in 4 directions, it can also be pushed straight in to function like an “Enter” key on the keyboard.

Also, two types of events “KYEDOWN” and “KEYUP” are generated for each input operation. Try pressing it down for a while, the timing of the event occurrence is easy to understand.

In, the window of pygame itself cannot be closed because it’s programmed to display “BYE” and terminate processing when trying to close the window (lines 16 to 18). If you want to quit, close IDEL console screen.


Joystick and LED display

Now let’s connect to the LED matrix display where the movement of the joystick is shown!

sense hat joystick

Figure 4

We’ll write a program to make only the certain matrix glow as we move the joystick.

The processing is roughly divided into two parts:

  • Turn off the currently lit LED
  • Turn on the destination LED

By performing two processes at the same time, it’s possible to move the light across the LED matrix based on the joystick movement.

To individually control the LEDs, please refer back to the “set_pixel” function introduced in The Sense HAT Add-On Board For Raspberry Pi – Operating The LED Display.


Sets an individual LED matrix pixel at the specified XY coordinate to the specified color.

The first argument can be X coordinate (0-7), the second argument can be Y coordinate (0-7), and the third to fifth argument can specify color. You can turn on / turn off the matrices using the following code:

sense hat joystick

Figure 5 / ©Raspberry Pi

The coordinates of the LED display start from 0 in the upper left corner as shown in Figure 5. The operation of the joystick and the change of X coordinate and Y coordinate are illustrated in the below diagram.

Operation of joystick Change in coordinates
Up y-1
Down y+1
Left x-1
Right x+1


The coordinates must be set between 0 and 7, respectively. If invalid coordinates are set, the following error will occur.

ValueError: Y position must be between 0 and 7

There’s a sample code addressing this error so the error can be avoided.

Line 16 is the initial process. It starts by lighting up the upper left LED (X coordinate 0, Y coordinate 0) first.

When there is an input, the program first turns off the LED of the current position, as indicated in line 21. Then it determines whether the coordinates are in the valid range (from 0 to 7), and in line 32 it lights up the destination LED (lines 23 to 30). If an invalid value is set (e.g. when trying to move further right from the right end of the matrix), it looks like there is no movement, but the same LED is turned off and turned on.

Let’s make a game!

Sense HAT Pong | Raspberry Pi Learning Resources

Worksheet – Sense HAT Pong | Raspberry Pi Learning Resources

Pong is an old-fashioned retro game. The purpose of the game is to strike a moving ball with an in-game paddle/bar.

The game can be played using a traditional keyboard but I would like to customize a program so that we can play the game with the joystick.

Let’s start with the game before customization. The completed program source is posted at the end of Chapter 8.


“Curses” imported in the 3rd line is used to detect input from the keyboard. Since curses does not seem to work with IDEL, let’s run it from the command line.

Run command

python3 /home/pi/nas/


Move the white bar on the left with the up and down keys on the keyboard to hit the blue ball. If you fail to return it, the word “You Lose” will flow across the matrix and the game will end. The ball moves at 0.15 second intervals (line 26), but it’s surprisingly fast and difficult!

Let’s adjust this program so that it can be operated using the joystick.


Run command

python3 /home/pi/nas/

Since pygame does not work well with SSH connection or Remote Desktop connection, let’s run it directly from the command line.

I replaced the curses library with the pygame library and changed it so that I can receive input from the joystick (you can also use the up and down keys on the keyboard).

The main changes are the pygame calling part in lines 3-4, the initial processing of pygame in lines 10-11, and the judgment of input values in lines 53-60.

I tried changing the speed to 0.5 seconds so that the movement of the ball is bit easier to predict. When slowing down the movement, the flickering due to “sense.clear ()” became noticeable (original program: line 55), so we modified it to rewrite only the necessary part.


The built in joystick has got to be one of the biggest features of the Sense HAT board. Speaking of the input device of Raspberry Pi, external connection was commonplace, so the Sense HAT with the joystick definitely stands out among other Pi models. You can also attach buttons to the GPIO pins.

In this tutorial, we recreated a Pong game so we can use the built in joystick to move a paddle and hit a ball. Although the display is pretty small, I think it’s pretty unique playing games like Pong on a small LED matrix. You can always change the speed of a ball so you score better!