Arduino Bluetooth Gloves Part 2 – Putting Everything Together

arduino bluetooth gloves

Click here to read Part 1 of this article >

arduino bluetooth gloves

In this article, we will continue to develop Arduino Bluetooth gloves. We’ll build a mechanical system for transformation of the finger’s bending. We’ll learn how to combine IMU data, pack and send data wirelessly, use Kinect, and utilize Arduino Bluetooth gloves in real-life application. If you’re new to the Arduino Glove project, we highly recommend you read Arduino Bluetooth Gloves Part 1 – Basics of this project first.

Hardware

  • Arduino UNO R3 x 2
  • IMU Sensor x 2
  • Wires
  • Potentiometer 0-10kOhm x 10
  • Resister 10 kOhm x 10
  • Xbee USB Adapter
  • Bluetooth Bee V2 x 2
  • 9V battery x 2
  • 9V battery connector x 2
  • Kinect
  • Glove x 2
  • Fasteners x 20

Software

  • Arduino IDE
  • Autodesk Inventor
  • LabVIEW
  • LabVIEW Robotics Module
  • Kinesthesia Toolkit for Microsoft Kinect
  • Kinect SDK 1.5
  • Github

Tools

  • PLA plastic
  • Nut M3 x 40
  • Self-locking nut M3 x 40
  • Soldering iron
  • Tin
  • Solder
  • Glue

Finger Bending Measurement

First, to measure flexion of the fingers (finger bending), we must transform the finger bending to a rotation of the potentiometer. We could make it with simple mechanical system as shown below:

arduino bluetooth gloves

Figure 1: Diagram of a mechanical system for measurement of the finger bending

To make it in real life, we have to model it in a solid state modeling editor, such as Autodesk Inventor.

arduino bluetooth gloves

Figure 2: Modeled mechanical system for measurement of the finger bending

Next, we have to 3D print the model. An STL file is a common type of 3D model among all 3D printers. Just export all your models to STL separately and send it to a 3D printer. To see the compiled 3D designs, please click here.

arduino bluetooth gloves

Figure 3: 3D printed parts for finger mechanical system

Take your potentiometer (without black knob/grip), screws, nuts, and the 3D printed parts and then assemble them as shown in Figure 4.

arduino bluetooth gloves

Figure 4: Finger base unit assembly

Screw in the joints as shown in Figure 5. Then, we’ll glue the black grip with screwed finger joint.

arduino bluetooth gloves

Figure 5: Finger joint assembly details

arduino bluetooth gloves

Figure 6: Base finger unit with fastener

Once the joints are screwed in, we’ll glue the assembled fingers to the glove. Use extra strong multi-purpose glue. Surfaces must be clean and dry before gluing. Then apply a thin layer of glue on both surfaces and wait for 10 to 15 minutes. Press the surfaces firmly for a few seconds. You can spread some additional glue around the unit if necessary. It’s recommended that you leave it to dry for about 24 hours.

*DO NOT stick your fingers into the glove while gluing. Use cardboard pieces or other material.

arduino bluetooth gloves

Figure 7: Gluing base unit onto the proximal phalanges (at the bottom of each finger)

Now we’re going to glue the finger joint. Take the joint part and another sticky fastener and stick them just above the finger’s nail (Figure 8 & 9).

arduino bluetooth gloves

Figure 8: Gluing finger joint

arduino bluetooth gloves

Figure 9: Finger joint assembly (glued)

Connect both assembled parts by inserting the black grip into the potentiometer hole.

Please be patient and repeat the same operation 9 more times with each finger.

A wire diagram of the entire module is shown below.

arduino bluetooth gloves

Figure 10: Arduino glove circuit diagram

Now, let’s wire up the fingers!

Prepare red and black wires and also three wires with white, yellow, and black markings. Color coding is strongly recommended to avoid confusion.

Solder the black-marked and yellow-marked wires to the potentiometer as it shown in Figure 11.

arduino bluetooth gloves

Figure 11: Soldering the potentiometer

Tie the wires with a plastic cable tie. This is necessary to protect it from any mechanical stress.

arduino bluetooth gloves

Figure 12: Fix wire with a cable tie

Next, you should solder the black-marked wire with the black wire & yellow and white-marked cables to the resistor. The red wire must be soldered to another side of the resistor. The white-marked wire should then be connected to the Arduino analog input.

arduino bluetooth gloves

Figure 13: Soldering one finger unit

arduino bluetooth gloves

Figure 14: Grouping black- and red-marked wires

After soldering all the units, we’re going to shrink the wires with heat-shrink tubes and group the wires with cable ties.

arduino bluetooth gloves

Figure 15: Fingers connected to the body

arduino bluetooth gloves

Figure 16: Assembled fingers connected to Arduino

arduino bluetooth gloves

Figure 17: The processor body (Arduino)

arduino bluetooth gloves

Figure 18: IMU sensor connection

arduino bluetooth gloves

Figure 19: Assembled Arduino glove

The assembly is done! Let’s move on to programming.

Install a new version of the IMU Library. Please refer back to Part 1 to learn how to install the library.

Open bending.ino and upload it to the Arduino Uno.

You will then see a result shown in Figure 20:

arduino bluetooth gloves

Figure 20: Finger bendings reading result

Note that you have to follow calibration procedure to be able to measure data accurately.

Fusion IMU data

In Arduino Bluetooth Gloves Part 1 – Basics, we covered the basics of IMU sensor. Now we will learn how to combine gyroscope and accelerometer data.

First, let’s go back and check out our previous measurements.

arduino bluetooth gloves

Figure 21: IMU sensor readings

If you investigate the L3G4200D datasheet and the IMU Library carefully, you can see that MEMS Gyroscope measures angle rate in raw digit units and IMU Library’s functions convert digit units into degree per second (dsp).

For example, the Gyroscope returns 12492 digit value:

  • If the Gyroscope is configured to Measurement range ±250 dps, then this value multiplies at 0.00875 dps/digit value to get 109.30 dps angle rate. (109.30 dps means that a body with this angle rate makes full rotation about the measured axis for about 4 seconds.)
  • If the Gyroscope is configured to Measurement range ±2000 dps, then this value multiplies at 0.07 dps/digit value to get 874.44 dps angle rate. (874.44 dps means that a body with this angle rate makes about 2.5 full rotation about the measured axis for a second.)

A sensor’s sensitivity means the smallest value change which could be detected with this sensor. Note that the wider the range the less sensivity you may get from the Gyroscope.

An angular rate is a value measured by gyroscope. Thus, a gyroscope is able to estimate how fast a body is rotated about a certain axis. In our application, it’s necessary to determine in which position our glove is. We could summarize all angular rate samples divided by change of time to estimate current angular position.

Because the measurement of angular rate is not continuous but discrete, we get an error called drift. It’s called “drift” because the sum of these changes of angular position will float in time even though a body is at rest.

Let’s look at the gyroscope drift example below:

If you leave your IMU sensor without any motion, you will see drift in action (Figure 22):

arduino bluetooth gloves

Figure 22: Gyroscope drift demonstration

As mentioned above, due to the tendency of drift, the sum will produce an error that increases in time. But we can mitigate the drift effect by calculating and utilizing zero-rate level.

The definition of the Zero-rate level is described well in the Gyroscope datasheet, at 2.6.2 Zero-rate level section. The main reason of this is a mechanical stress applied to IMU, for example, after soldering on printed curcuit board. After it’s mounted, the Zero-rate level varies slightly with temperature so this change may be neglected.

arduino bluetooth gloves

Figure 23: Zero-rate level based Gyroscope measurements

 

You can look at implementation details in the new version of IMU Library here.

After these enhancements, we could use the gyroscope data, but only for the short term. We still need something which is stable in the long term.

An accelerometer is a device that measures acceleration. Using accelerometers we’re able to determine angle of a sensor’s work plane based on calculated gravity vector. However, with smart gloves, a sensor will measure all other accelerations that include motions of the arm, which could corrupt the measured gravity vector.

A Low pass filter might be used to filter the gravity vector from random accelerations. The accelerometer’s measurements could be used only in the long term.

One of the most common ways to fuse sensor data is to use a powerful Kalman filter (which is difficult to understand and to implement on Arduino).

As a solution for combining IMU data, we could use a complementary filter, which is much easier to understand and implement on Arduino. The essential part of the complementary filter is as follows:

complementary filter

where Roll_i – is a calculated Roll angle from the previous iteration

gyroRoll_i+1 – is a change of angle from the last measurement based on the last gyroscope data

accelRoll_i+1 – is a calculated Roll angle based on the last accelerometer data

Roll_i+1 – is a new calculated Roll angle with complementary filter

This filter utilizes the gyroscope’s dynamics in the short term and gets rid of the error in the long term based on the accelerometer data. You can test it yourself, just open imu_2.ino in Arduino IDE and upload it to your Arduino UNO.

Update Message Sender

As we get measured data, we have to send this data to a PC using a wireless Bluetooth connection.

First, check that Bluetooth Bee is configured correctly. Install the Bluetooth Bee on the Xbee USB Adapter. Change the Bluetooth Bee to the AT Mode and then check UART configuration with the “AT+UART?” command. The Bluetooth Bee has to return “115200,0,0” (baud rate 115200 bits/s, 1 stop bit and none parity bit). If not, send “AT+UART=115200,0,0” command to configure the Bluetooth Bee and check UART configuration again. Don’t forget to switch off AT mode after checking!

One more thing that requires your attention is a “Serial Select” switch which is placed on the Wireless SD shield. This switch defines where to send data that was read from the wireless connection.

arduino bluetooth gloves

Figure 24: Serial select switch on a Wireless Shield / ©Arduino LLC

Switch it to the MICRO position. When in the Micro position, the Bluetooth module is connected to the Arduino. Note that the Arduino is still connected to the USB-to-serial converter. If you write data to the serial port from the Arduino, data will be sent in both ways: through USB and wirelessly.

Upload messaging.ino to the Arduino. Notice that every time the Arduino is powered on or resetting it will calibrate in 3 steps:

Gyroscope Zero-Rate Level calculation;

Measurement of minimum finger position;

Measurement of maximum finger position.

Before the Gyroscope Zero-Rate Level calculation, leave the IMU without any movement. Before measuring the minimum finger position, clench your fist with Arduino gloves on. Before measuring the maximum finger position, unclench your fist with Arduino gloves on.

Open messaging.ino in Arduino IDE and upload it to your Arduino UNO.

Power off the Arduino and place the Bluetooth Bee on the Wireless shield. Make sure that the Bluetooth Bee is not in AT Mode and that the Wireless Shield Serial select switch is switched to MICRO position.

Power your Arduino Uno with a 9V battery and pair with your Bluetooth Bee using “1234” as the password. Make sure that after pairing you can find your wireless Bluetooth connection as a Virtual COM port in the Device Manager.

arduino bluetooth gloves

Figure 25: Pairing with Bluetooth Bee

arduino bluetooth gloves

Figure 26: Virtual COM port in the Device Manager

Open Messaging.lvproj LabVIEW project and Messaging Test.vi from a Project explorer window. Choose corresponding COM port and run it.

arduino bluetooth gloves

Figure 27: Messages from Arduino Glove shown in a LabVIEW PC application

You will see that the data which Arduino Uno is writing in the Serial port is transferred via Bluetooth and read here from a Virtual COM port.

Adding Kinect Measurements

Kinect is an intuitive Xbox 360 game console device controller which is used as alternative to common joysticks. Actually, it is a powerful motion-capture controller that acquires your poses and allows you to control game processes with your hands or body. Even though it was developed for games, it can be used in real-world applications.

To use Kinect for Xbox 360 in a LabVIEW environment you need to install the following:

Open Kinect.lvproj LabVIEW Project and Kinect Test.vi from Project Explorer window, then Run it. If all requirements were installed correctly, you will see video and Kinect processing results similar to the one below:

arduino bluetooth gloves

Figure 28: Video and Kinect processing results example / ©Kinesthesia

Putting All Together

Now we need to put everything together in one application.

Open Arduino Gloves.lvproj LabVIEW Project. Then open Host Main.vi from the Project explorer window. Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that it’s ready to use.

Then put on Arduino Bluetooth gloves and power them on. Provide the required conditions for a calibration.

After this, run Host Main.vi and you will be able to see the following:

arduino bluetooth gloves

Figure 29: Getting all measurements in one application

Make sure that every finger flexion is measured correctly and that Roll, Pitch, and Yaw correspond to the actual attitude of your hand.

Robot Control

Arduino Bluetooth gloves could be useful for a real-life robot control application. All processed data could be transmitted to any third-party application with UDP messages.

arduino bluetooth gloves

Figure 30: Sending data to third-party application diagram

As an example of a third-party application, a Simulator of an industrial robot arm – Puma560 was picked and added UDP Receiver loop. Open Puma560 Simulator.lvproj, Host Main.vi and Puma560 Simulator.vi from Project Explorer Window.

Choose the corresponding COM port for the Left and Right Gloves Bluetooth connection. Plug in Kinect and make sure that its ready to use.

Then put on Arduino Bluetooth gloves and power them on, and provide the required conditions for a calibration.

After this Run both VIs and you will get the following results:

arduino bluetooth gloves

Figure 31: Exports processed data to Puma560 Simulator

Please note that you have to provide the right interpretation of the data that you get from the Smart Glove application. You shouldn’t use acquired data directly, without any additional processing. For example, the operator’s right hand coordinates are unlikely to be equal to robot’s arm gripper. You have to convert measured data from the operator’s coordinate system to your robot’s coordinate system.

DEMO: