Haptic Feedback Example Applications: Haptic Touchscreen

Haptic Feedback Example Applications: Haptic Touchscreen

Disclaimer!

These examples are provided as guides to using the Haptic Feedback Evaluation Kit for potential real-world applications. As discussed at the end, these are not full real-world solutions, but a useful starting step!

They outline the approach taken to solve the problem. They are not a guide to programming on the Arduino, therefore some prior knowledge of the Arduino language will be useful. Try the Arduino Learning or Reference pages for help with specific functions.

Introduction

Haptic Touchscreen Example Application

Haptic Touchscreen Example Application

Touchscreens continue to become more and more commonplace. In particular with whitegoods and machine controls where old fashion buttons/dials are being replaced with interactive software-based interfaces.

One drawback of touchscreens is confirming user input - the tactile response of pressing a physical button is lost. As a result, input errors and user frustration can increase. Manufacturers have previously solved this with an audible alert such as a beep, but this is not suitable for loud environments and can quickly become irritating.

Haptic feedback is alternative which can either replace or augment the audible alert. In mobile phones users are able to customise which alert the receive for button presses - most elect to only enable the haptic response.

Definition And Scope

To add haptic feedback to a touchscreen interface, utilising different vibration patterns when different buttons are pressed.

  • Display interface on a touchscreen, consisting of several 'buttons' drawn on-screen
  • Recognise user input
  • Play a specific haptic effect, dependant on the button being pressed

Of course, any touchscreen in a product is likely to be customised for that specific design. In addition, there are many different prototyping products available, some even with specifically designed Arduino shields.

For this reason, any tutorial on driving the touchscreen would likely be too specific for a general audience. Therefore, we will describe the approach taken - but limit the scope to the essentials.

Of course, if you have any questions you can contact us and we will be more than happy to help.

Hardware

  • 1 x Haptic Feedback Evaluation Kit

It is best to separate the Haptic Shield from the Arduino and use jumpers to connect the required pins.

  • 1 x Arduino Mega (Optional)

This is in addition to the Arduino Uno R3 included with the haptic kit. We’ve used a separate Arduino Mega to make things simple, for you, it will depend on your chosen touchscreen model. As mentioned above, some modules come with pre-designed shields, which would dictate your chosen microcontroller. It is also possible to connect the Haptic Shield to an Arduino Mega, enabling you to work with only one Arduino.

Photograph of the Arduino mega

The Arduino Mega

  • 1 x Touchscreen Module

We have a 320 x 240 resistive touchscreen, including the SSD1289 driver. Our unit also includes a built-in SD card reader which we can access through the headers.

Photograph of the TFT 320 x 240 Resistive Touchscreen Module

TFT 320 x 240 Resistive Touchscreen Module

  • 1 x SD Card Reader Module (+ SD Card) (Optional)

In order to build the interface, we are storing images in memory. Our touchscreen module included a convenient built-in SD module, if your choice does not you would either an external module or draw the interface manually through code (instead of loading a bitmap, explained later).

Built-in SD Card Module

Built-in SD Card Module

  • 25 x 50 kΩ Resistors (Optional)

These were required for our touchscreen module’s I/O pins, again these are optional.

  • 1 x C10-100

For a touchscreen application, we obviously want to have the haptic feedback played through the screen itself. We, therefore, need an external actuator (see how to connect one here) which we can mount to the screen, an LRA is perfect because it is flat with self-adhesive backing and has an excellent haptic response.

Software

Again, this will vary depending upon your choice of screen, as the software will be specific to the hardware - the code to operate our screen and SD card may not be compatible with your chosen modules.

However, there is an excellent set of libraries available that support a range of screens and SD card readers from Henning Karlsen. Please note there are some limitations to the libraries, they’re outlined in the download links below:

  • UTFT

This library handles the display, allowing you to draw shapes and colours on the screen. Obviously, we’ll need this to create an interface for the user - unless you plan to use the entire screen as one big button!

  • UTFT_tinyFAT

With this library, we can load images from the SD card to display on the screen. By loading images, we can create visually appealing interfaces, and even add some company branding if you desire. Note this library requires the UTFT (above) and tinyFAT (below) libraries too.

  • tinyFAT

The UTFT_tinyFAT library is built upon this library, as a note, it only supports FAT16 formatted SD cards and they must be below 2GB in size. More information on this is available through the link above. We’ll be pre-loading the files onto the card before connecting it to the Arduino, so a card reader, SD card port, or some other way of connecting the card to your PC would be helpful - otherwise, you can use this library to write files to the card using your Arduino.

  • UTouch

In order to register our user’s input, we need this library to handle the resistive touchscreen. Note that if you are using a screen with a specifically designed shield, there are some restrictions with this library.

  • Haptic Feedback Evaluation Kit Development Mode

Most importantly, you will need the Development Mode code to handle the Haptic Shield and play the haptic effects on the external C10-100.

Guide

The Interface Design

Here’s a system view of our touchscreen interface:

The Interface Design

The Interface Design

The Arduino Mega will be responsible for loading images from the SD card and displaying them on the screen as a user interface. It also registers touches from the user and will notify the Haptic Feedback Evaluation Kit to play the appropriate haptic effect (dependant upon the touch location).

On the note of touch locations, we need an interface for the user to work with. To keep the application as general as possible we created a generic interface from common buttons, including:

  • 4 x directional navigation buttons (up, down, left, right)
  • confirm/accept button
  • cancel/decline button
  • reset button
  • power button
On-Screen Interface

On-Screen Interface

Our TFT screen can display bitmap images which we will load onto the SD card (they’re too big to store on the processor). We can easily plan and create our interface on the computer and simply call the function to draw it on-screen.

In addition to the haptic effect, we will add a visual confirmation of the button press on the screen. This lets the user know the correct button press has been registered, especially important buttons in close proximity to each other. With our installed libraries, we can easily draw bitmap images and basic shapes like circles.

One issue is that the software simply draws on top of previous layers and doesn’t track layer properties like you might be used to with Photoshop, or similar. This means there’s no ‘clear circle’ function - instead, you must re-write the interface image above the circle. This can be a bit unsightly when writing an image over the entire screen, accessing the SD card can be slow and the image is drawn line by line.

As a solution, we’ll split the buttons into different image files and draw them independently. We discuss the code in full below, but when designing the interface we need to be aware that the;drawBitmap() function uses the top left pixel for positioning. However, the;drawCircle() function uses the centre of the circle for positioning:

Positioning the images to build the interface

Positioning the images to build the interface

The different buttons will play different effects to provide the user with additional information. Simple differences, like double clicks on particular buttons or functions, can make a big impact on the user experience:

Interface's Haptic Effects

Interface's Haptic Effects

Each button has a single effect assigned to it, with the exception of the power button which is using a ramp up when switched on and a ramp down when switched off. However, it’s more likely to be acting as a ‘sleep’ button. For example, a microwave with a touch panel is not truly switched off because it needs to monitor the panel for the user switching it back on. Of course, a power switch uses a physical button instead, but for our example, we need to keep track of the application’s ‘state’ - i.e. whether it is being switched on or off to play the appropriate effect.

In summary, our interface:

  • Has 8 buttons in total
  • Draws the images for the buttons independently
  • Registers the location of the user’s touch
  • Plays a haptic effect depending on the button
  • Draws a red circle on the button for visual confirmation
  • Re-draws the button image to clear the circle
  • Tracks the ‘power state’

The images for the buttons are created as bitmaps, then converted to ‘.raw’ files using the imageconverter565.exe tool in the UTFT library. You can download all the image files used for the interface in this tutorial in the top right of the page.

Hardware Wiring

Please note: after completing the hard work of wiring up the screen, it is easy to forget that we are using an external actuator - remember to disconnect the Haptic Grip from the Haptic Shield. Mounting the C10-100 to the screen was simplified by using the adhesive backing of the LRA with the PCB. For more information on mounting vibration motors to touchscreens, see Application Bulletin 014: Mechanical Layout of Vibration Motors for Typical User Interfaces and Controls.

C10-100 mounted to a touchscreen PCB

C10-100 mounted to a touchscreen PCB

This is where specially designed shields have a major advantage, no wires necessary. However, it is important that you continue to use the correct pins:

Display

Refer to the UTFT library download, the zip contains the document UTFT_Requirements.pdf which explains the connections for the screen.

SD Card

Refer to the tinyFAT library download, the zip contains the document tinyFAT.pdf which explains the connections for an SPI based SD module.

Touch

You can use any free pins, this is initialised in the code. We used pins 6 -> 2 for D_CLK, D_CS, D_DIN, D_OUT, DPenirq respectively. See the documentation on the;UTouch() function in the UTouch.pdf of the UTouch library.

50k Ohm Resistors

50k Ohm Resistors

If you are not using a shield, another important consideration is the voltage levels of each module. In our instance, we needed 50 kΩ resistors between the Arduino Mega and every pin on our touchscreen module except the power pins and the 4 SD card pins (SPI interface).

50k Ohm Resistors
It is strongly recommended you attempt to use the library examples to check the wirings. In particular:

Display: File -> Examples -> UTFT -> Arduino(AVR) -> UTFT_Demo_320 x 240
SD Card: File -> Examples -> UTFT_tinyFAT-> Demo_Landscape
Touch: File -> Examples -> UTFT -> Arduino -> UTouch_QuickDraw
We also need to connect the Arduino Mega to the Arduino Uno to signal button presses and play the haptic effect. Use any spare pins you like, we define which ones are used in code:

Button

Arduino Mega

Arduino Uno

Navigation

9

A3

Confirm / Decline

11

A2

Reset

12

3

Power

13

12

In order to make the connections in the table above, the Arduino Uno requires the Haptic Shield to be disconnected - unless you solder the connections to the top of the pins on the PCB (not recommended). This means you will need to manually connect the Uno to the pins on the Haptic Shield required by the DRV2605. We’ve provided a pin mapping resource here, the pins you need to connect are represented by the red dots.

Code

HapticTouchScreen.ino

The Haptic Touchscreen set up

The Haptic Touchscreen set up and showing the "splash screen"

This requires a fair amount of code to set up. First, we include all the required libraries (lines 4 - 7) and set up each object (lines 11 - 13). Take care to use the correct module in line 11, ours is the SSD1289. Refer to UTFT_Supported_display_modules_&_controllers.pdf for the correct model code.

The Haptic Touchscreen's "Splash Screen"

The Haptic Touchscreen's "Splash Screen"

We also need to set the pins connected to the Uno as outputs and ensure they are low to start (lines 23 - 31). Remaining setup code sets our screen to portrait mode and fills it with a white background.

At line 44 we load our first image, what is known as a "splash screen". A perfect opportunity to add some branding, although in reality a splash screen is normally used as a program loads in the background - ours just delays the loading of the program! If you had other tasks to set up before loading the user interface, this is the time to do it.

The red circle adds a visual confirmation

The red circle adds a visual confirmation

Line 48 loads our banner before drawing all the buttons, positioned according to our interface design above. We set the colour to red for drawing our circles and finally the serial monitor is used to help to debug (line 62).

To determine which button is touched, the coordinates of the touch location are saved to variables x and y (lines 72 and 73). We check the Y coordinate to find which row was pressed, then the X coordinate to see if it was on one of the buttons. This is achieved through nested if statements with the boolean AND operator &&.

Variable z is being used to aid debugging, it increases when a touch is registered but doesn’t fall inside a button. It can easily be removed if desired.

If a button is touched, we want a haptic effect to play as quickly as possible. Therefore the first function is always;digitalWrite() to set the appropriate pin high. We also use the;drawCircle() function to make the button red, after which the button’s image is immediately re-drawn with;myFiles.loadBitmap(). A short delay ensures the button press is communicated to the haptic kit, after which the pin is returned to the low state.

It can be important to distinguish between buttons in close proximity

It can be important to distinguish between buttons in close proximity

HapticTouchKit.ino

Based on the Development Mode, you can download the code used in this application in the top right of the page.

With all of the hard work being done on the other Arduino, this code is fairly simple. The variables are set up in lines 43 and 44, and the pin modes are set with two additional lines (130 and 131) in the;setupPins() function.

The main loop() simply looks for any of the 4 inputs (connected to the Mega) to read high. When one does, the code plays the appropriate effect before resetting the variable to low and waiting one second to avoid accidentally playing two effects on one button press.

The only exception is with the variablepower, where we are also tracking the power state with the boolean variableswitchon. When true the kit plays the ‘turning on’ effect, which is a ramp up. Line 44 sets toswitchon false on start-up, so the first time it is pressed a ‘turning off’ effect (ramp down) will play. Then the state of isswitchon inverted in line 105.

Note that all haptic effects are being played with the LRA library 6 and that we have set the DRV2605 to LRA mode by selecting motor 3 in the motor.selectMotor(); function on line 57.

Future Work

The most obvious improvement would be to reduce the application to one processor. Whilst splitting the functions between Arduinos made it easier for wiring and using the Development Mode, in a real design everything would ideally be performed by the one unit.

An example interface has been provided, some applications might use a separate interface panel from the main display screen. This would be common where a larger screen is needed to display all the pertinent information, but a touchscreen of that size would be costly.

Contact Us

1.07 Canterbury Court
1-3 Brixton Road, London, SW9-6DE
[email protected]
+44 (0) 1932 252 482
(English / FR / DE / ES / IT)
+44 (0) 1932 325 353
PrecisionMicrodrives
@PMDri
Precision Microdrives
Mon. – Fri. 9:00 – 18:00 (BST / UTC)
* Charges vary depending on service provider and country