Add Voice Commands to Your Next Project with Bluetooth LE

This project uses the Alorium Technology XLR8, a Bluetooth LE HID Implementation, and the EasyVR voice recognition module to add voice-activated movements to your next project. Or, you can build on the ideas to create a voice-activated remote control.

About the Project

This project uses an Alorium Technology XLR8 to integrate a Voice Recognition Module with a Bluetooth HID device to create a remote voice-activated controller. There are many mills, lathes, plasma tables, and printers on the market that are controlled by computer keyboards. Often, operators find that a part of the machine needs to be moved during loading and unloading of the machine when the operator is handling materials and away from the computer.
This project detects verbal commands and transmits keypresses via Bluetooth LE when a machine operator's hands are too full, too dirty, or too far away to safely operate the controls. It also recreates the HID Keyboard reports for "Page Up", "Page Down", and the keyboard "Arrow Keys", which correspond to directions of movement for the popular CNC controller Mach3.
Alternatively, use the ideas, hardware, and code provided here to build a different project—a voice-activated television remote control, a voice-activated mouse, or a voice-activated robot.
The finished project does not require internet connectivity.

Part Required	Cost	More Information
Alorium Technology XLR8	$75	Product Brief \| Website
Adafruit Bluefruit LE Shield	$20	Website Website
EasyVR Shield and Module	$52	EasyVR Manual
Bluetooth 4.0 USB Dongle	~$10	Purchase any Bluetooth 4.0 compliant
8Ω Speaker	$5	The speaker is optional but recommended
5V Power Supply	$10	Any 5V-1A (or greater) supply should do.

Alorium Technology XLR8

The Alorium Technology XLR8 is a drop-in replacement for the Arduino Uno R3. It was designed to allow Arduino Uno R3 users to have increased performance and capabilities without completely redesigning their projects for a more advanced 16- or 32-bit microcontroller. The XLR8 uses an Altera MAX 10 FPGA to emulate the Arduino's functionality (ATmega328 instruction set). The board is built to the dimensions of the UNO R3 and physically modeled off of the SparkFun RedBoard. For this project, AloriumTech provided the XLR8 board.

XLR8 pinout from Alorium Technology.

Not all of the 50,000 logic elements on the FPGA are needed to recreate the Arduino, which leaves plenty of room for configurable "Xcelerator Blocks" that provide improved performance and response over the ATMega328 in several areas.

Representation of gate use on the FPGA from Alorium Technology.

The XLR8 blocks are enabled and loaded as libraries through the Arduino IDE, and the end-user never needs to know that they are using an FPGA instead of an 8-bit microcontroller. Current XLR8 blocks include NeoPixel Control, Servo Control, Floating Point Math, and Enhanced Analog-to-Digital (ADC) converter (12-bit resolution at 154k samples/second). Additional improvements include additional Serial Peripheral Interface (SPI) ports.
The XLR8 can also be used for FPGA development, visit this page for more information.
Getting Started with the XLR8.

BlueFruit LE Shield

This shield communicates with the host controller through the SPI. This module can be configured for a variety of uses that range from transparent UART mode to HID keyboard emulation and responds to a full AT command set implemented by Adafruit. For this project, AAC purchased the Bluefruit LE Shield.

Image of the Bluefruit LE Shield from Adafruit.

To use this shield with the XLR8, connect the SPI CLK (pin 13) to GND with a 100Ω - 200Ω resistor.

EasyVR

The EasyVR Shield and Module is a multi-language voice recognition module that can recognize pre-programmed and user-programmed voice commands and execute arbitrary code on an Arduino. It is programmed through a 6-pin UART connection (J7) or with an Arduino by plugging the module into a shield. For this project, AAC purchased the shield and module.

The front and reverse of the easy VR 3.0 module.

The product webpage indicates that this module comes with 26 speaker-independent commands (in English, Italian, Japanese, German, Spanish, and French) and can be programmed with additional speaker dependent commands. It can also output 21 minutes of pre-recorded sounds or speech and can record and play back up to 120 seconds of live messages. The module can be programmed with or without a computer. I recommend using the computer and the included software EasyVR Commander.

Project Overview

For this project, an EasyVR 3.0 Module will be attached to an EasyVR Shield, its purpose it to recognize keywords and alert the Alorium Technology XLR8. The Alorium Technology XLR8 will then send corresponding commands to the Bluefruit LE Shield. The Bluefruit LE Shield will connect to a Bluetooth USB dongle plugged into a computer.

Project connection overview

This board arrangement uses the Adafruit Bluefruit SPI shield and the EasyVR3.0 shield stacked on top of the Alorium Technology XLR8.

Clockwise from top left: Top and bottom sides of the Bluefruit LE shield, the complete project, top and bottom of the EasyVR 3.0 shield.

EasyVR Shield Setup

You can solder the module directly to the shield with the included male headers, but it will be very difficult to remove later. If this is a concern, solder female headers to the shield and male headers to the module.
You must move the SMD resistors on the bottom of the board marked "101." The product ships connecting Tx/Rx to pins 12/13, but to avoid conflict with the other shield, Tx/Rx need to connect to pins 8/9.

Bluefruit Shield Setup

Solder stackable headers on the Bluefruit shield. Use a sharp knife to break the trace that connects CS to pin 8 and reattach CS to pin 10. Place a 100 Ω - 200 Ω resistor on the breadboard area of the shield and use it to connect the SPI CLK pin (13) to GND.

Bluefruit Bluetooth Background Information

Bluetooth Low Energy (BLE)

Image courtesy of Aislelabs.

Bluetooth allows for short-range data transmission between compatible devices. Fortunately for our project, Adafruit did the hard work of implementing the protocol stack and left us with devices that can be relatively quickly and easily programmed for our desired purpose. All of their products are thoroughly documented with one or many sample projects on Adafruit's website and all software & firmware can be found here.

About Bluetooth Connections

Two Bluetooth devices in radio range of one another begin an inquiry process. A device sends an inquiry and the other device responds to the inquiry with its MAC address, name, and other descriptors. The devices page (connect) with one another at the address sent during the inquiry. While connected, the devices pair, and share security information with one another, often with some type of authentication. While paired, the devices bond and save the authentication information so that they can automatically reconnect whenever they are within range of one another.
During prototyping, it is often necessary to return the Bluefruit device to a factory-default state, erasing its stored bonding information. Unfortunately, the Bluetooth USB dongle attached to the computer will not know that your device has reset.
The Bluefruit device begins the process at the inquiry stage again, while the device at the computer has already completed the bonding stage and the devices will never connect. To correct this, delete the Bluefruit device information from the computer and go through the process again. Once you are done prototyping, remove the software flag that allows for a factory reset to eliminate this issue.

Upgrading Bluefruit

Both Bluefruit devices can be connected and upgraded wirelessly using Adafruits "Bluefruit" app and the instructions on their page—the app is available in the Apple App Store and on Google Play.

Bluefruit Configuration

The Bluefruit SPI needs to be configured to be a keyboard—this is accomplished through a series of AT commands. First, change the name to something meaningful.

ble.sendCommandCheckOK("AT+GAPDEVNAME=AllAboutCircuitsKeyboard")

Then the device needs to enable services and advertise that it is a Human Interface Device (HID) by changing its Generic ATTribue (GATT).

ble.sendCommandCheckOK(F( "AT+BleHIDEn=On" )

Next, enable keyboard functionality.

ble.sendCommandCheckOK(F( "AT+BleKeyboardEn=On" )

A soft restart is required due to the change in services.

ble.reset()

Sending Printable and Non-Printable Characters and Key Combinations

The Bluefruit device can now send printable and non-printable characters to a linked device. Many characters are easy to send—most ASCII character codes sent through the UART Rx line will be interpreted and transmitted to a remote Bluetooth connected device without any additional work.
A microcontroller that transmits to the Bluefruits' UART Rx at 115200 Baud (8, N, 1) will be able to print or write characters to the serial line and have them appear on the computer screen. The Bluefruits are programmed to take the ASCII code and convert it to a Bluetooth HID keycode, and that keycode is sent to the remote computer.
Non-printable characters and key-combinations (Ctrl-Alt-Delete, Shift+Up Arrow, mute, volume up, function keys, etc...) must be delivered to the Bluefruit devices as HID Keyboard Reports.
Both Bluefruit devices can send printable and non-printable characters through the Bluetooth HID reports. To send a non-printable character (or characters) requires a string of hex characters followed by a release code.

ble.println("AT+BLEKEYBOARDCODE=02-00-04-00-00-00-00")
ble.println("AT+BLEKEYBOARDCODE=00-00")

**Note: This is equivalent to pressing and releasing [Shift]+[a] on a keyboard and immediately releasing it.

HID codes are not the same as American Standard Code for Information Exchange (ASCII) codes. ASCII codes have a unique number assigned to every character, in ASCII, an "A" is 65, and an "a" is 97. HID keycodes use modifier keys to expand a limited number of base codes. The HID Keycode sequence for an "a" is "00-00-04" and the sequence for "A" is "02-00-04." The first byte modifies the characters that follow.
The second byte that follows the modifier is reserved and is always 0x00. The third byte is the key that you want to press, (again, using the HID Key codes, not ASCII). Up to four additional key codes can be sent in the string to allow for complex multi-key macros. Next, ensure that the keys are released by sending "00-00". Using this method, you can duplicate any single key-press, any key-press combination, and any combination sequence to control almost any computer program or Bluetooth connected device—whether that is adding voice commands that allow hands-free fast forward in Adobe Premiere, or creating custom game controller combinations for a game console.
The HID Usage Tables provide a complete list of key codes and modifiers.

Preparing the EasyVR Software

Download and install the software EasyVR Commander from the product page. It installs the programs QuickSynthesis (Sensory->QuickSynthesis), Easy VR Commander (Veear->EasyVR Commander), and QuickT2SI (Sensory->QuickT2SI).

Quick Synthesis (optional—adds more sounds to your module)

Quick Synthesis allows you to prepare custom sounds for your module. The EasyVR module has one default beep built in, but by installing additional sounds you can programmatically link sounds to events to provide audible feedback for voice-recognition successes and errors.

Select several WAV sounds to use. Use a website such as http://www.bfxr.net/ to create and save three or more custom WAV sounds to a folder on your computer.
Open Sensory QuickSynthesis:
1. Select File » New
  At the prompt select "RSC4 Family"
2. Type a name for your project and save as type ".qxp"
3. Select Edit » Add WAV file
  Browse to the folder on the computer where you saved your WAV files, and select all that you wish to use
  Click Open
4. At the prompt, use the default values:
  SX-6 8K: 7800 bps
  ☑ Silence Detection Enable
  Dynamic Range Compression: None
  High-Frequency Pre-emphasis: None
  De-emphasis on Playback: None
  Select "Ok for All"
  Select "Auto Label All"
5. Select Compress»Compress
6. Select Build»Build
  ☑ Build Linkable Module (most cases)
  ☑ Load in CONST space (share ROM with program)
  ☑ Load above or at 0
  ◻ Include Lipsync Data
  Select "Ok"
  Select File » Save
  Select File » Exit.

Easy VR Commander (Required)

Easy VR Commander allows you to record Speaker Dependent words in 15 different groups, and also allows you to record a custom "Trigger" word in addition to the predefined Trigger word "Robot." Default Speaker Independent words built into the module include "Robot", "Action", "Move", "Turn", "Run", "Look", "Attack", "Stop", "Hello", "Left", "Right", "Up", "Down", "Forward", "Backward", "Zero", "One", "Two", "Three", "Four", "Five", "Six", "Seven", "Eight", "Nine", and "Ten." All words (Speaker Dependent and Speaker Independent), when detected, can be used to execute arbitrary code on the Arduino.

Begin by plugging your EasyVR module into the Easy VR shield, and the Easy VR shield into your XLR8. Set the jumper on the shield to the update (UP) position.
Select File » Port » COM X (where X is the UART port address at the location of the host microcontroller board):
1. Select Help » Update Firmware
2. Navigate to the firmware in the install directory "C:\Program Files (x86)\VeeaR\EasyVR Commander\EasyVR3_VW_REV4.EVRFW"
3. Select Open
4. ☑ Slow Transfer
5. Select Download
Once the firmware update is complete, disconnect the host microcontroller board from power and USB, and then move the jumper on the shield to "PC." Reconnect the host microcontroller to power and USB.
In Easy VR Commander, select File » Connect
Select "Trigger" Group.
Select Edit » Add Command
Type a name for your Trigger command (e.g. "Computer")
Select Edit » Train Command:
1. Select Phase 1.
2. Speak your word into the microphone.
3. Select Phase 2.
4. Speak your word into the microphone.

QuickT2SI Software

The QuickT2SI allows for even further customization of the module by adding additional speaker independent recognition keywords. However, it is not a free product, and not necessary for this project, so nothing needs to be done. To read more about all of the products (and watch YouTube videos of how to go through the previous steps) visit http://www.veear.eu/demos/.

Preparing the Bluefruit

Go to the Apple App Store or Google Play store and download the free app called "Bluefruit" by Adafruit Industries.
Plug the Bluefruit into the XLR8 board and connect to power.
Use the Bluefruit app on your phone to connect to the module and allow it to update.
Go to your phone's Bluetooth settings and "forget" the Bluefruit module.

Preparing the Arduino IDE

Download and Install the latest version of the Arduino IDE.
Select Sketch » Include Library » Manage Libraries
1. Search for and install "Easy VR"
2. Search for and install "Adafruit nRF51" Libraries
3. Select "Close"
Select File » Preferences
1. Under Additional Boards Manager URLs, enter or append (after adding a comma) https://raw.githubusercontent.com/AloriumTechnology/Arduino_Boards/master/package_aloriumtech_index.json to the text box.
Select Tools » Board » "XLR8 Ver 2"

Testing the XLR8

Open the Arduino IDE.
Select File » Examples » XLR8Infor » GetXLR8Version
Select Tools » Board » XLR8 Rev 2
Select Tools » FPGA Image » 16 MHz Float, Servo, Neopixel r1682
Select Tools » Programmer » AVRISP mkII
Select Sketch » Upload
Select Tools » Serial Monitor
Select 115200 In the drop-down in the lower right-hand corner of the Serial Monitor

XLR8 Troubleshooting Ideas

No COM port visible: You need to install FTDI drivers. Visit https://learn.sparkfun.com/tutorials/how-to-install-ftdi-drivers
Serial port data garbled: Ensure that you selected 115200 Baud
Select Tools » FPGA Image » 16 MHz Float, Servo, Neopixel r1682. Then Select Tools » Burn Bootloader
Visit AloriumTech's support page to download and read the latest Users Manual (PDF).

Testing the EasyVR Shield

Attach the EasyVR Shield to the XLR8 board.
Ensure the jumper is in the "SW" position.
Connect the XLR8 board to an available USB port on your computer with a mini-USB cable.
In the Arduino IDE:
1. Select File » Examples » Easy VR » TestEasyVR
2. Change line 64 from "SoftwareSerial port(12, 13);" to "SoftwareSerial port(8, 9);"
3. Select Sketch » Upload
4. Select Tools » Serial Monitor

EasyVR Troubleshooting Ideas

Double check all soldering (look for cold solder joints, solder bridges, and mistakes around the 100Ω resistor).
Double check jumper location.
Visit http://www.veear.eu/faq/.

Testing the Bluefruit Shield

Attach only the Bluefruit Shield to the XLR8 board.
Connect the XLR8 board to an available USB port on your computer with a mini-USB cable.
In the Arduino IDE:
1. Select File » Examples » Adafruit nRF51 » HID Keyboard
2. Select Sketch » Upload
3. Select Tools » Serial Monitor
Go to your computer's settings, look for and add the device.

Bluefruit Troubleshooting Ideas

Use a Bluetooth 4.0 or greater USB adapter.
Ensure that you can connect and disconnect to the Bluefruit with your phone (Bluefruit will not connect to your computer while bonded to your phone).
Disable the "Factory Reset" flag.
Visit http://forums.adafruit.com and http://learn.adafruit.com.

Putting It All Together

Plug the Bluefruit shield into the XLR8, and plug the EasyVR shield into the Bluefruit Shield. Ensure that the mode jumper is in the "SW" position. Load the code below into the Arduino IDE and upload it to the XLR8 board.

About the Code

This sketch makes extensive use of switch cases to display errors returned from the Easy VR to a variable named "Err" and to change the values of two volatile variables called "Direction" and "Repetitions". If the Speaker Dependent keyword "Computer" is detected, the sketch expects to next hear a direction, and then a number of times to repeat the keypress.
Different circumstances might require that a key is pressed for a longer length of time instead of pressed quickly multiple times. This can be accomplished by inserting a delay before sending the "00-00" command that releases all depressed keys. The sketch makes extensive (although still not complete) efforts to handle errors and returns useful debug information to users through the serial port.
If the Speaker Independent keyword "Robot" is detected, the code will echo any characters typed into the serial monitor until a CR\NL is detected, at which point it starts listening for Trigger Words again.

                    /******************************************************************************************************/
/***************  Alorium Technology XLR8, EasyVR 3.0, Adafruit Bluefruit SPI Integration *************/
/******************************************************************************************************/
/*  This program allows users to create a voice-activate keyboard that can send desired keypresses and
    keycodes.

                Alorium Technology XLR8 Board  |  http://aloriumtech.com
                            Easy VR 3.0 Board  |  http://Veear.eu
                         Bluefruit SPI Shield  |  http://adafruit.com

  ARDUINO IDE INSTRUCTIONS
    In Arduino IDE, add "Adafruit Bluefruit NRF41" and "EasyVR" libraries.  Boards tested include XLR8
    and Sparkfun Redboard

  EASYVR V3.0 or SHIELD SETUP INSTRUCTIONS
    If using the shield note jumper location (should be in SW position.  Hot air rework component marked
    "101" (100Ω resistor) connecting TX/RX pins 12/13 to 8/9 to avoid conflict with Bluefruit Shield.
    Alternatively, remove mode jumper and wirewrap GND, 5V to Arduino Gnd and 5V, and module Tx/Rx to
    Arduino pins 8/9 (Tx -> 8, Rx -> 9)

  BLUEFRUIT SHIELD SPI SETUP INSTRUCTIONS
    Solder 2x6, 2x8, 2x8, and 2x10 pass-through headers on the shield.  On the underside of the shield,
    break the trace between CS and pin 8 and reattach CS to pin 10 with wirewrap or other method.

    Program by Mark Hughes for AllAboutCircuits.com.
    Code is based and heavily influenced by examples provided by Adafruit.com (lady ada, Kevin Townsend)
    and Veear.eu (library examples)

*/
// Software Includes
#include                               // #include 
#include                    // #include 
#include                            // #include 
#include                 // #include 
#include                      // #include 
#include                  // #include 
#include          // #include 

// Pin Definitions
#define BLUEFRUIT_SPI_SCK     13                // Attach 100Ω Pull-Down Resistor
#define BLUEFRUIT_SPI_MISO    12                // 
#define BLUEFRUIT_SPI_MOSI    11                // 
#define BLUEFRUIT_SPI_CS      10                // Default pin 8 conflicts with EasyVR shield
#define BLUEFRUIT_SPI_IRQ     7
#define BLUEFRUIT_SPI_RST     4
#define EasyVR_RX             8                 // Must move resistors on shield from pins 12/13
#define EasyVR_TX             9                 // to pins 8/9 to avoid SPI conflict

// Constants
#define BUFSIZE                        128      // Size of the read buffer for incoming data
#define VERBOSE_MODE                   false    // If set to 'true' enables bluefruit debug output
#define pcSerial SERIAL_PORT_MONITOR            // Used to debug and send keystrokes
#define FACTORYRESET_ENABLE 0                   // Set to 1 to reset bluefruit
#define MINIMUM_FIRMWARE_VERSION    "0.6.6"

// Global Variables
volatile char directions;                       // Shortcut to pass value without rewriting function
volatile char repetitions;                      // Shortcut to pass value without rewriting function

// Create Easy VR on a Software Serial Port
SoftwareSerial port(EasyVR_RX, EasyVR_TX);      // Create a software serial port
EasyVR easyvr(port);                            // Attach EasyVR to software serial port

//Create Bluefruit object with SPI
Adafruit_BluefruitLE_SPI ble(BLUEFRUIT_SPI_SCK, BLUEFRUIT_SPI_MISO, BLUEFRUIT_SPI_MOSI,
                             BLUEFRUIT_SPI_CS, BLUEFRUIT_SPI_IRQ, BLUEFRUIT_SPI_RST);

void error(const __FlashStringHelper*errorText) { // Print errors using pointer
  Serial.println();
  Serial.println(errorText);

}

//Groups and Commands
enum Groups { GROUP_0  = 0,};                 // Default start keyword "Robot"
enum Group0 { G0_COMPUTER = 0,};              // Trained keyword "Computer"

//Grammars and Words
enum Wordsets {SET_1 = -1, SET_2 = -2, SET_3 = -3}; // Action, Direction, Scale
enum Wordset1 { S1_ACTION = 0, S1_MOVE = 1, S1_TURN = 2, S1_RUN = 3, S1_LOOK = 4, S1_ATTACK = 5, S1_STOP = 6, S1_HELLO = 7 };
enum Wordset2 { S2_LEFT = 0, S2_RIGHT = 1, S2_UP = 2, S2_DOWN = 3, S2_FORWARD = 4, S2_BACKWARD = 5 };
enum Wordset3 { S3_ZERO = 0, S3_ONE = 1, S3_TWO = 2, S3_THREE = 3, S3_FOUR = 4, S3_FIVE = 5, S3_SIX = 6, S3_SEVEN = 7, S3_EIGHT = 8, S3_NINE = 9, S3_TEN = 10};

int8_t group, idx;                            // use negative group for wordsets

void setup()
{
  pcSerial.begin(115200);                     // setup PC serial port

  Serial.print(F("Initialising the Bluefruit LE module: "));
  if ( !ble.begin(VERBOSE_MODE) )
  {
    error(F("Couldn't find Bluefruit, make sure it's in CoMmanD mode & check wiring?"));
  }
  Serial.println( F("OK!") );

  if ( FACTORYRESET_ENABLE )
  {
    /* Perform a factory reset to make sure everything is in a known state */
    Serial.println(F("Performing a factory reset: "));
    if ( ! ble.factoryReset() ) {
      error(F("Couldn't factory reset"));
    } // end ! ble.factoryReset()
  } // end FACTORYRESET_ENABLE
  ble.echo(false);
  Serial.println("Requesting Bluefruit info:");
  ble.info();   // Print Bluefruit information

  Serial.println(F("Setting device name to 'AACVoiceKeyboard': "));
  if (! ble.sendCommandCheckOK(F( "AT+GAPDEVNAME=AACVoiceKeyboard" )) ) {
    pcSerial.println(F("Could not set device name!"));
  }

  /* Enable HID Service */
  Serial.println(F("Enable HID Service (including Keyboard): "));
  if ( ble.isVersionAtLeast(MINIMUM_FIRMWARE_VERSION) )
  { if ( !ble.sendCommandCheckOK(F( "AT+BleHIDEn=On" ))) {
      pcSerial.println(F("Could not enable Keyboard"));
    }
    else
    { if (! ble.sendCommandCheckOK(F( "AT+BleKeyboardEn=On" ))) {
        pcSerial.println(F("Could not enable Keyboard")); // End if
      }
    }
  } //end if(min version)

  /* Add or remove service requires a reset */
  Serial.println(F("Performing a SW reset (service changes require a reset): "));
  if (! ble.reset() ) {
    pcSerial.println(F("Couldn't reset"));
  }

  Serial.println();
  Serial.println(F("Go to your devices Bluetooth settings to pair your device"));
  Serial.println(F("then open an application that accepts keyboard input"));
  Serial.println();

bridge:
  // bridge mode?
  int mode = easyvr.bridgeRequested(pcSerial);
  switch (mode)
  {
    case EasyVR::BRIDGE_NONE:
      /* attempt to communicate at 115200 baud.  If not, establish communication at 9600 baud and change settings
      */
      port.begin(115200);                                         // Communicate at 115200 baud
      if (!easyvr.detect()) {                                     // If easyVR doesn't respond,
        port.end();                                               // close port
        port.begin(9600);                                         // Communicate at 9600 baud
        if (!easyvr.detect()) {                                   // If easy VR doesn't respond,error
          error(F("No EasyVR Detected at 9600 or 115200 Baud. Check Jumper Settings."));
        } else {                                                  // If easy VR does respond, change baud.
          if (easyvr.changeBaudrate(EasyVR::B115200)) {
            pcSerial.println("Baud rate increased from 9600->115200 Baud");
            port.end();                                           // Need to stop 9600 baud communication
            port.begin(115200);                                   // Reestablished at 115200 baud until next reboot.
          } // end if(easyvr.changebaurrate)
        } // end else
      } // end if(!easyvr.detect()

      pcSerial.println(F("Bridge not requested, run normally"));
      pcSerial.println(F("---"));
      break;

    case EasyVR::BRIDGE_NORMAL:
      port.begin(9600);           // setup EasyVR serial port (low speed)
      // soft-connect the two serial ports (PC and EasyVR)
      easyvr.bridgeLoop(pcSerial);
      // resume normally if aborted
      pcSerial.println(F("Bridge connection aborted"));
      pcSerial.println(F("----------------"));
      break;

    case EasyVR::BRIDGE_BOOT:
      // setup EasyVR serial port (high speed)
      port.begin(115200);
      pcSerial.begin(115200);
      // soft-connect the two serial ports (PC and EasyVR)
      easyvr.bridgeLoop(pcSerial);
      // resume normally if aborted
      pcSerial.println(F("Bridge connection aborted"));
      pcSerial.println(F("---"));
      break;
  } // end Switch mode

  // initialize EasyVR
  while (!easyvr.detect())
  {
    pcSerial.println(F("EasyVR not detected!"));
    for (int i = 0; i < 10; ++i)
    {
      if (pcSerial.read() == '?')
        goto bridge;
      delay(100);
    }
  }

  pcSerial.print(F("EasyVR detected, version "));
  pcSerial.println(easyvr.getID());

  if (easyvr.getID() < EasyVR::EASYVR3)
    easyvr.setPinOutput(EasyVR::IO1, LOW); // Shield 2.0 LED off

  if (easyvr.getID() < EasyVR::EASYVR)
    pcSerial.print(F(" = VRbot module"));
  else if (easyvr.getID() < EasyVR::EASYVR2)
    pcSerial.print(F(" = EasyVR module"));
  else if (easyvr.getID() < EasyVR::EASYVR3)
    pcSerial.print(F(" = EasyVR 2 module"));
  else
    pcSerial.print(F(" = EasyVR 3 module"));
  pcSerial.print(F(", FW Rev."));
  pcSerial.println(easyvr.getID() & 7);

  easyvr.setDelay(0); // speed-up replies

  easyvr.setTimeout(5);
  easyvr.setLanguage(0);

  group = EasyVR::TRIGGER;                  //<-- customize="" group="" span="" start="">
}

void loop(void)
{
  if (easyvr.getID() < EasyVR::EASYVR3)
    easyvr.setPinOutput(EasyVR::IO1, HIGH); // LED on (listening)

  if (group < 0) // SI wordset/grammar
  {
    switch (-group)
    {
      case 2: pcSerial.print("Say a Direction:  "); break;
      case 3: pcSerial.print("Say a Distance:   "); break;
    }
    easyvr.recognizeWord(-group);
  }
  else // SD group
  {
    pcSerial.println("Say a trigger word: \"Computer\" or \"Robot\" ");
    easyvr.recognizeCommand(group);
  }

  do
  {
    // allows Commander to request bridge on Zero (may interfere with user protocol)
    if (pcSerial.read() == '?')
    {
      setup(); return;
    }
  }
  while (!easyvr.hasFinished());

  if (easyvr.getID() < EasyVR::EASYVR3)
    easyvr.setPinOutput(EasyVR::IO1, LOW); // LED off

  idx = easyvr.getWord();
  if (idx == 0 && group == EasyVR::TRIGGER)         // detect built in word Robot
  {
    easyvr.playSound(0, EasyVR::VOL_FULL);          // beep

    pcSerial.println("Word: ROBOT");                // print debug message
    easyvr.playSound(3, EasyVR::VOL_FULL);          // Custom Beep (0 default)
    Serial.println(F("Enter the character(s) to send:"));
    Serial.println(F("- \\r for Enter"));
    Serial.println(F("- \\n for newline"));
    Serial.println(F("- \\t for tab"));
    Serial.println(F("- \\b for backspace"));

    Serial.print(F("keyboard > "));
    char keys[BUFSIZE + 1];
    getUserInput(keys, BUFSIZE);
    easyvr.playSound(2, EasyVR::VOL_FULL);          // Custom Beep "down up down"
    Serial.print("\nSending "); Serial.println(keys);
    ble.print("AT+BleKeyboard="); ble.println(keys);
    keys[BUFSIZE + 1] = {};

    if ( ble.waitForOK() )
    {
      Serial.println( F("OK!") );
    }
    else {
      error( F("FAILED!") );
      easyvr.playSound(1, EasyVR::VOL_FULL);
    }

    return;
  }

  else if (idx >= 0)
  {
    // beep
    easyvr.playSound(0, EasyVR::VOL_FULL);
    // print debug message
    uint8_t flags = 0, num = 0;
    char name[32] = "";
    pcSerial.print("Word: ");
    pcSerial.print(idx);
    if (easyvr.dumpGrammar(-group, flags, num))
    {
      for (uint8_t pos = 0; pos < num; ++pos)
      {
        if (!easyvr.getNextWordLabel(name))
          break;
        if (pos != idx)
          continue;
        pcSerial.print(F(" = "));
        pcSerial.println(name);
        String name[32] = "";
        break;
      } // end for uint_8t
    } // end easyvr.dumpGrammar
    action();  // get the words and numbers

    for (int n = 1; n <= repetitions; n++) {
      ble.print(F("AT+BLEKEYBOARDCODE="));
      pcSerial.print(F("AT+BLEKEYBOARDCODE="));
      switch (directions) {
        case 1: //Left Arrow
          ble.println(F("00-00-50-00-00-00-00"));
          pcSerial.println(F("00-00-50-00-00-00-00"));
          break;

        case 2: //Right Arrow
          ble.println(F("00-00-4F-00-00-00-00"));
          pcSerial.println(F("00-00-4F-00-00-00-00"));
          break;

        case 3: //Up Torch (Page Up)
          ble.println(F("00-00-4B-00-00-00-00"));
          pcSerial.println(F("00-00-4B-00-00-00-00"));
          break;

        case 4: //Down Torch (Page Down)
          ble.println(F("00-00-4E-00-00-00-00"));
          pcSerial.println(F("00-00-4E-00-00-00-00"));
          break;

        case 5: //Forward (Up Arrow)
          ble.println(F("00-00-52-00-00-00-00"));
          pcSerial.println(F("00-00-52-00-00-00-00"));
          break;

        case 6: //Backward(Down Arrow)
          ble.println(F("00-00-51-00-00-00-00"));
          pcSerial.println(F("00-00-51-00-00-00-00"));
          break;
      } // End Switch (Directions)
      delay(25);  // Need some delay between repetitive keypress
      ble.println(F("AT+BLEKEYBOARDCODE=00-00"));
      pcSerial.println(F("AT+BLEKEYBOARDCODE=00-00"));
      repetitions = 0;        // Reset Repetitions to prevent unintended loop trigger
    } // End for loop (repetitions)
    return;
  } // end else if (idx)
  idx = easyvr.getCommand();
  if (idx >= 0)
  {
    easyvr.playSound(0, EasyVR::VOL_FULL);  // beep
    // print debug message
    uint8_t train = 0;
    char name[32];
    pcSerial.print("Command: ");
    //    pcSerial.print(idx);
    if (easyvr.dumpCommand(group, idx, name, train))
    {
      //
      //      pcSerial.println(name); // "THREE", "DOWN", etc...
    }
    else
      pcSerial.println();
    // perform some action
    action();
  } // end if(idx)
  else // errors and timeout
  {
    if (easyvr.isTimeout()) {
      pcSerial.println(F("Timed out, try again..."));
    }
    int16_t err = easyvr.getError();
    if (err >= 0)
    {
      pcSerial.print("Easy VR Error: ");
      pcSerial.print(err, HEX);
      easyvr.playSound(1, EasyVR::VOL_FULL);  // Play error sound
      switch (err) {
        case 0x02: error(F("Too long")); break;
        case 0x03: error(F("Too noisy")); break;
        case 0x04: error(F("Spoken too softly")); break;
        case 0x05: error(F("Spoken too loud")); break;
        case 0x06: error(F("Spoken too soon")); break;
        case 0x07: error(F("Too complex")); break;
        case 0x08: error(F("Invalid SI weights")); break;
        case 0x09: error(F("Invalid setup")); break;
        case 0x11: error(F("Recognition failure")); break;
        case 0x12: error(F("Recognition result doubtful")); break;
        case 0x13: error(F("Recognition result maybe")); break;
        case 0x14: error(F("Invalid SD/SV template")); break;
        case 0x15: error(F("Invalid SI weights")); break;
        case 0x17: error(F("Incomplete pattern duration")); break;
        case 0x21: error(F("T2si state structure is too big")); break;
        case 0x22: error(F("T2si RSC code version/ Grammer Rom don't match")); break;
        case 0x23: error(F("Out of t2si Ram!")); break;
        case 0x24: error(F("Unexpected t2si pcSerial.print occured")); break;
        case 0x25: error(F("Ran out of time")); break;
        case 0x26: error(F("T2si bad macro or grammar parameter")); break;
        case 0x29: error(F("T2si layer size out of limits")); break;
        case 0x2A: error(F("T2si net structure incompatibility")); break;
        case 0x2B: error(F("T2si initialization not complete")); break;
        case 0x2C: error(F("T2si incorrect number of layers")); break;
        case 0x2D: error(F("T2si trigger recognised Out of Vocabulary")); break;
        case 0x2F: error(F("T2si utterance too short")); break;
        case 0x31: error(F("Play -- illegal compression level")); break;
        case 0x38: error(F("Play, erase, copy - mag doesn't exist")); break;
        case 0x39: error(F("Rec, copy - mag already exists")); break;
        case 0x4A: error(F("Bad release nubmer in speech file")); break;
        case 0x4E: error(F("Bad message data or SX technology file missing")); break;
        case 0x80: error(F("None of the above (out of grammar)")); break;
        case 0x81: error(F("Invalid data (for memory check)")); break;
        case 0xC0: error(F("No room left in software stack")); break;
        case 0xCC: error(F("T2si test mode error")); break;
      } // end switch(err)
    } // end if (err>=0)
  } // end else
} // end void loop()

void action()
{
  switch (group)
  {
    case GROUP_0:
      switch (idx)
      {
        case G0_COMPUTER:
          group = -2; // goto directions (left, right, etc...) wordset.
          break;
      }
      break;

    case SET_2:
      switch (idx) {  // directions Choice
        case S2_LEFT:     directions = 1; group = -3; break;
        case S2_RIGHT:    directions = 2; group = -3; break;
        case S2_UP:       directions = 3; group = -3; break;
        case S2_DOWN:     directions = 4; group = -3; break;
        case S2_FORWARD:  directions = 5; group = -3; break;
        case S2_BACKWARD: directions = 6; group = -3; break;
      } // end switch(idx)
      break;
    case SET_3:
      switch (idx) {  // Repetition choice
        case S3_ZERO:     repetitions = 0;  group = 0; break;
        case S3_ONE:      repetitions = 1;  group = 0; break;
        case S3_TWO:      repetitions = 2;  group = 0; break;
        case S3_THREE:    repetitions = 3;  group = 0; break;
        case S3_FOUR:     repetitions = 4;  group = 0; break;
        case S3_FIVE:     repetitions = 5;  group = 0; break;
        case S3_SIX:      repetitions = 6;  group = 0; break;
        case S3_SEVEN:    repetitions = 7;  group = 0; break;
        case S3_EIGHT:    repetitions = 8;  group = 0; break;
        case S3_NINE:     repetitions = 9;  group = 0; break;
        case S3_TEN:      repetitions = 10; group = 0; break;
      } // end switch(idx)
      break;
  } // end switch(group)
} // end action()

void getUserInput(char buffer[], uint8_t maxSize)
{
  memset(buffer, 0, maxSize);
  while ( Serial.available() == 0 ) {
    delay(1);
  }

  uint8_t count = 0;

  do
  {
    count += Serial.readBytes(buffer + count, maxSize);
    delay(2);
  } while ( (count < maxSize) && !(Serial.available() == 0) );
}

/****************************** How to Send Custom Keypresses and Macros ******************************/
/******************************************************************************************************/
/*
  To send custom macros and keypress combinations, send the command AT+BLEKEYBOARDCODE= followed by
  three to seven, dash separated bytes that consist of 2-digit hex codes that correspond to HID
  Keyboard codes (07).  Immediately follow with AT+BLEKEYBOARDCODE=00-00 to indicate that all keys
  were released.
    For example
      ble.println("AT+BLEKEYBOARDCODE=02-00-04-16-06-0C-0C"); // This sends a key combination
      ble.println("AT+BLEKEYBOARDCODE=00-00"); // This mandatory command release all depressed keys
    Generically, the full command looks like this:
      ble.println("AT+BLEKEYBOARDCODE=XX-00-YY-NN-NN-NN-NN");
        Where:
          MM (byte 1) is the hex code for a modifier key
          00 (byte 2) is a reserved hex code (always 00)
          NN (byte 3) is the desired key-press
          PP (byte 4-7) are optional additional key-presses
  ble.println("AT+BLEKEYBOARDCODE=00-00");

  Keyboard code format http://www.usb.org/developers/hidpage/Hut1_12v2.pdf  (pages 53-59)

    +-----------------------------------------------------------------------------------+
    |                              HID Keyboard Code Format                             |
    +-----------------------------------------------------------------------------------+
    |                http://www.usb.org/developers/hidpage/Hut1_12v2.pdf                |
    +-----------------------------------------------------------------------------------+
    | Byte 0x01 | Byte 0x02 | Byte 0x03 | Byte 0x04 | Byte 0x05 | Byte 0x06 | Byte 0x07 |
    +-----------+-----------+-----------+-----------+-----------+-----------+-----------+
    |  Modifer  |  Reserved | Keycode 1 | Keycode 2 | Keycode 3 | Keycode 4 | Keycode 5 |
    +-----------+-----------+-----------+-----------+-----------+-----------+-----------+
    |    0xMM   |    0x00   |    0xNN   |    0xPP   |    0xPP   |    0xPP   |    0xPP   |
    +-----------+-----------+-----------+-----------+-----------+-----------+-----------+
    +------------------------------------+    +----------------------------------------------------+
    |       Byte 01 -- Hex Modifier      |    | Modifier       Reserved       Key Code      Result |
    +------------------------------------+    +----------------------------------------------------+
    |    Modifer    |   Binary  | Hex ID |    |    00      -      00      -      04    |  |    a   |
    +---------------+-----------+--------+    +----------------------------------------+--+--------+
    |      None     | B00000000 |  0x00  |    |    02      -      00      -      04    |  |    A   |
    +---------------+-----------+--------+    +----------------------------------------+--+--------+
    |  Left Control | B00000001 |  0x01  |    |    00      -      00      -      1E    |  |    1   |
    +---------------+-----------+--------+    +----------------------------------------+--+--------+
    |   Left Shift  | B00000010 |  0x02  |    |    02      -      00      -      1E    |  |    !   |
    +---------------+-----------+--------+    +----------------------------------------------------+
    |    Left Alt   | B00000100 |  0x04  |
    +---------------+-----------+--------+
    |  Left Window  | B00001000 |  0x08  |
    +---------------+-----------+--------+
    | Right Control | B00010000 |  0x10  |
    +---------------+-----------+--------+
    |  Right Shift  | B00100000 |  0x20  |
    +---------------+-----------+--------+
    |   Right Alt   | B01000000 |  0x40  |
    +---------------+-----------+--------+
    |  Right Window | B10000000 |  0x80  |
    +---------------+-----------+--------+
*/

Going Further

If you are going to create this project, consider adding an operator passcode and additional Speaker Dependent words in the EasyVR Commander software. Also, consider moving the #defines, variables, and unchanging error codes out of the main sketch—I only included them in a single file to prevent information loss as the code makes its rounds on the web.

Sound Files and Code

Conclusion

This project demonstrates a way to add a voice activated keyboard and mouse to a machine, but it can be readily modified to add voice commanded key presses and macros to any device that can interpret Bluetooth LE—from a CNC plasma table to an XBOX One Controller.
And the next time that you create a project with an Arduino Uno R3, and the demands of the project exceed the technical capabilities of the Arduino Uno R3, consider Alorium Tech's XLR8 Board.

Breaking

19 May 2017