• Overview
• Architecture
  • Alexa Skill
  • Interfacing with the Raspberry Pi
  • Communication flow controlling the TV via voice
  • Design Caveats
    • Request/Response over Pub/Sub
    • Pub/Sub inside the Lambda
• Setup
  • Setup IoT
  • Login with Amazon (LWA)
  • Create the Smart Home Skill
  • Create the Lambda for the Skill
  • Create S3 bucket to store the Lambda sourcecode
  • Package the sources and upload to S3
  • Deploy the Lambda into the AWS cloud
  • Configure the Skill
  • Account Linking and Device Discovery
  • Setup the Raspberry Pi
• Debugging
  • Local deployment of the Lambda
  • Lambda deployed to the cloud
  • MQTT
  • Node-RED
  • Alexa

This project integrates Amazon Alexa with a Raspberry Pi in order to control home devices. The device used as an example is a Gutmann cooking hook. Gutmann provides the TZ 602 IR remote to control the device. A Irdroid USB Infrared Transceiver is attached to the Raspberry Pi to mimic the signals of the TS 602. Adapting the functionality to other devices should be straight forward.

In order to interact with Alexa a skill needs to be created. There are a variety of Alexa skill types. The Smart Home Skill has the advantage that it is aware of the typical voice interactions with a smart device. The alexa-smarthome project explains the basics how to setup this skill type. Note that a Custom Skill could have been used, too. However, this has the disadvantage that the skill name must be mentioned in all the interactions with Alexa. So instead of alexa, turn of the cooking hood you will have to say alexa, turn of the cooking hood using my smart home.

From a programmers model the central component of a skill is a AWS Lambda function that gets called by Alexa services. To simplify the deployment and testing of the Lambda this project uses the AWS Serverless Application Model (SAM). To leverage the advantages of a typed language the Lambda is written in TypeScript

There is no way that the Alexa device can directly interact with other local devices on the home network. Instead it transfers voice data to the AWS cloud where it gets processed and required functions are triggered. This means, that the Raspberry PI must somehow be reachable from the internet which raises security questions. For example it is considered a bad security practice to forwarding a port from your home router to the Raspberry Pi. Instead Amazon IoT core will be used to expose a topic through an MQTT-Broker. The Raspberry Pi actively subscribes as a client to this topic though an encrypted channel. On the Raspberry Pi Node-RED handles the piping of the incoming MQTT messages to the Irdroid device.

An example flow of events is shown in the following sequence diagram:

If the smart device would be securely reachable by the Lambda, a conventional HTTP request/response between the two would be the straight forward solution. Since IoT was used to ensure security and reachability, the semantics of request/response must be mapped onto a pub/sub system. As shown in the communication flow, this creates some complexity managing subscribing and publishing to request topics and to response topics, each unique for every Lambda function call. Note that the functionality is contained in the IotTransceiver class of this project.

Typically a Lambda communicates with IoT through the IoT REST API. The AWS SDK contains the AWS.IoTData class for this purpose. This API has the advantage that it is simple and fast. It however does not allow subscriptions to topics. Thus it is not possible to receive responses back from the TV device.

In order to get feedback from the smart device, the Skill Lambda therefore uses the aws-iot-device-sdk to connect to IoT with the MQTT over WebSocket protocol. This however means that in case of a cold Lambda some time (1-2 seconds) is required to setup the connection between the Lambda and IoT core.

In order to manage Skills, IoT and to deploy the lambda an Amazon developer account and an AWS account are required.

IoT should be setup first, since the endpoint of its message broker must be configured in both the Skill Lambda and the Raspberry Pi, afterwards. IoT allows physical devices like the Raspberry Pi to communicate via Pub/Sub with an MQTT broker managed by Amazon. The device is represented by a so called Thing in the AWS cloud. Certificate files deployed to the device are used to secure the communication with the broker. The set of certificate files is represented by a Certificate object and associated with the Thing. Finally a Certificate is associated with one or more Policies. A Policy contains a JSON document that describes which operations or resources a device can use. The following figure shows the relation between all involved items.

Setting up IoT involves the following steps:

• In the AWS Console select the IoT Core service.
• In the AWS IoT menu click Secure -> Policies -> Create
• Provide RaspberryPi as policy name and click Advanced mode
• Copy&paste the contents of this iot policy document into the JSON editor
• in the Resource sections of the file adjust the AWS region and the AWS Account ID and click Create
• In the AWS IoT menu click Manage -> Things -> Create -> Create a single thing
• In the Add your device to the thing registry dialog provide the name RaspberryPi and click Next
• In the Add a certificate for your thing dialog click the recommended One-click certificate creation
• Download the certificate, the public key and the private key
• Follow the link to download the root CA for AWS IoT
• click Attach a policy
• select the previously create policy with the name RaspberryPi and click Register Thing
• In the AWS IoT menu click on Settings and note down the name of the Endpoint
• As a final step copy the four downloaded certificate files onto the Raspberry Pi.

Typically Smart Home hardware vendors operate their own cloud platform. Their users sign up and register their devices with the platform making them accessible from the internet. Within the Alexa app the user links his platform account with Alexa and hereby makes it accessible via an Alexa skill that is typically provided by the vendor.

Since there is no vendor cloud platform in this project the account linking required by the Smart Home Skill is done using Login with Amazon. This service acts as an OAuth 2 provider required to perform the account linking. Log into the developer console and under Login with Amazon create a new Security Profile.

Provide the following values for the required fields:

After clicking Save, make sure you write down the Client ID and the Client-Secret. It will be required later.

1. Log into your developer account on https://developer.amazon.com/login.html
2. Go to Alexa > Alexa Console > Skills > Create Skill
3. In the dialog provide the name of the skill: mysmarthome and select Smart Home as model
4. In the Smart Home dialog note down the Skill ID and click Save

• Open the file template.yml and insert the Skill ID into the AlexaSkillKit -> Id entry at the top of the file.
• Open the file src/iot/IotTransceiverFactory.ts and change the variable iotEndpoint to the Endpoint noted down before when setting up IoT

Decide for a S3 bucket name to store the sourcecode. Insert this name as bucketName in the config section of the file package.json. Now create the bucket either in the AWS Console or with the following aws cli command. Note that the name mysmarthome-skill was chosen in this example.

aws s3 mb s3://mysmarthome-skill

Use the following npm command to transfer the Lambda code into the S3 bucket:

npm run package

After the sources have been uploaded to S3 the Lambda can be deployed with the command:

npm run deploy

In the AWS console find the Lambda that has just been created and note down its ARN.

In the skill's configuration tab enter the following values:

• Default endpoint: ARN of the Lambda that was noted down before

Under Account Linking configure the following values:

• Authorization URI = https://www.amazon.com/ap/oa
• Access Token URI: https://api.amazon.com/auth/o2/token
• Your Client ID: LWA Client ID that was noted down before
• Your Secret: your client secret from LWA noted in a previous step
• Scope: profile:user_id (click Add Scope first to add)
• Client Authentication Scheme: HTTP Basic (Recommended)
• Note down the Alexa Redirect URLs

Now open the LWA security profile created earlier and visit the Web Settings dialog. Provide each of the Redirect URL values from your skill in the Allowed Return URLs field.

After the Login with Amazon is configured and the Skill Lambda is deployed the user's account must be linked with the skill and the TV device must be discovered. The Alexa mobile app can be used for this purpose:

Note that the dotted arrows indicate that the Lambda could use the Raspberry Pi to discover other devices in the home network. In this example project the Discover.Response is hard coded into the Lambda.

Node-RED needs to be installed on the Raspberry Pi and configured to Autostart on boot. Node-RED can then be reached with a browser under the IP-address or hostname of the Raspberry Pi using the URL: http://<hostname>:1880

After downloading and importing the required flow the browser should show this figure:

Before deployment, the two MQTT nodes receive request and publish response need to be configured with the downloaded certificate files:

• Double click on the node and edit the MySmartHome MQTT server
• In the Server field enter the Endpoint noted down before when setting up IoT
• Edit the TLS-Configuration called AWS-RaspberryPi
• Provide the path to the certificate, the private key and the root CA and click Update

Now the flow can be deployed to the device by pressing the deploy button in the upper right. The following will happen on the Raspberry Pi:

• The receive request node connects to the MQTT broker and subscribes on a request topic for incoming request messages. The message is a string encoded JSON array of key names to be pressed on the TV. E.g. "[\"VolumeUp\",\"VolumeUp\"]".
• In the to object node each incoming message is converted into a JavaScript object
• The split keys node now splits up the array and generates a message for each element
• The key json node transforms the message into a payload that can be posted to the TV. E.g. {"key":"VolumeUp"}
• Since the API of the Philips TV cannot handle bursts of requests it needs to be rate limited by the rate limit node
• The post key node posts the payload to the REST-API of the TV which sets a statusCode in the message
• The Join node waits until all TV response codes from the split messages have arrived
• The response json node creates another payload that shall be sent back through MQTT. E.g. {"status":"OK"}
• The publish response node publishes the payload on a response topic

Using the Serverless Application Model Command Line Interface (SAM CLI) the Skill can be started and debugged locally. Run the following command to send a Discovery.request event to the lambda and open a debugger port. Note that the function name MySmartHomeSkill represents the name given to the Skill in the template.yml file.

sam local invoke -e test/events/Discovery.request.json -d 5858 MySmartHomeSkill

The debugging option -d 5858 can be omitted. If present the command waits for a debugger to connect on port 5858. In case you want to connect Intellij IDEA as debugger select Attach to Node.js/Chrome as Run/Debug Configuration and adjust the port to 5858.

The Lambda logs to the AWS CloudWatch service. The logs can be found in the AWS Console under CloudWatch -> Protocols.

In the AWS Console und IoT Core -> Settings it is possible to make the service report problems to CloudWatch as well. This turned out to be helpful debugging authentication/connection problems.

In Node-RED one can activate the green debug nodes by pushing the button on their side. After clicking the deploy button again Node-RED writes the messages of the connected node into the debug area on the ride side of the screen. By clicking the button on the side of the inject example node it is also possible to simulate an incoming MQTT message from the cloud.

In the Alexa developer console select mysmarthome skill -> Test -> Alexa Simulator. In the text field it is possible to type utterances and see/hear the results from Alexa.

The Raspberry PI uses the lirc package to send infra-red signals through the Irdroid USB Infrared Transceiver. Execute the following commands to install the package:

sudo apt-get update
sudo apt-get install lirc

Note that Irdroid requires the ir_toy driver. This driver is supported by lirc starting from version 0.9.2.

Several configuration files need to be copied into the /etc folder on the Raspberry Pi. Afterwards restart the lircd.service with the command:

sudo systemctl restart lircd.service

Other relevant commands for setting up lirc are:

# Use Irdroid to record the infra-red signals from the original TZ 602 remote 
irrecord -d /dev/lirc0 Gutmann-TZ602.conf

# send a lirc command as infra-red signal
irsend SEND_ONCE TZ_602 KEY_POWER