Last post I showed you how to create a 'thing' and to get the necessary certificates and keys to enable communication from the Pi up to the Amazon AWS; now we need to look into sending some data up there. I tried the Amazon library that is supplied with the AWSIot, but frankly, it had a bug I couldn't tolerate as well as really pretty, but incomplete documentation. The bug is that you can't run two instances of it on a Pi. If you start up a second instance, the first one starts to fail, and I want to have more than one thing that can send data. The documentation only shows how to use it with Java and Javascript; the python descriptions are just not there. Also, each piece of it leaves things out that you need and I didn't want to spend even more hours experimenting.
The tutorials helped some, but like I said, not a one of them actually worked when I tried them, so I used Nick's example from my last post and built a process that could send data up, then I converted it to use simple mqtt calls and simplified it a whole lot. Nick's example can't receive data from Amazon, so that part had to wait until I got some experience.
To send data from your Pi up to Amazon is actually pretty simple, so let's step through some code to illustrate it. First, you have to have mqtt client installed. You don't actually need mqtt server unless you already use it for your stuff, so install mosquito. You can refer back to my post where I discovered how easy to use mqtt actually is for my instructions <link>, or use one of the hundreds of others out there. Once you've got it in place and tested a bit, you're ready to put something together that will send data up to Amazon AWSIot; yep, I'm going to use the acronyms; you should be used to them by now.
import sys
import time
import paho.mqtt.client as mqtt
import ssl
import json
import pprint
from houseutils import timer, checkTimer
pp = pprint.PrettyPrinter(indent=2)
def on_awsConnect(client, userdata, flags, rc):
print("mqtt connection to AWSIoT returned result: " + str(rc) )
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed. You still have to do the
# reconnect in code because that doesn't happen automatically
client.subscribe ([(awsShadowDelta , 1 ),
(awsShadowDocuments, 1)])
# If you want to see the shadow documents to observe what is going on'
# uncomment the prints below.
def on_awsMessage(client, userdata, msg):
#print "TOPIC = ",
#print msg.topic
#print "PAYLOAD = ",
payload = {}
payload = json.loads(msg.payload)
#pp.pprint (payload)
#print ""
if msg.topic == awsShadowDocuments:
print "got entire shadow document"
#pp.pprint (reported)
elif msg.topic == awsShadowDelta:
print ("got a delta")
#print (pp.pformat(payload["state"]))
def updateIotShadow():
temperature = 79.1
# Create report in JSON format; this should be an object, etc.
# but for now, this will do.
report = "{ \"state\" : { \"reported\": {"
report += "\"temp\": \"%s\", " %(int(round(temperature)))
report += "\"lastEntry\": \"isHere\"" #This entry is only to make it easier on me
report += "} } }"
# Print something to show it's alive
print report
print("Tick")
err = awsMqtt.publish(awsShadowUpdate,report)
if err[0] != 0:
print("got error {} on publish".format(err[0]))
if __name__ == "__main__":
# these are the two aws subscriptions you need to operate with
# the 'delta' is for changes that need to be taken care of
# and the 'documents' is where the various states and such
# are kept
awsShadowUpdate = "$aws/things/house/shadow/update"
awsShadowDelta = "$aws/things/house/shadow/update/delta"
awsShadowDocuments = "$aws/things/house/shadow/update/documents"
# create an aws mqtt client and set up the connect handlers
awsMqtt = mqtt.Client()
awsMqtt.on_connect = on_awsConnect
awsMqtt.on_message = on_awsMessage
# certificates, host and port to use
awsHost = "data.iot.us-east-1.amazonaws.com"
awsPort = 8883
caPath = "/home/pi/src/house/keys/aws-iot-rootCA.crt"
certPath = "/home/pi/src/house/keys/cert.pem"
keyPath = "/home/pi/src/house/keys/privkey.pem"
# now set up encryption and connect
awsMqtt.tls_set(caPath, certfile=certPath, keyfile=keyPath, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLSv1_2, ciphers=None)
awsMqtt.connect(awsHost, awsPort, keepalive=60)
print ("did the connect to AWSIoT")
# Now that everything is ready start the mqtt loop
awsMqtt.loop_start()
print ("mqtt loop started")
# this timer fires every so often to update the
# Amazon alexa device shaddow; check 'seconds' below
shadowUpdateTimer = timer(updateIotShadow, seconds=10)
print("Alexa Handling started")
# The main loop
while True:
# Wait a bit
checkTimer.tick()
time.sleep(0.5)
The code above is more than an illustration of what to do, it actually runs on my machine and will update the AWSIoT shadow I'm currently using. I took a lot out of it because you probably don't do the sensors the same way I do, and there's no point in showing how I read the sensor data out of the database and format it.
Let's start at the top and work our way down in the code. The first two routines, on_awsConnect() and on_awsMessage() are the mqtt callbacks that you'll use to connect, subscribe and publish to the AWSIoT mqtt server. In on_awsConnect() I subscribe to the general topic,
"$aws/things/house/shadow/update/documents"
and
"$aws/things/house/shadow/update/delta".
The 'documents' one is where you can see the entire shadow document and the 'delta' is where you derive the command sent to your code by Alexa. But, for now, we're only going to discuss the data we want to send up and that is published to the topic,
"$aws/things/house/shadow/update",
because that is the first thing you need to conquer in getting this running. I only illustrate the subscriptions here so you can get a feel for the message interaction.
Now, specifically in on_awsMessage() I publish to
"$aws/things/house/shadow/update",
a specific format that AWSIot expects, it actually looks like this,
{ "state" : { "reported":
{
"temp": "79",
"lastEntry": "isHere"}
}
}
if you remove all the "\" crap. I did it as a string (probably a mistake) to cut down on the time it was taking me to try things. The "lastEntry" was added by me to make it easier to add entries to the JSON string as I implemented different sensors. I just copied that line, pasted it back in and changed the stuff to be what I wanted to work on next.
Yes, it's just that simple to send an update message, the hard part was finding out what the format was and the specific mqtt topic to publish to.
Now, down in the main code there's a few declarations of the topics used to make it somewhat simpler to code, and then the objects for the mqtt client and callbacks. The next bit,
awsHost = "data.iot.us-east-1.amazonaws.com"
awsPort = 8883
caPath = "/home/pi/src/house/keys/aws-iot-rootCA.crt"
certPath = "/home/pi/src/house/keys/cert.pem"
keyPath = "/home/pi/src/house/keys/privkey.pem"
awsMqtt.tls_set(caPath, certfile=certPath, keyfile=keyPath, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLSv1_2, ciphers=None)
awsMqtt.connect(awsHost, awsPort, keepalive=60)
The tutorials helped some, but like I said, not a one of them actually worked when I tried them, so I used Nick's example from my last post and built a process that could send data up, then I converted it to use simple mqtt calls and simplified it a whole lot. Nick's example can't receive data from Amazon, so that part had to wait until I got some experience.
To send data from your Pi up to Amazon is actually pretty simple, so let's step through some code to illustrate it. First, you have to have mqtt client installed. You don't actually need mqtt server unless you already use it for your stuff, so install mosquito. You can refer back to my post where I discovered how easy to use mqtt actually is for my instructions <link>, or use one of the hundreds of others out there. Once you've got it in place and tested a bit, you're ready to put something together that will send data up to Amazon AWSIot; yep, I'm going to use the acronyms; you should be used to them by now.
#!/usr/bin/python
import osimport sys
import time
import paho.mqtt.client as mqtt
import ssl
import json
import pprint
from houseutils import timer, checkTimer
pp = pprint.PrettyPrinter(indent=2)
def on_awsConnect(client, userdata, flags, rc):
print("mqtt connection to AWSIoT returned result: " + str(rc) )
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed. You still have to do the
# reconnect in code because that doesn't happen automatically
client.subscribe ([(awsShadowDelta , 1 ),
(awsShadowDocuments, 1)])
# If you want to see the shadow documents to observe what is going on'
# uncomment the prints below.
def on_awsMessage(client, userdata, msg):
#print "TOPIC = ",
#print msg.topic
#print "PAYLOAD = ",
payload = {}
payload = json.loads(msg.payload)
#pp.pprint (payload)
#print ""
if msg.topic == awsShadowDocuments:
print "got entire shadow document"
#pp.pprint (reported)
elif msg.topic == awsShadowDelta:
print ("got a delta")
#print (pp.pformat(payload["state"]))
def updateIotShadow():
temperature = 79.1
# Create report in JSON format; this should be an object, etc.
# but for now, this will do.
report = "{ \"state\" : { \"reported\": {"
report += "\"temp\": \"%s\", " %(int(round(temperature)))
report += "\"lastEntry\": \"isHere\"" #This entry is only to make it easier on me
report += "} } }"
# Print something to show it's alive
print report
print("Tick")
err = awsMqtt.publish(awsShadowUpdate,report)
if err[0] != 0:
print("got error {} on publish".format(err[0]))
if __name__ == "__main__":
# these are the two aws subscriptions you need to operate with
# the 'delta' is for changes that need to be taken care of
# and the 'documents' is where the various states and such
# are kept
awsShadowUpdate = "$aws/things/house/shadow/update"
awsShadowDelta = "$aws/things/house/shadow/update/delta"
awsShadowDocuments = "$aws/things/house/shadow/update/documents"
# create an aws mqtt client and set up the connect handlers
awsMqtt = mqtt.Client()
awsMqtt.on_connect = on_awsConnect
awsMqtt.on_message = on_awsMessage
# certificates, host and port to use
awsHost = "data.iot.us-east-1.amazonaws.com"
awsPort = 8883
caPath = "/home/pi/src/house/keys/aws-iot-rootCA.crt"
certPath = "/home/pi/src/house/keys/cert.pem"
keyPath = "/home/pi/src/house/keys/privkey.pem"
# now set up encryption and connect
awsMqtt.tls_set(caPath, certfile=certPath, keyfile=keyPath, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLSv1_2, ciphers=None)
awsMqtt.connect(awsHost, awsPort, keepalive=60)
print ("did the connect to AWSIoT")
# Now that everything is ready start the mqtt loop
awsMqtt.loop_start()
print ("mqtt loop started")
# this timer fires every so often to update the
# Amazon alexa device shaddow; check 'seconds' below
shadowUpdateTimer = timer(updateIotShadow, seconds=10)
print("Alexa Handling started")
# The main loop
while True:
# Wait a bit
checkTimer.tick()
time.sleep(0.5)
The code above is more than an illustration of what to do, it actually runs on my machine and will update the AWSIoT shadow I'm currently using. I took a lot out of it because you probably don't do the sensors the same way I do, and there's no point in showing how I read the sensor data out of the database and format it.
Let's start at the top and work our way down in the code. The first two routines, on_awsConnect() and on_awsMessage() are the mqtt callbacks that you'll use to connect, subscribe and publish to the AWSIoT mqtt server. In on_awsConnect() I subscribe to the general topic,
"$aws/things/house/shadow/update/documents"
and
"$aws/things/house/shadow/update/delta".
The 'documents' one is where you can see the entire shadow document and the 'delta' is where you derive the command sent to your code by Alexa. But, for now, we're only going to discuss the data we want to send up and that is published to the topic,
"$aws/things/house/shadow/update",
because that is the first thing you need to conquer in getting this running. I only illustrate the subscriptions here so you can get a feel for the message interaction.
Now, specifically in on_awsMessage() I publish to
"$aws/things/house/shadow/update",
a specific format that AWSIot expects, it actually looks like this,
{ "state" : { "reported":
{
"temp": "79",
"lastEntry": "isHere"}
}
}
if you remove all the "\" crap. I did it as a string (probably a mistake) to cut down on the time it was taking me to try things. The "lastEntry" was added by me to make it easier to add entries to the JSON string as I implemented different sensors. I just copied that line, pasted it back in and changed the stuff to be what I wanted to work on next.
Yes, it's just that simple to send an update message, the hard part was finding out what the format was and the specific mqtt topic to publish to.
Now, down in the main code there's a few declarations of the topics used to make it somewhat simpler to code, and then the objects for the mqtt client and callbacks. The next bit,
awsHost = "data.iot.us-east-1.amazonaws.com"
awsPort = 8883
caPath = "/home/pi/src/house/keys/aws-iot-rootCA.crt"
certPath = "/home/pi/src/house/keys/cert.pem"
keyPath = "/home/pi/src/house/keys/privkey.pem"
are pretty much boiler plate for specifying the port you have to use, the actual server you send to and the fully qualified path names to your certificates. You got the certs from the previous example and should have saved them somewhere. The reason for the full path names is so this code can run in any directory. Port 8883 is the one chosen by Amazon, you can't change it. Then,
awsMqtt.tls_set(caPath, certfile=certPath, keyfile=keyPath, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLSv1_2, ciphers=None)
awsMqtt.connect(awsHost, awsPort, keepalive=60)
cause tls encryption to be implemented (this comes free with mosquitto client) and the actual connection to the server. Next,
awsMqtt.loop_start()
you must have an mqtt loop of some kind so the queue can be read, and awsMqtt.loop_start() is non-blocking. This is important so you can control when the updates are sent to AWSIoT. The last little bit is using the simple timer I created to keep from starting a separate thread for timing something this simple. I used to use a much more complex timer set up, but that was a waste of resources. The timer code I used is described here <link>, but of course, you can do anything you want.
The way this code works is every so often (10 seconds in this example) the timer fires and calls updateIotShadow() which formats a message holding a temperature I just hard coded in, and sends it up to a specific topic on AWSIot using my credentials. This connects it to the right place and suddenly, I have the data available on Amazon.
Yes, that's literally all there is to sending data and having it show up there. You can verify this by going to the AWSIoT console and looking at your 'thing'; it will have an entry for 'temp' and a value of '79'. Now what you have to do is implement all your sensors in the message which will make you realize why I put the 'LastEntry' in there, and check it out in the console on Amazon.
So, now would be an excellent time to bookmark the Amazon AWSIoT console in your browser; you're going to be using it a lot. You'll also use bookmarks to the Amazon Lambda server, and the Alexa server a lot.
Next time, We're going to work on a Amazon Lambda function to receive requests from Alexa (I'll be using an Amazon Dot) and respond with the temperature we just sent up. It'll be in two parts because it's mildly complex and it's best to take this stuff on a bite at a time.
Have fun.