Quantcast
Channel: Desert Home
Viewing all 218 articles
Browse latest View live

Raspberry Pi Home Controller; In service and working

$
0
0
I finally made the plunge yesterday and brought my Rasberry Pi online and disconnected my old controller.  The Pi is now doing all the functions of the controller and presenting a web interface for monitoring control.  It was quite the learning experience.

In this post <link> I described the architecture I chose for the new device; I extended that to handle scheduled events such as turning the pool light on for an hour at 8PM to attract bats.  Bats are fun to watch when they swoop over the pool catching bugs.  Eventually I'll have a number of events because I intend to hook my drip watering system up to the network as well as a number of lights around the house.

I also added the three cloud services I update data to: Pachube (legacy Xively), the new Xively <link>, emoncms <link>, and ThingSpeak <link>.  They fit right in the architecture without changing much at all.  Over time I expect to extend it a bit more and have a tablet based application that can control the house like a remote control.  The hooks are all there to add things over time.

Wow, was there a lot to think about.  I had to make the processes restart after a power failure, which meant taking on the init process of linux.  I upgraded the init process to use upstart.  This piece of code is much friendlier than the older init process, but was a scary piece of work.  See, if I screwed that one up I would have to start all over because it would have made my little Pi fail to boot up.  So, I backed up the operating system image before I started and several times during the process.  Fortunately, I did it right and the system came right up and worked.  Now, all the processes start on boot up and are restarted whenever they die.  This makes it cool because I can make a few changes, kill the old process, and the new one takes off and does whatever job I added to it.

I also added an update process for dyndns.  When you get a controller like this working, it just makes sense to put it on the internet so it can be monitored from anywhere.  Services like dyndns make this possible for us regular folk that have normal consumer DSL service.  Our ip addresses change from time to time and we have to keep the internet name servers up to date.  To do this, I had to take on the mysteries of the cron process.  What a pain in the behind that was.  No, the documentation out there is correct, but it's so easy to make a mistake, and so hard to find it that I spent almost a day messing around with that piece alone.

But I got it working.  Here's a block diagram of the process architecture as it exists right this minute:


Notice how the cloud update services Xively, ThingSpeak, and emoncms just read from the database to update over the internet.  Using this technique, I can try out other cloud services without making any changes at all to the ones that already exist.  The event handler communicates with the House Monitor directly, it doesn't need to mess with any other processes to do its job.  The Web Server has interfaces to the controls and database to keep the internet away.  I'm trying to keep my neighbors from playing with my garage doors.

It's strange to imagine that a little arduino 2560 did most of this stuff for a couple of years.  When you think about it though, the arduino was written in c++ and hand tailored to handle stuff over a long period of development.  I only added the things I knew it could handle.  Now, I can add capabilities that the arduino just couldn't do, which also means that I can off load some of the work that is handled automatically by other network devices if I need to in the future.

One of the great things I haven't experimented with yet is the web interface.  I can easily edit new pages off the one I have now to show graphs, pictures, etc.  This could get to be fun and offer a platform for other folks to get ideas from (that I would then steal and put in mine).

Later today I hope to massively update my page on the House Controller (see the tabs at the top of the page) so that I can share the code and ideas that are currently running.  It'll be a bit bittersweet though, since I put so much time into the care and feeding of the old controller.  But, I'm sure I'll find a use for the parts in some future expansion of the system.

You can see the web page here <link> be kind though, it's a Raspberry Pi, not a giant web server.

Xively, I've just about given up on it.

$
0
0
I was reviewing a couple of my pages after changing my House Controller to a Raspberry Pi, and noticed that one of my charts was several hours behind the current readings. Then I noticed that the chart that I present on my House Controller was presenting yesterdays readings. After about three hours of chasing problems, it turned out that the Xively server is caching pages.

Yes, I'm trying to get the latest data I uploaded and the server is returning me the data from hours or even days ago because it sees the request hasn't changed and simply returns what it has in its cache. No, I wasn't changing the request I sent; it was a request for the latest data. Why would I change a request that asked for the latest?

 What I had to do to overcome the problem was to get the date and time and ask for data that ended with the current time instead. This made the request change each time I sent it and the server had to read the data to fulfill the request. This explains a lot of the problems mentioned on StackOverflow that I read while trying to get my own stuff to work. Additionally, I wanted to step through Xively's process of 'Deploy' and couldn't.  Seems the device itself has to have code on it to activate itself. There's no support for this in their library and I really didn't want to spend hours going through the debugging process that I would incur to try and meet their API. They could have simply put in a single unit deployment process on their web site so that little guys like me could step through it.

Their rates are huge, a grand being the lowest level with transaction costs on top of that; a far cry from the 10 bucks or so a month that pachube had back in the good old days before some faceless corporation bought them out. So, let's list a few things that I've noticed. They preserved the legacy feeds that we've been using, but provide no reasonable way of moving them to the new system. They cache data so we're very likely to get data back that is days old. They have no way to deploy a device in their new system without the device itself doing the work. They took away their forum for exchanging ideas, instead they expect us to post on StackOverflow which is specifically designed for problems. They even removed the forum so that the old ideas and techniques have disappeared. Yet, they post on Twitter that they are attending shows, giving speeches, and publishing articles on the prevalence of the Internet of Things.

Does anyone actually like this service since the latest change? Am I the only one that has seen a steady decline since the original developers presented this cool new tool that us home automation and control freaks went nuts over?

Sad.

GroveStreams Another Data Service

$
0
0
In my continuing search for sites to put my house data on I encountered Grovestreams.com on the Arduino forum.  Well, since I have a cool setup to try these things out I decided to give it a try.  At first I got totally lost in the jargon inherent to cloud services.  These things talk about streams of data, component architecture, registering a service; you know, the kind of language that simply puts people that make things to sleep.  I persevered.

Once I started to understand what they meant by messing up example after example, I was ready to try some code.  I grabbed their python example and updated it to save one of my devices.  Since I'm a power usage freak, I chose my real time measurement of power usage.  It worked.  That was actually really cool, so I added outside temperature.  It worked too.  Then I went a little nuts and added the rest of my house data to it.

Then I was on a roll.  I modified the code again to use the techniques I used on the other cloud services.  I put it under the APSscheduler to do one minute updates, added it to the init table so it would restart if it had trouble and let it go.  It's running right now keeping track of the data I collect around the house for me.  So, what about a chart to look at?  Well, they have a dashboard like some of the other services and I created one to hold some charts; these charts are embeddable:



And here's the outside temperature.



Pretty slick.  And yes, these are real time and will update to the latest data if you watch them long enough. We've seen this before, but it's nice to see a new service like this implement it first instead of years later. They also have other 'widgets', gauges and things. What impressed me about this site is the versatility.  They have capabilities that I won't have time to try out. They can derive data from other data and present it.  It should be possible to build a chart based on my power usage and the billing charges multiplied together and actually show graphically how much money it's costing me to keep cool in the summer.  That's actually a little depressing when you think about it.

They even have alerts that can send SMS and email to you.  I created an alert to tell me when my house power usage goes over 10kWh.  It was annoying when it came in the first time and even more annoying when it came in the second time.  I actually have real time notification that I'm wasting money that comes in on my phone.  I could easily expand this to alert me when the garage door is open after sundown.  That would help a lot keeping the pack rats from stealing my tools and burying them in a mound outside.

Here's the code I'm actually running to update my house data on their site:

The python Sketch
#!/usr/bin/python
import time
from time import sleep
from datetime import datetime
import sys
from apscheduler.scheduler import Scheduler
import logging
from simplejson import encoder as jsonEncoder
import httplib
import StringIO
import gzip
import sqlite3
import pprint

# I keep all current values in a data base
DATABASE='/home/pi/database/desert-home'

# These are the secret numbers I use to get into
# the grovestreams service
org = "putyourownsecretthinghere";
api_key = "samewithyoursecretapikey";

#If you want lots of messages and debug information
# set this to true
DEBUG = False

# the site accepts compression, might as well use it
def compressBuf(buf):
zbuf = StringIO.StringIO()
zfile = gzip.GzipFile(mode = 'wb',fileobj = zbuf, compresslevel = 9)
zfile.write(buf)
zfile.close()
return zbuf.getvalue()

def updateGrovestreams():
# This is VERY different from their examples.  I named
# my streams with something I could understand and read
# Probably not the best way to do it, but it works really
# well for me.
component_id = "desert-home-id"
rpowerStream_id = "power_usage"
otempStream_id = "outside_temp"
apowerStream_id = "apparent_power"
voltageStream_id = "voltage"
currentStream_id = "current"
pfactorStream_id = "power_factor"
itempStream_id = "inside_temp"
ptempStream_id = "pool_temp"
pmotorStream_id = "pool_motor"

# This object really helps when displaying
# the rather complex data below
pp = pprint.PrettyPrinter(depth=10)

#get the millis since epoch
# in unix the epoch began back in 1970, look it up
now = datetime.now()
nowEpoch = int(time.mktime(now.timetuple())) * 1000

#assemble feed and convert it to a JSON string
feed = {};
feed['feed'] = {}
feed['feed']['component'] = []
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()

comp = {}
comp['stream'] = []
comp['componentId'] = component_id
feed['feed']['component'].append(comp)
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()

# Now I'm going to fill in the stream values, open database
# I took a brute force approach to building the dictionary that
# is converted into JSON.  I could have been much more elegant
# in building it, but the folks just starting out would have
# had a tough time understanding it
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# So, you make a stream to stuff things into.  It's actually
# a python dictionary that we'll pass to a JSON encoder a ways
# down into the code.  I'll be adding entries to this as I pull
# items out of the database
stream1 = {}  
stream1['streamId'] = rpowerStream_id
stream1['time'] = [] 
stream1['data'] = []
comp['stream'].append(stream1)

current_value = c.execute("select rpower from power").fetchone()[0] 
stream1['time'].append(nowEpoch)
stream1['data'].append(float(current_value))
# this is a cool way to debug this kind of thing.
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()
# notice how I get and item out of the database
# and add it to the dictionary.  I'll do this
# several times
stream2 = {}
stream2['streamId'] = otempStream_id
stream2['time'] = []
stream2['data'] = []
comp['stream'].append(stream2)
current_value = c.execute(
"select currenttemp from xbeetemp").fetchone()[0]
stream2['time'].append(nowEpoch)
stream2['data'].append(float(current_value))

stream3 = {}
stream3['streamId'] = apowerStream_id
stream3['time'] = []
stream3['data'] = []
comp['stream'].append(stream3)
current_value = c.execute(
"select apower from power").fetchone()[0]
stream3['time'].append(nowEpoch)
stream3['data'].append(float(current_value))

stream4 = {}
stream4['streamId'] = voltageStream_id
stream4['time'] = []
stream4['data'] = []
comp['stream'].append(stream4)
current_value = c.execute(
"select voltage from power").fetchone()[0]
stream4['time'].append(nowEpoch)
stream4['data'].append(float(current_value))

stream5 = {}
stream5['streamId'] = currentStream_id
stream5['time'] = []
stream5['data'] = []
comp['stream'].append(stream5)
current_value = c.execute(
"select current from power").fetchone()[0]
stream5['time'].append(nowEpoch)
stream5['data'].append(float(current_value))

stream6 = {}
stream6['streamId'] = pfactorStream_id
stream6['time'] = []
stream6['data'] = []
comp['stream'].append(stream6)
current_value = c.execute(
"select pfactor from power").fetchone()[0]
stream6['time'].append(nowEpoch)
stream6['data'].append(float(current_value))

stream7 = {}
stream7['streamId'] = itempStream_id
stream7['time'] = []
stream7['data'] = []
comp['stream'].append(stream7)
current_value = c.execute(
"select avg(\"temp-reading\") from thermostats").fetchone()[0]
stream7['time'].append(nowEpoch)
stream7['data'].append(float(current_value))

stream8 = {}
stream8['streamId'] = ptempStream_id
stream8['time'] = []
stream8['data'] = []
comp['stream'].append(stream8)
current_value = c.execute(
"select ptemp from pool").fetchone()[0]
stream8['time'].append(nowEpoch)
stream8['data'].append(float(current_value))

stream9 = {}
stream9['streamId'] = pmotorStream_id
stream9['time'] = []
stream9['data'] = []
comp['stream'].append(stream9)
tmp = c.execute("select motor from pool").fetchone()[0];
if (tmp == 'High'): # a little special handling for the pool motor
motor = 2
elif (tmp == 'Low'):
motor = 1
else:
motor = 0
stream9['time'].append(nowEpoch)
stream9['data'].append(int(motor))

# all the values are filled in, close the database
dbconn.close() # close the data base
# This will print the entire database I just constructed
# so you can see what is going on
if DEBUG:
pp.pprint(feed)
print
sys.stdout.flush()
# exit() # I put this in for debugging.  It exits before
# the JSON string is constructed and sent off to grovestreams
# Of course you want to keep it commented until needed
#
# And this is where the JSON string is built
encoder = jsonEncoder.JSONEncoder()
json = encoder.encode(feed);
# and this will print it so you can see what is happening
if DEBUG:
print json # for debugging
print
sys.stdout.flush()

#Upload the feed
try:
print "Updating GroveStream ", time.strftime("%A, %B %d at %H:%M:%S")
sys.stdout.flush()
conn = httplib.HTTPConnection('www.grovestreams.com')

url = '/api/feed?&org=%s&api_key=%s' % (org, api_key)

compress = True
if compress:
body = compressBuf(json)
headers = {"Content-type": "application/json", "Content-Encoding" : "gzip"}
else:
body = json
headers = {"Content-type": "application/json", "charset":"UTF-8"}

conn.request("PUT", url, body, headers)

response = conn.getresponse()
status = response.status

if status != 200 and status != 201:
try:
if (response.reason != None):
print('reason: ' + response.reason + ' body: ' + response.read())
sys.stdout.flush()
else:
print('body: ' + response.read())
sys.stdout.flush()
except Exception:
print('HTTP Fail Status: %d' % (status) )
sys.stdout.flush()

except Exception as e:
print('HTTP Send Error: ' + str(e))
sys.stdout.flush()
   
finally:
if conn != None:
conn.close()

# I just discovered the statement below.
# someday I'll have go figure out what it really does.
if __name__ == '__main__':
print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
sys.stdout.flush()
logging.basicConfig()

#------------------Stuff I schedule to happen -----
scheditem = Scheduler()
scheditem.start()
# every minute update the data store on Xively
scheditem.add_interval_job(updateGrovestreams, seconds=60)
#
# A couple of people asked me why I put this statement in
# since I have it scheduled to happen every 60 seconds already
# Well, when you're debugging something it sucks to have to
# wait 60 seconds to see if you fixed it, so I do it 
# first, then let the scheduler take care of the rest.
#
updateGrovestreams()
while True:
time.sleep(20) #This doesn't matter much since it is schedule driven
 

Yes, in my usual form, it's got way too many comments.  It's also very inelegant; I'm not a big fan of strange statements that run really well but take an hour to understand so I just brute forced the URL and body creation of the request sent.  Notice that there isn't a library with obscure calls and methods and stuff.  This is all python code that I simply stole from them and added my particular items to.  Makes it nice that I don't have to learn how to use some special library.  I also put in a variable to control debugging so you can see what is happening.  It gets a little complex at a couple of points.

I didn't stop there.  I took their arduino example and modified it to update some items as well.  It worked first try.  The example had a little trouble since Grovestreams used Strings as a way of making the code more easily read and it started losing memory.  The poor little arduino has so little memory that I knew it would die over time, so I sent them a note describing the problem.  Guess what?  They responded!

Yes, in this 21st century world of minimal text messages, they actually answered me and came up with a new example that works better.  Unfortunately, I didn't save that one as an illustration for you, you'll just have to go to their site and get it yourself.  However, I did save the first version I tried:

The Arduino Sketch
/*

 Arduino GroveStreams Stream Feed via Ethernet

 The GroveStreams client sketch is designed for the Arduino and Ethernet.
 A full "how to" guide for this sketh can be found at https://www.grovestreams.com/developers/getting_started_arduino_temp.html
 This sketch updates several stream feeds with an analog input reading,
 from a temperature probe, via the GroveStreams API: https://www.grovestreams.com/developers/apibatchfeed.html
 The Arduino uses DHCP and DNS for a simpler network setup.
 The sketch also includes a Watchdog / Reset function to make sure the
 Arduino stays connected and/or regains connectivity after a network outage.
 Use the Serial Monitor on the Arduino IDE to see verbose network feedback
 and GroveStreams connectivity status.

 GroveStreams Setup:

 * Sign Up for Free User Account - https://www.grovestreams.com
 * Create a GroveStreams organization and select the Arduino blueprint
 * Enter a unique MAC Address for this network in this sketch under "Local Network Settings" 
 *    (Newer shields have the mac address on a sticker on the shield. Use that.)
 *    (A MAC address can also be generated within a GroveStreams organization: tools - Generate MAC Address)
 * Enter the GroveStreams org uid under "GroveStreams Settings" in this sketch 
 *    (Can be retrieved from a GroveStreams organization: tools - View Organization UID)
 * Enter the GroveStreams api key under "GroveStreams Settings" in this sketch  
 *    (Can be retrieved from a GroveStreams organization: click the Api Keys toolbar button, 
 *     select your Api Key, and click View Secret Key)

 Arduino Requirements:

 * Arduino with Ethernet Shield or Arduino Ethernet
 * Arduino 1.0 IDE

 Network Requirements:

 * Ethernet port on Router    
 * DHCP enabled on Router
 * Unique MAC Address for Arduino

 Additional Credits:
 Example sketches from Arduino team, Ethernet by David A. Mellis

 */

#include <SPI.h>
#include <Ethernet.h>
#include <MemoryFree.h>


// Local Network Settings
byte mac[] = {0x90, 0xA2, 0xDA, 0x00, 0x33, 0x33};        // Change this!!! Must be unique on local network. 
                                               // Look for a sticker on the back of your Ethernet shield.

// GroveStreams Settings
String gsApiKey = "You\'re going to need your own number here";   //Change This!!!
String gsOrg = "another number you need to put in";      //Change This!!!
String gsComponentName = "Temperature";                     //Optionally change. Set this to give your component a name when it initially registers.

char gsDomain[] = "grovestreams.com";   //Don't change. The Grove Streams domain. 
String gsComponentTemplateId = "temp";  //Don't change. Tells GS what template to use when the feed initially arrives and a new component needs to be created.
                                        // The blueprint is expecting "temp".

//GroveStreams Stream IDs. Stream IDs tell GroveStreams which component streams the values will be assigned to.
//Don't change these unless you edit your GroveStreams component definition and change the stream ID to match this.
String gsStreamId1 = "s1";   //Temp C - Random Stream. 
String gsStreamId2 = "s2";   //Temp F - Random Stream. Don't change.
String gsStreamId3 = "s3";   //Temp C - Interval Stream (20 second intervals). Don't change.
String gsStreamId4 = "s4";   //Temp F - Interval Stream (20 second Intervals). Don't change.

const int updateFrequency = 20 * 1000; // GroveStreams update frequency in milliseconds (the GS blueprint is expecting 20s)
const int temperaturePin = 0;          // Then Temperature pin number.    

// Variable Setup
String myIPAddress;  //Set below from DHCP. Needed by GroveStreams to verify that a device is not uploading more than once every 10s.
String myMac;        //Set below from the above mac variable. The readable Mac is used by GS to determine which component the feeds are uploading into.

long lastConnectionTime = 0;    //Don't change. Used to determine if the Ethernet needs to be reconnected. 
boolean lastConnected = false;  //Don't change. 
int failedCounter = 0;          //Don't change. 

// Initialize Arduino Ethernet Client
EthernetClient client;


void setup()
{
  // Start Serial for debugging on the Serial Monitor
  Serial.begin(9600);

  // Start Ethernet on Arduino
  startEthernet();
  randomSeed(analogRead(1));
}

void loop()
{

  // Print Update Response to Serial Monitor
  if (client.available())
  {
    char c = client.read();
    Serial.print(c);
  }

  // Disconnect from GroveStreams
  if (!client.connected() && lastConnected)
  {
    Serial.println("...disconnected");
    Serial.println();
    showMem();

    client.stop();
  }

  // Update sensor data to GroveStreams
  if(!client.connected() && (millis() - lastConnectionTime > updateFrequency))
  {
    String tempC = getTemperatureC();
    Serial.print (tempC);
    String tempF = getTemperatureF();
    Serial.print (tempF);
    updateGroveStreams(tempC, tempF);
  }

  // Check if Arduino Ethernet needs to be restarted
  if (failedCounter > 3 ) {
    
    //Too many failures. Restart Ethernet.
    startEthernet();
  }

  lastConnected = client.connected();
}

void updateGroveStreams(String tempC, String tempF)
{
  Serial.println("\nin update routine");
  if (client.connect(gsDomain, 80))
  {         

    //Assemble the url used to pass the temperature readings to GroveStreams
    // The Arduino String class contains many memory bugs and char arrays should be used instead, but
    // to make this example simple to understand we have chosen to use the String class. 
    // No none memory issues have been seen with this example to date.
    //We are passing temperature readings into two types of GroveStreams streams, Random and Interval streams.
    String url = "PUT /api/feed?&compTmplId=" + gsComponentTemplateId + "&compId=" + myMac + "&compName=" + gsComponentName;
    url += "&org=" + gsOrg + "&api_key=" + gsApiKey;

    url += "&" + gsStreamId1 + "=" + tempC;  //Temp C - Random Stream
    url += "&" + gsStreamId2 + "=" + tempF;  //Temp F - Random Stream
    url += "&" + gsStreamId3 + "=" + tempC;  //Temp C - Interval Stream (20 second intervals)
    url += "&" + gsStreamId4 + "=" + tempF;  //Temp F - Interval Stream (20 second intervals)

    url += " HTTP/1.1";
    Serial.print(url);
    client.println(url);  //Send the url with temp readings in one println(..) to decrease the chance of dropped packets
    client.println("Host: " + String(gsDomain));
    client.println("Connection: close");
    client.println("X-Forwarded-For: "+ myIPAddress); //Include this line if you have more than one device uploading behind 
                                                      // your outward facing router (avoids the GS 10 second upload rule)
    client.println("Content-Type: application/json");
    client.println();

    if (client.available())
    {
      //Read the response and display in the the console
      char c = client.read();
      Serial.print(c);
    }

    lastConnectionTime = millis();

    if (client.connected())
    {     
      failedCounter = 0;
    }
    else
    {
      //Connection failed. Increase failed counter
      failedCounter++;

      Serial.println("Connection to GroveStreams failed ("+String(failedCounter, DEC)+")");   
      Serial.println();
    }

  }
  else
  {
     //Connection failed. Increase failed counter
    failedCounter++;

    Serial.println("Connection to GroveStreams Failed ("+String(failedCounter, DEC)+")");   
    Serial.println();

    lastConnectionTime = millis(); 
  }
}

void startEthernet()
{
  //Start or restart the Ethernet connection.
  client.stop();

  Serial.println("Connecting Arduino to network...");
  Serial.println();  

  //Wait for the connection to finish stopping
  delay(1000);

  //Connect to the network and obtain an IP address using DHCP
  if (Ethernet.begin(mac) == 0)
  {
    Serial.println("DHCP Failed, reset Arduino to try again");
    Serial.println();
  }
  else
  {

    Serial.println("Arduino connected to network using DHCP");
    Serial.println();

    //Wait to ensure the connection finished
    delay(1000);

    //Set the mac and ip variables so that they can be used during sensor uploads later
    myMac =getMacReadable();
    Serial.println("MAC: " + myMac);

    myIPAddress = getIpReadable(Ethernet.localIP());
    Serial.println("IP address: " + myIPAddress);
  }

}

String getMacReadable() 
{
  //Convert the mac address to a readable string
  char macstr[20];
  snprintf(macstr, 100, "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
  return String(macstr);
}

String getIpReadable(IPAddress p)
{
  //Convert the ip address to a readable string
  String ip;
  for (int i =0; i < 3; i++)
  {
    ip += String(p[i], DEC);
    ip += ".";
  }
  ip +=String(p[3], DEC);
  return ip;
}

String getTemperatureF() 
{
  //Get the temperature analog reading and convert it to a string
  float voltage, degreesC, degreesF; 

  //voltage = (analogRead(temperaturePin) * 0.004882814);
  //degreesF = degreesC * (9.0/5.0) + 32.0;
  degreesF = float(random(70, 120));

  char temp[20] = {0}; //Initialize buffer to nulls
  dtostrf(degreesF, 12, 3, temp); //Convert float to string

  String stemp = temp;
  stemp.trim();  //Trim off head and tail spaces
  return stemp; 
}

String getTemperatureC() 
{
  //Get the temperature analog reading and convert it to a string
  float voltage, degreesC, degreesF; 

  //voltage = (analogRead(temperaturePin) * 0.004882814);
  //degreesC = (voltage - 0.5) * 100.0;
  degreesC = float(random(20, 40));

  char temp[20] = {0}; //Initialize buffer to nulls
  dtostrf(degreesC, 12, 3, temp); //Convert float to string

  String stemp = temp;
  stemp.trim();  //Trim off head and tail spaces
  return stemp; 
}

void showMem(){
  char Dbuf [100];
  
  strcpy_P(Dbuf,PSTR("Mem = "));
  Serial.print(Dbuf);
  Serial.println(freeMemory());
}



This worked really well, but like I said above, it will die over time because the Streams library will run the arduino out of memory.  I left the code in it that I used to check memory usage and also the random number generation I used to test it.

Notice that the call to send the data is different?  Of course you did.  The call is part of their batch update API that is much simpler to use than the JSON call I used in the python example.  This makes getting the data up there much easier for the arduino folks.  They're even working on improvements to make the API more easily understood.  The python folk seem to like infinitely complex data structures, so I'm leaving that example as it is.

They even have a full blown tutorial on hooking the arduino up to their service.  Unlike a lot of the tutorials out there, this one can actually be read.  You don't even have to click on page after page like the darn instructables we've all learned to love.

I really like this site.  Like I said, I haven't gotten past the very surface of its capabilities, and probably never will, but it was nice to see a site where the folks running it actually care if people can use it.

Floats and Strings Over a Serial Link (like an XBee)

$
0
0
I've mentioned many times how I use a regular old ascii string as the payload when communicating between XBees whenever possible.  There's several reasons for this, but the biggest is debugging the interaction of the devices.  It's a royal pain in the behind to try and interpret the data traveling over the air as binary using XCTU or a monitor XBee.  I convert integers, floats and such to text, send it, and reconstruct what I need at the other end. This way I can actually read what is traveling between XBees using a monitor XBee or the little sniffer I constructed exactly for this purpose.

However, this has generated questions about how to do such a thing from folks that are just starting out using an Arduino hooked to an XBee because it isn't obvious.  Yes, there are a bazillion examples out there, but not specifically related to passing data between XBees.  Additionally, there are operators in other languages that take care of this kind of thing totally behind the scenes and programmers don't have to worry about it.

Adding insult to injury, the Stream data type in the Arduino IDE has problems when running for a very long time: it will run your board out of memory and cause it to fail.  This isn't a complaint about the Arduino development environment, just a simple fact.  The little Arduino only has 2K of memory to play with and you simply run out if you try to do too much.  It's not like a laptop with 8Gb of memory to play around with, you have to control yourself and your code. So, doing things with as little memory usage as possible using simple tools is the way to make a device that can run for a week without failing.

So, here's a sketch that illustrates the some of the data types including the float.  The integer and long datatypes are relatively easy, but the float confuses some people. The float has generated probably a hundred questions and even more misunderstandings.  This code is waaay over commented and  will compile and run on an Arduino.  It basically takes a float, long and integer, converts them into ascii in a string with other stuff, and then gets it back out into variables to be used further.  The middle part where the string is sent between two devices can be found in other places on this blog. This will compile and run on an Arduino under IDE version 1.0 and up.

The Arduino Sketch
#include <stdio.h>

int intVariable;
long longVariable;
float floatVariable;

// this little function will return the first two digits after the decimal
// point of a float as an int to help with sprintf() (won't work for negative values)
// the .005 is there for rounding.
int frac(float num){
  return( ((num + .005) - (int)num) * 100);
}
// this function prints the characters of a c string one at a time
// without any formatting to confuse or hide things
void printbuffer(char *buffer){
  while(*buffer){
    Serial.write(*buffer++);
  }
}

void setup(){
  Serial.begin(9600);
}

char buff[100]; // we're going to use this to hold our string

void loop(){
  Serial.println("starting ...");
  
  intVariable = 11; // integers are just that, integers, no decimal point possible
  Serial.println(intVariable);
  
  longVariable = 12.45; // longs are really just big integers, they can't have a decimal
  Serial.println(longVariable); // This will show you what happens with a long
  
  floatVariable = 13.45; // floats are a different animal
  Serial.println(floatVariable);
  
  // now I'm putting these in a string.  For this I'm using sprintf() because
  // it makes this kind of thing so much easier.  I use the little frac() routine
  // up above.  This is simply cutting the float into two integers, one being the 
  // part to the left of the decimal and the other being two digits to the right
  // of the decimal.  The long uses a special format specification %ld as in 
  // 'long decimal', and int is just stuffed in there directly
  sprintf(buff, "This whole thing is a string: %d.%2d, %ld, %d\n", 
          int(floatVariable), frac(floatVariable), // the two halves of the float
          longVariable,  // and the long
          intVariable);  // and finally the integer
  
  // the %d means construct it as a decimal number (as in base 10), the '.' is 
  // just a period to be placed in the string, %2d means a 2 digit number with leading 
  // zeros.  Google the printf specification for several million examples.
  
  // Now let's print it one character at a time to illustrate what's in the string
  // without the formatting capabilities of Serial.print() confusing things
  printbuffer(buff);
  
  // Now, buff has a string of characters in it with the ascii 
  // representation of the variables in it.  You can send this
  // string through an XBee, over a wire using one of the serial 
  // techniques, or store it in a file.  It's also pretty good for 
  // serial LCD displays.
  
  // So, let's get the number out of the string.  To do this, you have to know
  // how the string is constructed.  If you choose a method to construct the string
  // that is easy to take apart (like a comma separated string) things are much
  // easier.  However, this string is actually pretty nasty.  So, we'll first find the 
  // colon.
  char *tmp = strchr(buff, ':');
  printbuffer(tmp);
  
  // So, now that we have a pointer into the string that begins at the colon, let's
  // skip the ': ' (colon space) and we'll be right at the number
  tmp += 2;
  printbuffer(tmp);
  
  // OK, now let's get the darn ascii number out of the string and back into a float.
  float recoveredFloat = atof(tmp);
  Serial.print("Float recovered was: ");
  Serial.println(recoveredFloat);
  
  // Now you have your float variable back as a float to do with as you please.
  // So, move over to the ',' that is before the long
  tmp = strchr(tmp, ',');
  printbuffer(tmp);
  // and skip the ', " to get to the number
  tmp += 2;
  printbuffer(tmp);
  // and get it out into a variable
  long recoveredLong = atol(tmp);
  Serial.print("Long recovered was: ");
  Serial.println(recoveredLong);
  // and the whole thing over again for the integer
  tmp = strchr(tmp, ',');
  printbuffer(tmp);
  tmp += 2;
  printbuffer(tmp);
  int recoveredInt = atoi(tmp);
  Serial.print("Int recovered was: ");
  Serial.println(recoveredInt);

  Serial.println("done.");
  while(1){}
}

Yes, I eat up substantial code memory using sprintf(), but it's worth it.  You can reuse the buffer over and over and you only pay the price for sprintf() once, not over and over again like you do with the String datatype.  Notice I didn't get into a long discussion of how floats are stored in memory and how operations on them work.  That's documented in about a million places and I don't want to add confusion by getting into that discussion.  If you need to know, go look it up.

There are three c library routines that are used here: atoi(), atol(), and atof().  These are documented on the web, so take a look at what they do.  One of the keys to understanding this stuff is to do a LOT of looking around for various solutions.  Anything you want to do has been done in some part before and someone probably wrote about it somewhere.

Keep in mind that there are as many ways to do this as there are programmers doing it.  So this is just one way.  I chose this to illustrate it as completely and simply as I could so folks would leave with a few less questions than they came with.

Now, when these questions come up, I can simply point to this page and let folks play with the code until they get the idea.  When people first start out using an Arduino to try and control things, it's a tough enough hurdle just getting the first code to work; I hope this helps some of them.

GroveStreams and the SteelSeries Gauges

$
0
0
If you've been following my rantings, you've seen the SteelSeries gauges.  I think they are the very best analog display widgets on the web.  At least right now.  Well, GroveStreams enabled them as a embedded object in their 'Dashboard'.  I couldn't wait to try them out to see if they were as nice as the ones I came up with for other things.  Take a look:


Nice aren't they?  I intentionally chose different appearances and styles for them to illustrate a tiny portion of what is possible.  There are other styles, horizonal, LCDs, well, you can look for yourself.  The really cool thing is you can experiment with the look and type right on the screen to get what you want.  Yes, these are (almost) real time.  I update every minute, so if you wait long enough, they'll change.  The little blue and red ticks on them represent the high and low for the last day.  Think about what they would look like with a weather station.

Then, later when you change your mind, go back to GroveStreams dashboard and change them.  Since the way they embed them is to use the dashboard object, all the places you've used them change as well.

Slick, really slick.

Belkin's Wemo Light Switch Part 1

$
0
0
This is an ongoing project (aren't they all) so I'm calling this part one.

I recently (couple three weeks ago) bought a Wemo Light Switch (link).  I was fascinated by the fact that it runs embedded Linux and could be hacked a bit to do some things I want.  They cost about $50 (US), so they aren't the cheapest devices out there, but have you looked at light switches that can be operated remotely lately?  Fifty bucks isn't that bad, some of the other ones cost a whole lot more and many come in at around the same price.  So, it's not as bad as it sounds.

However, if you scan the web, you'll find a lot of articles that are obviously trying to get hits because all they say is what Belkin already says on their site.  There are also articles that recount how they work; something that you can get from any of the suppliers out there.  But, hidden among the chaff are a few really interesting tidbits.  First, is that the device came out quite a while ago, but only to work with the iphone.  Then they came out with a beta version of an Android app for people to try.  The reviews of the device are mixed.  Some love it and it does exactly what they want, other hate the device and Belkin because the device doesn't work and Belkin support doesn't respond worth a darn.

So, with people being totally disgusted with the device, a marginal (if even that) Android app, and terrible support, I bought one.  Nothing like jumping head first into the shallow end of the pool.

The device came in and I rushed right in to install it to replace one of those X10 switches that are so darn unreliable.  I carefully read the almost non-existent instructions, downloaded the Android app, and sure enough, it didn't work.  The set up is like some other devices, the switch has a Wifi chip in it and it is initialized to set up an ad-hoc network so you can initialize it to your in-house wireless network.  So, you take your phone, connect to the switch's ad-hoc usecured network and configure the chip to talk to your own wireless.  My problem was that the Wemo Android app wouldn't talk to the switch; my phone would, but the app couldn't see it.

I searched the web, Belkin's support site ... nothing.  I did find a number of folk that had hacked into it using some pretty clever techniques, but nothing that I could use.  Finally, I picked up the phone and called Belkin support.  I got the usual (for technical support everywhere) scripted responses and a promise to get a call back from a higher level technician.  That NEVER happened.  So, I thought, "Who has an Android phone near me?" As you all know by now, I live in the sticks, so I had to do some searching, but I found a neighbor with a phone that would help.  I loaded the Wemo Android app on their phone and tried it again.  It worked!!

Yes, a different phone worked.  The neighbor's phone was running a slightly older version of Android and was of course, a different brand.  So, I set up the switch and tried my phone again since the switch was now talking to my Wifi network.  No luck, the app running on my phone couldn't see the switch.  Fine, there's more than one way to talk over ethernet.  Once again out to the web to see what I could find.

As luck would have it, I found the perfect solution for me, and probably a lot of other people out there.  This guy named Ian McCracken had already messed with the switches and developed a set of tools and API for the Wemo switch.  Bless Him.  He put together a nice python library that supports discovery and use of the switches and documented a ton of the intricacies involved.  Basically, this guy saved my butt on this little switch.  His web page where he describes it is here <link> and his library is here <link>.  Now just to be fair, another great guy: Isaac Kelly did some discovery work that made it pretty clear how to manipulate the Wemo switch.  Isaac's stuff is here <link> and he used a tool called Miranda created by /dev/ttys0 <link>.  But, Miranda had to be patched to support the Wemo switch because it uses a little known technique; of course, Isaac Kelly suggested the patch.  Miranda is here <link> and the patch is described here <link>.

So, with a patched Miranda, and a cool python library, I got the switch to work in my house.  Whew!  I still couldn't operate the switch with my phone, but I could turn it on and off with my Raspberry Pi.  Slick.

This all happened while I was waiting for the return call from Belkin.  That call never happened.  So, I decided to make a butthead out of myself and annoy the heck out of Belkin.  How can a normal person like me annoy a big company?  I went to their support site for the device <link> and nothing there could help me.  Next I went to getsatisfaction.com and found a Wemo forum there.  I posted a couple of times about the problem and how their response took forever and didn't accomplish anything <link>.  When that didn't work, I got a brainstorm.

I looked on Twitter and among all the crap was a Belkin marketing user.  Cool.  Right there in their marketing hype I posted about how I couldn't get any help for a product that they made that didn't work.  That got an immediate response.  I tweeted the forum entry I put up and they got on the stick and sent me mail telling me that they were referring the problem to someone to handle.  After a couple more of these, I actually got a phone call and a contact point at Belkin.

This nice lady tried with me over the phone to get the cell phone to talk to the Wemo, no luck.  She sent me some debugging instructions and I tried them to gather some information to help them diagnose the problem.  Suddenly, my phone started working.  Yep, you guessed it something changed and the Android app started working with the Wemo switch.  The sequence of events that got my phone to work with the switch is too long to go into in this post, but I will post it if anyone is interested.

So, I can turn on a light with a piece of code running on my Raspberry Pi, or over the internet using my cell phone.  I plan on extending this to allow my web interface to operate the switch as well, since that will only be a tiny bit of additional code.  It also opens up the possibility of adding more Wemo devices for things like appliances and other lights.  Belkin is also introducing a device that will control and monitor the power usage of an appliance; bet you know what I'll be ordering soon.

The lesson to be learned here is Legion. First, don't give up too easily.  Second, hit them where they will notice it: their online marketing. Third, expect a fix from the oddest circumstances.  Fourth, there are lots of other folk out there that have the same kind of problem, look for them.  Last, there are technical folk out there that can help with a lot of things; you just have to find them.

Belkin's Wemo Light Switch Part 2

$
0
0
It's been over a month (well, almost two months) since I wrote about the Wemo switch.  During that time, Belkin has come out with the Insight switch and a couple of iterations of the Android software.  It still doesn't work well on my phone, their online server has problems when you add switches.  Seems they forget you already had other switches.  I'm not willing to reinitialize everything in my house each time I do something to one of them.  But:

I have three of the switches now and they're all hooked into my Raspberry Pi house controller.  Over the last couple of months I've found a couple of problems with the ouimeaux python library that the author is looking at, and of course, with my code to run them.  Unlike the Android software, the wall switches work really well.  Occasionally the switch that is at the edge of my Wifi network will fail to report and twice now I've had one of the switches respond to a command with an error.  Really good performance for a remote switch and far, far better than the X10 stuff they replaced.

The really cool thing is that the switches report when they change state.  So, if I change it over the phone, the switch tells my Pi that it happened.  If I push the button, same thing.  This is great because you can check the state of a switch at any time to see if you left the lights on.  Using my Pi to control them works great as well.  I don't have to rely on Belkin's server or that 'if this then that' website to control things.  I can set up as many schedules as I want and don't have to worry about somebody else's server having problems.

For you folk out there that would like to mess with these switches, here's the code I'm using to control them:

The Python Script
#! /usr/bin/python
import os
import signal
import sys
import logging
from apscheduler.scheduler import Scheduler
from threading import Thread
import datetime
from datetime import timedelta
import time
import sysv_ipc


from ouimeaux.upnp import UPnPLoopbackException
from ouimeaux.environment import Environment
from ouimeaux.config import get_cache, in_home, WemoConfiguration
from ouimeaux.utils import matcher

#-------------------------------------------------

def lprint(text):
print time.strftime("%A, %B, %d at %H:%M:%S"),text
sys.stdout.flush()

def signal_handler(signal,frame):
lprint (" Exit Signal")
sys.exit(0)

def on_switch(switch):
lprint (" Switch found! " + switch.name)

def on_motion(motion):
lprint (" Motion found!" + switch.motion)

def outsideLightsOn():
lprint (" Outside lights on")
garageLightSwitch.on()
cactusLightSwitch.on()
fPorchLightSwitch.on()

def outsideLightsOff():
lprint (" Outside lights off")
garageLightSwitch.off()
cactusLightSwitch.off()
fPorchLightSwitch.off()

def fPorchToggle():
if (int(fPorchLightSwitch.get_state()) == 1):
fPorchLightSwitch.off()
else:
fPorchLightSwitch.on()

def garageToggle():
if (int(garageLightSwitch.get_state()) == 1):
garageLightSwitch.off()
else:
garageLightSwitch.on()

def cactusToggle():
if (int(cactusLightSwitch.get_state()) == 1):
cactusLightSwitch.off()
else:
cactusLightSwitch.on()

def updateDatabase(whichone, status):
pass

def cactusSwitchChange(value):
if (int(value) == 1):
lprint (" Cactus Spot On")
updateDatabase('cactusspot', 'On')
elif (int(value) == 0):
lprint (" Cactus Spot Off")
updateDatabase('cactusspot', 'Off')
else:
lprint (" Cactus Spot Switch Unknown value: " + value)

def garageSwitchChange(value):
if (int(value) == 1):
lprint (" Garage Outside Light On")
updateDatabase('outsidegarage', 'On')
elif (int(value) == 0):
lprint (" Garage Outside Light Off")
updateDatabase('outsidegarage', 'Off')
else:
lprint (" Garage Outside Light Switch Unknown value: " + value)

def fPorchSwitchChange(value):
if (int(value) == 1):
lprint (" Front Porch Light On")
updateDatabase('frontporch', 'On')
elif (int(value) == 0):
lprint (" Front Porch Light Off")
updateDatabase('frontporch', 'Off')
else:
lprint (" Front Porch Light Switch Unknown value: " + value)

def handleCommand(command):
#lprint("" + str(command))
# the command comes in from php as something like
# ('s:17:"AcidPump, pumpOff";', 2)
# so command[0] is 's:17:"AcidPump, pumpOff'
# then split it at the " and take the second item
try:
c = str(command[0].split('\"')[1]).split(',')
except IndexError:
c = str(command[0]).split('') #this is for something I sent from another process
#lprint(c)
if (c[0] == 'OutsideLightsOn'):
outsideLightsOn()
elif (c[0] == 'OutsideLightsOff'):
outsideLightsOff()
elif (c[0] == 'fPorchToggle'):
fPorchToggle()
elif(c[0] == 'garageToggle'):
garageToggle()
elif (c[0] == 'cactusToggle'):
cactusToggle()
else:
lprint(" Weird command = " + str(c))

def showTime():
lprint("timecheck");

if __name__ == '__main__':
#Catch Control-C input so it isn't ugly
signal.signal(signal.SIGINT, signal_handler)
logging.basicConfig(format='%(levelname)s:%(message)s',level=logging.WARNING)
#When looking at a log, this will tell me when it is restarted
lprint (" started")

env = Environment(on_switch, on_motion, with_cache=False)
env.start()
lprint (" Starting device discovery")
env.discover(seconds=5)
lprint (" done with discovery")
garageLightSwitch = env.get_switch('outsidegarage')
cactusLightSwitch = env.get_switch('cactusspot')
fPorchLightSwitch = env.get_switch('frontporch')
# lprint (" testing lights")
# garageLightSwitch.basicevent.SetBinaryState(BinaryState=1)
# cactusLightSwitch.basicevent.SetBinaryState(BinaryState=1)
# fPorchLightSwitch.basicevent.SetBinaryState(BinaryState=1)
# time.sleep(2)
# lprint (' garage ' + str(garageLightSwitch.basicevent.GetBinaryState()))
# lprint (' cactus ' + str(cactusLightSwitch.basicevent.GetBinaryState()))
# lprint (' front porch ' + str(fPorchLightSwitch.basicevent.GetBinaryState()))
# garageLightSwitch.basicevent.SetBinaryState(BinaryState=0)
# cactusLightSwitch.basicevent.SetBinaryState(BinaryState=0)
# fPorchLightSwitch.basicevent.SetBinaryState(BinaryState=0)
# lprint (' garage ' + str(garageLightSwitch.basicevent.GetBinaryState()))
# lprint (' cactus ' + str(cactusLightSwitch.basicevent.GetBinaryState()))
# lprint (' front porch ' + str(fPorchLightSwitch.basicevent.GetBinaryState()))
# If I want to see the various attributes of the device
# lprint ('' + garageLightSwitch.explain())
# lprint (" done testing")
# examine the status of the lights right now and update the database
lprint (' checking current state of switches')
garageSwitchChange(garageLightSwitch.get_state())
cactusSwitchChange(cactusLightSwitch.get_state())
fPorchSwitchChange(fPorchLightSwitch.get_state())

lprint (" Setting up the callbacks for the devices")
cactusLightSwitch.register_listener(cactusSwitchChange)
garageLightSwitch.register_listener(garageSwitchChange)
fPorchLightSwitch.register_listener(fPorchSwitchChange)

lprint (" Setting up scheduled items")
#------------------Stuff I schedule to happen -----
scheditem = Scheduler()
# every day at 1900, turn the outside lights on
#scheditem.add_cron_job(outsideLightsOn, hour=19, minute=01)
# every day at 2200, turn the outside lights off
#scheditem.add_cron_job(outsideLightsOff, hour=22, minute=0)
#just an interval timer for putting time in the log
scheditem.add_interval_job(showTime, minutes=30)
lprint (" starting scheduler")
scheditem.start()
# Create the message queue where commands can be read
# I just chose an identifier of 13 because the house monitor
# already took the number 12.
Cqueue = sysv_ipc.MessageQueue(13, sysv_ipc.IPC_CREAT,mode=0666)
firstTime = True

lprint (" Going into the processing loop")
while True:
try:
if (firstTime):
while(True):
try:
# commands could have piled up while this was
# not running. Clear them out.
junk = Cqueue.receive(block=False, type=0)
print "purging leftover commands", str(junk)
except sysv_ipc.BusyError:
break
firstTime=False
newCommand = Cqueue.receive(block=False, type=0)
# type=0 above means suck every message off the
# queue. If I used a number above that, I'd
# have to worry about the type in other ways.
# note, I'm reserving type 1 messages for
# test messages I may send from time to
# time. Type 2 are messages that are
# sent by the php code in the web interface.
# Type 3 are from the event handler. This is just like
# the house monitor code in that respect.
# I haven't decided on any others yet.
handleCommand(newCommand)
except sysv_ipc.BusyError:
pass # Only means there wasn't anything there

# Turn control over to the Wemo environment handler for a second so
# changes happening can be recorded and handled
env.wait(time=1)



I cut out the code to update my database since that is very specific to my implementation, and you'll notice down at the bottom, that control of the lights is handled by Sys V messages.  I don't control the lights with a console, everything comes from other processes sending messages.  For example, my web server software sends a message to this process to turn the lights on and off.  Another process sends messages to turn them on and off at certain times of the day.  That kind of thing.

Just cut that code out and insert whatever kind of control you want to use.  I also made a change to the ouimeaux python library to support asyncronous operation.  The library as it was constructed hung and waited for changes to happen.  I modified it a bit to return after a env.wait call; that's what the very last line in the code is doing; calling the wait for a second then getting back control so it can do other things.

There's a ton of things that can be done to make my control code better, but for now, it works.  I'll loop back to this project some time and add more versatility.  Maybe when the ouimeaux library is changed.

Arduino and the Iris Zigbee switch

$
0
0
A friend of mine is looking for a way to control a light remotely.  His problem, and mine also, is that these darn things are expensive and require a controller.  He specifically wanted something that used the Zigbee protocol because they are capable of being controlled by an XBee device.  He pointed me to the Iris line at Lowe's.  Being eaten up with curiosity, I went to Lowe's and looked at the devices.

They have an entire line of devices for lights, doors, garage, etc.  The problem I saw was that, once again, you're stuck with their controller that you can't change, their website that will probably go down at the least useful time (and has a number of times), and a monthly charge that they can raise any time they want to.  I even prowled through their terms of service (yuck) and it looks to me like they can use the data for anything they want to.  Also, they can change their terms of service at any time, leaving your data to their use.  I hate that crap.  Here we go, buy their stuff and then are subject to whatever they want to do over the years.  Why don't they just sell us the darn switches, publish a reasonable API that we can use, and let us live our lives outside their monthly charges and control?

What's needed here is for someone to figure out their switches, make them work with a little computer of some kind, and put how they did it on the web for the entire world to play with.  That'll show them.

Welcome to my work hooking an Arduino to an XBee and controlling an Iris light switch remotely.

First, I chose their Iris Smart Plug <link> because it looks cool, can be tested without installing it into the wall, and measures the power going through it.  Does this sound like my perfect device or what?  I can use one of these to measure power anywhere in the house and have it report to my Raspberry Pi where I can forward it to a cloud server (or three, or four) for examination over time.  This little device is right down my alley.  Now, all I have to do is make it work ... right ?




I could have bought their controller and decoded the interaction with a sniffer and proceeded to duplicate it on my Pi, but there were some problems with this idea:  I don't have a spare Pi right now, the interaction is encrypted, I don't have a sniffer; what the heck am I going to do with this $30 paperweight?  Off to the web I go.

I only found one, yes one, place where anyone had any sort of success hacking this device.  Over at Jeelabs a contributor, CapnBry, had managed to make it work using code on a Windows PC <link>, but his description of how it worked read like classic Greek; totally out of my league.  Try as I might, there just wasn't anyplace else that turned up with something more comprehensible to me.  I had to just bite the bullet and start trying to talk to the switch.  After all, I've worked with XBees for years; how hard can it be?

At this point, you're probably thinking that this is just another post about how I tried to make it work and gave up because I just couldn't get enough information, didn't have the time, or the hardware didn't live up to expectations.  Well, not so; I have the working switch being remotely controlled by an Arduino setting right over there.  And yes, I'm going to tell you how I did it, and provide the code so you can repeat the experiment.  Hopefully, you'll expand on my efforts and let me know about it so we can all benefit and move along.

Like I said earlier, I wanted to put this on a Raspberry Pi, but I didn't have a spare one to experiment with, so I got an Arduino and XBee out of my parts bin, slapped them together, and programmed the XBee as a controller.  This is where I ran into my first obstacle.  How to set up the XBee?  First, you have to use a Series 2 XBee or better; none of this will work on a Series 1 XBee (now do you understand why I chose series 2)?  The notable items in the setup of the XBee are:

ZigBee Stack Profile 2
Encryption Enable 1
Encryption Options 1
API Enable 2
API Output Mode 3
XBee Modem XB24-ZB, ZIGBEE Coordinator API, Version 21A7 (or better)

And, Encryption Key set to something that you can remember.  You cannot read this register after you set it, so put in something you can't forget if you need it later.  Once set, I haven't needed it since, but you never know.

All the parameters can be set using XCTU and it is relatively easy to set up for this once you know what you're doing.  The other parameters can be matched to whatever you're used to using.  Don't think this came easily!  It took me almost a full day of messing around to get it working at all, so if you have trouble, double check everything.

Next, I wanted to use Andrew Rapp's XBee library for the Arduino, but he didn't put in support for the special messages needed to communicate with a ZigBee device.  Specifically there are two messages that are used (almost) exclusively for ZigBee communications: Explicit Addressing ZigBee Command Frame 0x11 and ZigBee Explicit Rx Indicator 0x91; these are not supported by Andrew's library.  However, the library is too nice to allow something like that to stop me, so I extended the library to support these two messages and added support to the ZBTxStatusResponse to be able to get the frame ID back (he missed that little thing).  These changes allowed me to use the library and all its features to speed up the hack.

OK, armed with a library specially modified to work with ZigBee devices and an XBee that should be able to talk to them, I put some code together to monitor traffic.  I immediately got traffic from the switch.  There was a series of messages that I couldn't understand and a lot of bytes to figure out what they meant.  Back to the link above where the guy stated that he had made it work.  The problem was with language.  He said things like, "You: (Endpoint=0x02, Profile=0xc216, Cluster=0x00f0) FrameControl=0x19 ClusterCmd=0xfa data=0x00 0x01" What is an Endpoint?  A Profile?  These things were totally foreign and strange.  So I hunted down the ZigBee specification <link> and it was HUGE and filled with jargon specific to the protocol that made reading it an exercise in learning to speak ancient Sanskrit.  I also found an Endpoint document that talked about the interaction of devices during a process called 'Joining'<link>.  After reading a significant portion of these I started to understand.  None of the web sites I visited actually described what most of this stuff was, but it boils down to this:

There's a device that supports ZigBee as an end point.  It's things like light switches, door locks, devices that actually do something; these are called ZigBee Device Objects.  There's things that control these devices, they're called servers.  Each device has profiles, these profiles are code that are specific to the device.  Each profile has clusters; these clusters are where the code to do something is hooked.  So, you'll send a message to a device, profile something, cluster something, with some data about what you want to do.  Add to this the fact that each device has a 64 bit address, a 16 bit address, and needs special formatting to the message and the task becomes a bit daunting ... a whole lot daunting.


After prowling around documentation for hours I started to get a glimmer of what was going on and I finally found a key to these things that was way down in the XBee user's guide.  Round about page 122 the Digi folk talk about how to send and receive messages to a ZigBee device.  That was the key that got me going.  Now that I understood a little more than half of what the guy CapnBry was talking about, I started deconstructing the messages from the switch and looking at what was going on.


The switch sends a message announcing that it exists, then it sends another message that tells some stuff about what it can do.  These messages come after the hardware itself gets set up.  There are messages (we don't have to worry about) that take place just to get the XBee coordinator to recognize it.  Then the little device sends another message specifically from the hardware.  See, the ZigBee protocol is general purpose and has support for devices that haven't been invented yet, but the manufacturers ignore that and roll their own stuff under the 'Manufacturer Specific' provisions of the specification.  That means that even if you support the Zigbee protocol, it won't work with the various devices because they hide everything under the special provisions ... jerks.  So, you have to fiddle with things to get them to work.  The manufacturer AlertMe is notable for this tactic, and AlertMe manufactures the devices that Lowe's sells.  


So, these three messages come out of the switch and it's your responsibility to respond correctly to them and get the device to recognize your code as a valid controller; that's what meant by 'Joining'.  What happens is that the switch has a hardware 64 bit address and randomly chooses a 16 bit address to send the messages with.  If it doesn't get a proper response, it steps to another channel and tries the same thing again.  It does this for quite a while before it give up and just shuts down.  So you watch the messages, it stops a while and the messages start again.  Eventually it just give up all together.  Each time you see the set of three messages, it will have a different 16 bit address, so you have to save this address to respond to so that it will listen.  If you're too slow, it won't pay attention because it has already changed the address it pays attention to.


The sequence is documented in the code below, but basically, you wait for the first two messages to come in then respond to both of them.  Then, you interact with the device and it will join with your homemade controller.  From that point on, it will report the power usage every three seconds with a summary every minute.  There's other stuff that is sent by the switch, but I wasn't interested enough in it to bother decoding it; consider those items an exercise for the student.


Once you get it working, you'll find out how nice this switch is.  It monitors the power and reports it every three seconds or so.  It latches the state of the switch such that if the power fails, it comes back in the same state it was in when the power failed.  The little light on it doesn't follow properly, but that can be controlled with software.  It reports a state change back.  That means that if I walk over and push the button, it will send a message that the light has been changed.  You can ask the switch the state of the light and it will answer back to you.  So you can check to see if the outside lights were left on.  This switch is actually pretty nice.


But enough of the bragging.  Here's the code, but remember, this is an example of how to do it.  It isn't an example of coding style, proper formatting, or even the right way to do it.  It's the actual code I used to figure out how the switch works. It will compile using the Arduino IDE 1.0.5 and needs the special modifications to the XBee library I added to support the Zigbee specific messages.  Just let me know if you need the changes and I'll put them somewhere you can grab them.  It also needs the latest SoftwareSerial library because I use software pins to connect to the XBee and the console to monitor and send commands.


The Arduino Sketch


/*
This is an examination of Zigbee device communication using an XBee

Specifically using the Lowe's Iris switch. This device plugs into an outlet
and has a plug on the front for your appliance. It controls the on/off of the appliance
and measures the power usage as well. It's a lot like a remote control Kill-a-Watt.

*/

#include <XBee.h>
#include <SoftwareSerial.h>

XBee xbee = XBee();
XBeeResponse response = XBeeResponse();
// create response and command objects we expect to handle
ZBExpRxResponse rx = ZBExpRxResponse();
XBeeAddress64 switchLongAddress;
uint16_t switchShortAddress;
uint16_t myFrameId=1; // for debugging, it's nice to know which messages is being handled

// Define NewSoftSerial TX/RX pins
// Connect Arduino pin 2 to Tx and 3 to Rx of the XBee
// I know this sounds backwards, but remember that output
// from the Arduino is input to the Xbee
#define ssRX 2
#define ssTX 3
SoftwareSerial nss(ssRX, ssTX);


void setup() {
// start serial
Serial.begin(9600);
// and the software serial port
nss.begin(9600);
// now that they are started, hook the XBee into
// Software Serial
xbee.setSerial(nss);
// I think this is the only line actually left over
// from Andrew's original example
Serial.println("started");
}

void loop() {
// doing the read without a timer makes it non-blocking, so
// you can do other stuff in loop() as well. Things like
// looking at the console for something to turn the switch on
// or off (see waaay down below)
xbee.readPacket();
// so the read above will set the available up to
// work when you check it.
if (xbee.getResponse().isAvailable()) {
// got something
Serial.println();
Serial.print("Frame Type is ");
// Andrew called the XBee frame type ApiId, it's the first byte
// of the frame specific data in the packet.
Serial.println(xbee.getResponse().getApiId(), HEX);
//
// All ZigBee device interaction is handled by the two XBee message type
// ZB_EXPLICIT_RX_RESPONSE (ZigBee Explicit Rx Indicator Type 91)
// ZB_EXPLICIT_TX_REQUEST (Explicit Addressing ZigBee Command Frame Type 11)
// This test code only uses these and the Transmit Status message
//
if (xbee.getResponse().getApiId() == ZB_EXPLICIT_RX_RESPONSE) {
// now that you know it's a Zigbee receive packet
// fill in the values
xbee.getResponse().getZBExpRxResponse(rx);

// get the 64 bit address out of the incoming packet so you know
// which device it came from
Serial.print("Got a Zigbee explicit packet from: ");
XBeeAddress64 senderLongAddress = rx.getRemoteAddress64();
print32Bits(senderLongAddress.getMsb());
Serial.print("");
print32Bits(senderLongAddress.getLsb());

// this is how to get the sender's
// 16 bit address and show it
uint16_t senderShortAddress = rx.getRemoteAddress16();
Serial.print(" (");
print16Bits(senderShortAddress);
Serial.println(")");

// for right now, since I'm only working with one switch
// save the addresses globally for the entire test module
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();

//Serial.print("checksum is 0x");
//Serial.println(rx.getChecksum(), HEX);

// this is the frame length
//Serial.print("frame data length is ");
int frameDataLength = rx.getFrameDataLength();
//Serial.println(frameDataLength, DEC);

uint8_t* frameData = rx.getFrameData();
// display everything after first 10 bytes
// this is the Zigbee data after the XBee supplied addresses
Serial.println("Zigbee Specific Data from Device: ");
for (int i = 10; i < frameDataLength; i++) {
print8Bits(frameData[i]);
Serial.print("");
}
Serial.println();
// get the source endpoint
Serial.print("Source Endpoint: ");
print8Bits(rx.getSrcEndpoint());
Serial.println();
// byte 1 is the destination endpoint
Serial.print("Destination Endpoint: ");
print8Bits(rx.getDestEndpoint());
Serial.println();
// bytes 2 and 3 are the cluster id
// a cluster id of 0x13 is the device announce message
Serial.print("Cluster ID: ");
uint16_t clusterId = (rx.getClusterId());
print16Bits(clusterId);
Serial.println();
// bytes 4 and 5 are the profile id
Serial.print("Profile ID: ");
print16Bits(rx.getProfileId());
Serial.println();
// byte 6 is the receive options
Serial.print("Receive Options: ");
print8Bits(rx.getRxOptions());
Serial.println();
Serial.print("Length of RF Data: ");
Serial.print(rx.getRFDataLength());
Serial.println();
Serial.print("RF Data Received: ");
for(int i=0; i < rx.getRFDataLength(); i++){
print8Bits(rx.getRFData()[i]);
Serial.print('');
}
Serial.println();
//
// I have the message and it's from a ZigBee device
// so I have to deal with things like cluster ID, Profile ID
// and the other strangely named fields that these devices use
// for information and control
//
if (clusterId == 0x13){
Serial.println("*** Device Announce Message");
// In the announce message:
// the next bytes are a 16 bit address and a 64 bit address (10) bytes
// that are sent 'little endian' which means backwards such
// that the most significant byte is last.
// then the capabilities byte of the actual device, but
// we don't need some of them because the XBee does most of the work
// for us.
//
// so save the long and short addresses
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();
// the data carried by the Device Announce Zigbee messaage is 18 bytes over
// 2 for src & dest endpoints, 4 for cluster and profile ID,
// receive options 1, sequence number 1, short address 2,
// long address 8 ... after that is the data specific to
// this Zigbee message
Serial.print("Sequence Number: ");
print8Bits(rx.getRFData()[0]);
Serial.println();
Serial.print("Device Capabilities: ");
print8Bits(rx.getRFData()[11]);
Serial.println();
}

if (clusterId == 0x8005){ // Active endpoint response
Serial.println("*** Active Endpoint Response");
// You should get a transmit responnse packet back from the
// XBee first, this will tell you the other end received
// something.
// Then, an Active Endpoint Response from the end device
// which will be Source Endpoint 0, Dest Endpoint 0,
// Cluster ID 8005, Profile 0
// it will have a payload, but the format returned by the
// Iris switch doesn't match the specifications.
//
// Also, I tried responding to this message directly after
// its receipt, but that didn't work. When I moved the
// response to follow the receipt of the Match Descriptor
// Request, it started working. So look below for where I
// send the response
}
if (clusterId == 0x0006){ // Match descriptor request
Serial.println("*** Match Descriptor Request");
// This is where I send the Active Endpoint Request
// which is endpoint 0x00, profile (0), cluster 0x0005
uint8_t payload1[] = {0,0};
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x0005, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
payload1, //payload
sizeof(payload1), //payload length
myFrameId++); // frame ID
xbee.send(tx);
Serial.println();
Serial.print("sent active endpoint request frame ID: ");
Serial.println(myFrameId-1);
//
// So, send the next message, Match Descriptor Response,
// cluster ID 0x8006, profile 0x0000, src and dest endpoints
// 0x0000; there's also a payload byte
//
// {00.02} gave clicks
uint8_t payload2[] = {0x00,0x00,0x00,0x00,0x01,02};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x8006, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
payload2, //payload
sizeof(payload2), //payload length
myFrameId++); // frame ID
xbee.send(tx);
Serial.println();
Serial.print("sent Match Descriptor Response frame ID: ");
Serial.println(myFrameId-1);

//
// Odd hardware message #1. The next two messages are related
// to control of the hardware. The Iris device won't join with
// the coordinator without both of these messages
//
uint8_t payload3[] = {0x11,0x01,0x01};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00f6, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload3, //payload
sizeof(payload3), //payload length
myFrameId++); // frame ID
xbee.send(tx);
Serial.println();
Serial.print("sent funny hardware message #1 frame ID: ");
Serial.println(myFrameId-1);
//
// Odd hardware message #2
//
uint8_t payload4[] = {0x19,0x01,0xfa,0x00,0x01};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00f0, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload4, //payload
sizeof(payload4), //payload length
myFrameId++); // frame ID
xbee.send(tx);
Serial.println();
Serial.print("sent funny hardware message #2 frame ID: ");
Serial.println(myFrameId-1);

}
else if (clusterId == 0xf6){
// This is something specific to the Endpoint devices
// and I haven't been able to find documentation on it
// anywhere
Serial.println("*** Cluster ID 0xf6");
}
if (clusterId == 0x00ef){
//
// This is a power report, there are two kinds, instant and summary
//
Serial.print("*** Power Data, ");
// The first byte is what Digi calls 'Frame Control'
// The second is 'Transaction Sequence Number'
// The third is 'Command ID'
if (rx.getRFData()[2] == 0x81){
// this is the place where instant power is sent
// but it's sent 'little endian' meaning backwards
int power = rx.getRFData()[3] + (rx.getRFData()[4] << 8);
Serial.print("Instantaneous Power is: ");
Serial.println(power);
}
else if (rx.getRFData()[2] == 0x82){
unsigned long minuteStat = (uint32_t)rx.getRFData()[3] +
((uint32_t)rx.getRFData()[4] << 8) +
((uint32_t)rx.getRFData()[5] << 16) +
((uint32_t)rx.getRFData()[6] << 24);
unsigned long uptime = (uint32_t)rx.getRFData()[7] +
((uint32_t)rx.getRFData()[8] << 8) +
((uint32_t)rx.getRFData()[9] << 16) +
((uint32_t)rx.getRFData()[10] << 24);
int resetInd = rx.getRFData()[11];
Serial.print("Minute Stat: ");
Serial.print(minuteStat);
Serial.print(" watt seconds, Uptime: ");
Serial.print(uptime);
Serial.print(" seconds, Reset Ind: ");
Serial.println(resetInd);
}
}
if (clusterId == 0x00ee){
//
// This is where the current status of the switch is reported
//
// If the 'cluster command' is 80, then it's a report, there
// are other cluster commands, but they are controls to change
// the switch. I'm only checking the low order bit of the first
// byte; I don't know what the other bits are yet.
if (rx.getRFData()[2] == 0x80){
if (rx.getRFData()[3] & 0x01)
Serial.println("Light switched on");
else
Serial.println("Light switched off");
}
}
}
else if (xbee.getResponse().getApiId() == ZB_TX_STATUS_RESPONSE) {
ZBTxStatusResponse txStatus;
xbee.getResponse().getZBTxStatusResponse(txStatus);
Serial.print("Status Response: ");
Serial.println(txStatus.getDeliveryStatus(), HEX);
Serial.print("To Frame ID: ");
Serial.println(txStatus.getFrameId());
}
else {
Serial.print("Got frame type: ");
Serial.println(xbee.getResponse().getApiId(), HEX);
}
}
else if (xbee.getResponse().isError()) {
// some kind of error happened, I put the stars in so
// it could easily be found
Serial.print("************************************* error code:");
Serial.println(xbee.getResponse().getErrorCode(),DEC);
}
else {
// I hate else statements that don't have some kind
// ending. This is where you handle other things
}
if (Serial.available() > 0) {
char incomingByte;

incomingByte = Serial.read();
Serial.println(atoi(&incomingByte), DEC);
if (atoi(&incomingByte) == 0){
// turn the light off
lightSet(0);
}
else if (atoi(&incomingByte) == 1){
// turn the light on
lightSet(1);
}
}
}

void lightSet(int val){
uint8_t payload1[] = {0x11,0x00,0x01,03};
uint8_t payload2[] = {0x11,0x00,0x02,0x00,0x01};

if (val==0){
Serial.println("Light Off");
}
else {
Serial.println("Light On");
payload2[3] = 0x01;
}
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00ee, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload1, //payload
sizeof(payload1), //payload length
myFrameId++); // frame ID

xbee.send(tx);
Serial.println();
Serial.print("sent switch off 1 frame ID: ");
Serial.println(myFrameId-1);

tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00ee, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload2, //payload
sizeof(payload2), //payload length
myFrameId++); // frame ID

xbee.send(tx);
Serial.println();
Serial.print("sent switch off 2 frame ID: ");
Serial.println(myFrameId-1);

}

// these routines are just to print the data with
// leading zeros and allow formatting such that it
// will be easy to read.
void print32Bits(uint32_t dw){
print16Bits(dw >> 16);
print16Bits(dw & 0xFFFF);
}

void print16Bits(uint16_t w){
print8Bits(w >> 8);
print8Bits(w & 0x00FF);
}

void print8Bits(byte c){
uint8_t nibble = (c >> 4);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);

nibble = (uint8_t) (c & 0x0F);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);
}



Once you get it running you can turn the switch on by typing a 1 in the input line of the Arduino IDE terminal and clicking send.  A zero will turn the switch off.  That's all I really supported in this version, future work will obviously expand the capabilities.  Also, if you kill the sketch, wait until at least one message has been sent by the switch.  The code above needs the 16 and 64 bit address of the switch to work and I didn't put any provisions in to save it; it has to come from the switch.  Every message from the switch carries the addresses, so just wait for one to come in before trying to send something.  Since this is the very first version of this effort, the switch can sometimes fail to 'join'.  This isn't a problem, just let the two devices interact for about 20 seconds or so, unplug the switch and plug it back in.  It'll take off and work.

Edit: About an hour after I posted this I got a brainstorm and figured out how to get the switch to 'join' reliably.  Now, you can reset the switch by unplugging it, press the button to discharge any caps, plug it in, and then press the switch 8 times within about 8 seconds.  The switch will start all over in its interaction, and then start sending power readings.  When (notice I didn't say 'if') I get another one, I'll have to modify this code to support two devices, but one works fine.  The code box above has been updated to hold the latest.

Here's the output from the Arduino from first start up after joining.  I turn the switch on and off during the session.  Notice that the power usage is 83 (two little bulbs in a lamp) and that the switch is constantly sending its status over the network.  There's an extra piece of debugging in this; I print all the bytes sent to the XBee, so the lines that begin with the 7E are lines that are actually being sent to the switch.

started

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EF
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 00 81 53 00
Power Data, Instantaneous Power is: 83

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00F0
Profile ID: C216
Length of RF Data: 16
RF Data Received: 09 00 FB 1C 26 25 DA 03 4A 32 00 00 CB EA 01 00
0
Light Off
7E 0 18 11 1 0 D 6F 0 2 37 B2 5A A6 C4 0 2 0 EE C2 16 0 0 11 0 1 3 E5
sent switch off 1 frame ID: 1
7E 0 19 11 2 0 D 6F 0 2 37 B2 5A A6 C4 0 2 0 EE C2 16 0 0 11 0 2 0 1 E5
sent switch off 2 frame ID: 2

Frame Type is 8B
Status Response: 0
To Frame ID: 2

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EE
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 70 80 06 E0
Light switched off

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EF
Profile ID: C216
Length of RF Data: 12
RF Data Received: 09 00 82 A4 0D 00 00 90 F6 00 00 00
Power Data, Minute Stat: 3492 watt seconds, Uptime: 63120 seconds, Reset Ind: 0

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EF
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 00 81 50 00
Power Data, Instantaneous Power is: 80

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EF
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 00 81 00 00
Power Data, Instantaneous Power is: 0
1
Light On
7E 0 18 11 3 0 D 6F 0 2 37 B2 5A A6 C4 0 2 0 EE C2 16 0 0 11 0 1 3 E3
sent switch off 1 frame ID: 3
7E 0 19 11 4 0 D 6F 0 2 37 B2 5A A6 C4 0 2 0 EE C2 16 0 0 11 0 2 1 1 E2
sent switch off 2 frame ID: 4

Frame Type is 8B
Status Response: 0
To Frame ID: 4

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EE
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 70 80 07 00
Light switched on

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00EF
Profile ID: C216
Length of RF Data: 5
RF Data Received: 09 00 81 53 00
Power Data, Instantaneous Power is: 83

Frame Type is 91
Got a Zigbee explicit packet from: 000D6F00 0237B25A (A6C4)
Source Endpoint: 02
Destination Endpoint: 02
Cluster ID: 00F0
Profile ID: C216
Length of RF Data: 16
RF Data Received: 09 00 FB 1C 26 9D DA 03 4A 32 00 00 C6 F6 01 00


Have fun.



Arduino and the Iris Zigbee switch, part 2

$
0
0
I've had a few days to play with the Iris switch <link> and I like it.  However it doesn't play well with others.  In that I mean I tried to bring it up on the network with my other devices and managed to kill my network of XBees.  Yes, I was down with all my devices not talking to each other and I had to visit each device and fix it.  This is totally avoidable, I was an idiot.  Do not follow in my footsteps in this.

First there are some things to know.  As I pointed out in my last post <link> about this switch, it takes special settings on the XBee for it to work with an Iris switch:

ZigBee Stack Profile 2
Encryption Enable 1
Encryption Options 1
API Enable 2
API Output Mode 3
Some Encryption Key
XBee Modem XB24-ZB, ZIGBEE Coordinator API, Version 21A7 (or better)

This is radically different from what I have been using in the past.  First, the Zigbee Stack Profile is 2, which means use the ZigBee Pro stack; I normally use a 0 in that spot.  When I changed it to profile 2, it stopped passing all the messages from my existing XBees through to the serial port.  Naturally, I thought that I could simply change the other devices to use the same profile parameter and everything should be fine.  Not so.

It seems the API Output Mode has a similar effect.  When you change API Output mode to something other than 0, you lose all the messages from other XBees except the ones that are explicitly allowed.  The message: ZigBee Explicit Rx Indicator (frame id=91) comes through fine, but you lose the ZigBee Receive Packet (frame id=90) which is what you normally get.  That means my remote XBees were happily sending messages and the receiver didn't pass them on to the serial port.  Like I said, I was an idiot because the setting clearly indicates either 'traditional' or 'explicit'.  I should have gotten a clue from this before I messed with the controller.

So, I couldn't see anything that was being sent by my other devices.  Naturally, I found this out AFTER I had reprogrammed my XBee controller.  My whole network was down.  The various XBee devices had saved their connection with the controller as it was set up before and I had to visit each one of them and make them drop the connection and establish a new one.  What a royal pain in the behind.

While I was doing that I decided there had to be a way to keep from having this happen in the future.  It turns out there is.  By adjusting some of the parameters, the remote XBees will sense that the controller is gone and automatically hunt for a new controller.  This is a mixed blessing because if your controller dies, your remote devices will stop talking to each other.  I've had exactly zero problems with the XBee controller, so I decided to set the network up this way so that future experiments like this won't mean taking things apart to get to the XBee and forcing it to initialize the network.  The parameters are:

Network Watchdog Timeout 1
Channel Verification 1 (enabled)

The watchdog timeout causes the XBee to reestablish the network connection if it hasn't heard from the coordinator in three minutes.  Since my coordinator is sending the time repeatedly, all devices will hear from it often enough that the network shouldn't ever have a problem.  The Channel Verification being enabled means that any time the power fails the XBee will check to be sure there is an XBee controller out there when the power comes back on.  If it can't find the controller, it'll go looking for one.

These changes mean that if my controller dies, the network dies.  Fine, I can live with that.  It also means that there may be a very slight pause whenever the power comes back on from a failure.  When I tested that by unplugging a device, it took less than a second for it to find the controller and get back online.  So, now if I mess up the controller with some experiment, I won't have to go visit each darn XBee and make it talk to the new controller set up.  They'll do it by themselves after a three minute timeout, or I can unplug the power and make them do it right now.  Too bad I didn't discover this a couple of years ago.

Now, these changes may not be the right thing for you to do.  However, if you have an XBee in a hard to get to location, or setting way out there in the field hooked to a solar charger, it might be a good idea.  So, you understand your needs better than I can; make an informed decision.

All this information means something else: The Iris switch and my network won't work together.  Simple, I'll just set up a controller for the Iris Zigbee devices I may eventually have separate from the network I am using now and control them separately.  I should be able to interface an Arduino that has the code for Iris to anything I want to and go merrily on my way.  That may be the next project.

So, there was something good that came out of messing up my network of devices.  I learned a lot about modifying the network in general.  I also got to visit each of my XBees and clean out the spider webs and dead flies.  There was live scorpion in one of them, but since it's cool right now and the arthropod couldn't move very fast, I won.  I also decided to retire the separate Arduino for the Acid Pump and move the code and connection over to my Pool Controller.  Since I was already in the code for the Pool Controller, I hooked in the Septic Tank Float so I can get an alarm in the house if the tank has problems.  These were chores that I've been putting aside for months now.

How many people do you know that have a septic tank that can send them email?

Arduino and the Iris Zigbee switch, part 3

$
0
0
I've been working with the Iris Smart Switch for three weeks or so and I believe I have enough hacked out of it to make a useful addition to my home automation.  As I mentioned before, it controls an appliance as well as measuring the power usage of the device plugged into it.  Below is the code I ended up with in testing the device.  It's pretty verbose in the output as well as having a ton of comments to help the next person that wants to dig into this switch.

There are eight selections that can be chosen by typing a number into the arduino IDE input line and pressing send:

0 - turn the switch off
1 - turn the switch on
2 - special command routine
3 - get version data
4 - get current switch state (on, off)
5 - reset switch to normal mode
6 - range test
7 - local lock mode
8 - silent Mode

The special command routine is where you can try various commands to the switch.  Simply add code to your requirements and have at it. I used this selection to find various commands.  Range test returns the RSSI value for the switch and can be useful to tell if you have an RF path back to the controller.  Local lock mode disables the button on the switch, it can still be remote controlled, in this mode it doesn't return the periodic power data.  Silent Mode allows local control with the button as well as remote control, but the periodic data is not returned.

There's probably commands and operational characteristics to be found, but I think this is the critical set to put the switch into use.  My next project will be to actually use the darn thing to do something useful.

The Arduino Sketch
/**
This is an examination of Zigbee device communication using an XBee
and an Iris Smart Switch from Lowe's
*/

#include <XBee.h>
#include <SoftwareSerial.h>
#include <Time.h>
#include <TimeAlarms.h>

XBee xbee = XBee();
XBeeResponse response = XBeeResponse();
// create reusable response objects for responses we expect to handle
ZBExpRxResponse rx = ZBExpRxResponse();

// Define NewSoftSerial TX/RX pins
// Connect Arduino pin 2 to Tx and 3 to Rx of the XBee
// I know this sounds backwards, but remember that output
// from the Arduino is input to the Xbee
#define ssRX 2
#define ssTX 3
SoftwareSerial nss(ssRX, ssTX);

XBeeAddress64 switchLongAddress;
uint16_t switchShortAddress;
uint16_t payload[50];
uint16_t myFrameId=1;

void setup() {
// start serial
Serial.begin(9600);
// and the software serial port
nss.begin(9600);
// now that they are started, hook the XBee into
// Software Serial
xbee.setSerial(nss);
// I think this is the only line actually left over
// from Andrew's original example
setTime(0,0,0,1,1,14); // just so alarms work well, I don't really need the time.
Serial.println("started");
}

void loop() {
// doing the read without a timer makes it non-blocking, so
// you can do other stuff in loop() as well. Things like
// looking at the console for something to turn the switch on
// or off (see waaay down below)
xbee.readPacket();
// so the read above will set the available up to
// work when you check it.
if (xbee.getResponse().isAvailable()) {
// got something
// Serial.println();
// Serial.print("Frame Type is ");
// Andrew called the XBee frame type ApiId, it's the first byte
// of the frame specific data in the packet.
// Serial.println(xbee.getResponse().getApiId(), HEX);
//
// All ZigBee device interaction is handled by the two XBee message type
// ZB_EXPLICIT_RX_RESPONSE (ZigBee Explicit Rx Indicator Type 91)
// ZB_EXPLICIT_TX_REQUEST (Explicit Addressing ZigBee Command Frame Type 11)
// This test code only uses these and the Transmit Status message
//
if (xbee.getResponse().getApiId() == ZB_EXPLICIT_RX_RESPONSE) {
// now that you know it's a Zigbee receive packet
// fill in the values
xbee.getResponse().getZBExpRxResponse(rx);

// get the 64 bit address out of the incoming packet so you know
// which device it came from
// Serial.print("Got a Zigbee explicit packet from: ");
XBeeAddress64 senderLongAddress = rx.getRemoteAddress64();
// print32Bits(senderLongAddress.getMsb());
// Serial.print("");
// print32Bits(senderLongAddress.getLsb());

// this is how to get the sender's
// 16 bit address and show it
uint16_t senderShortAddress = rx.getRemoteAddress16();
// Serial.print(" (");
// print16Bits(senderShortAddress);
// Serial.println(")");

// for right now, since I'm only working with one switch
// save the addresses globally for the entire test module
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();

//Serial.print("checksum is 0x");
//Serial.println(rx.getChecksum(), HEX);

// this is the frame length
//Serial.print("frame data length is ");
int frameDataLength = rx.getFrameDataLength();
//Serial.println(frameDataLength, DEC);

uint8_t* frameData = rx.getFrameData();
// display everything after first 10 bytes
// this is the Zigbee data after the XBee supplied addresses
// Serial.println("Zigbee Specific Data from Device: ");
// for (int i = 10; i < frameDataLength; i++) {
// print8Bits(frameData[i]);
// Serial.print("");
// }
// Serial.println();
// get the source endpoint
// Serial.print("Source Endpoint: ");
// print8Bits(rx.getSrcEndpoint());
// Serial.println();
// byte 1 is the destination endpoint
// Serial.print("Destination Endpoint: ");
// print8Bits(rx.getDestEndpoint());
// Serial.println();
// bytes 2 and 3 are the cluster id
// a cluster id of 0x13 is the device announce message
// Serial.print("Cluster ID: ");
uint16_t clusterId = (rx.getClusterId());
print16Bits(clusterId);
Serial.print(": ");
// bytes 4 and 5 are the profile id
// Serial.print("Profile ID: ");
// print16Bits(rx.getProfileId());
// Serial.println();
// // byte 6 is the receive options
// Serial.print("Receive Options: ");
// print8Bits(rx.getRxOptions());
// Serial.println();
// Serial.print("Length of RF Data: ");
// Serial.print(rx.getRFDataLength());
// Serial.println();
// Serial.print("RF Data Received: ");
// for(int i=0; i < rx.getRFDataLength(); i++){
// print8Bits(rx.getRFData()[i]);
// Serial.print('');
// }
// Serial.println();
//
// I have the message and it's from a ZigBee device
// so I have to deal with things like cluster ID, Profile ID
// and the other strangely named fields that these devices use
// for information and control
//
if (clusterId == 0x13){
Serial.println("*** Device Announce Message");
// In the announce message:
// the next bytes are a 16 bit address and a 64 bit address (10) bytes
// that are sent 'little endian' which means backwards such
// that the most significant byte is last.
// then the capabilities byte of the actual device, but
// we don't need some of them because the XBee does most of the work
// for us.
//
// so save the long and short addresses
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();
// the data carried by the Device Announce Zigbee messaage is 18 bytes over
// 2 for src & dest endpoints, 4 for cluster and profile ID,
// receive options 1, sequence number 1, short address 2,
// long address 8 ... after that is the data specific to
// this Zigbee message
// Serial.print("Sequence Number: ");
// print8Bits(rx.getRFData()[0]);
// Serial.println();
// Serial.print("Device Capabilities: ");
// print8Bits(rx.getRFData()[11]);
// Serial.println();
}
if (clusterId == 0x8005){ // Active endpoint response
Serial.println("*** Active Endpoint Response");
// You should get a transmit responnse packet back from the
// XBee first, this will tell you the other end received
// something.
// Then, an Active Endpoint Response from the end device
// which will be Source Endpoint 0, Dest Endpoint 0,
// Cluster ID 8005, Profile 0
// it will have a payload, but the format returned by the
// Iris switch doesn't match the specifications.
//
// Also, I tried responding to this message directly after
// its receipt, but that didn't work. When I moved the
// response to follow the receipt of the Match Descriptor
// Request, it started working. So look below for where I
// send the response
}
if (clusterId == 0x0006){ // Match descriptor request
Serial.println("*** Match Descriptor Request");
// This is where I send the Active Endpoint Request
// which is endpoint 0x00, profile (0), cluster 0x0005
uint8_t payload1[] = {0,0};
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x0005, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
payload1, //payload
sizeof(payload1), //payload length
myFrameId++); // frame ID
xbee.send(tx);
// Serial.println();
//sendSwitch(0, 0, 0x0005, 0x0000, 0, 0, 0);

Serial.print("sent active endpoint request ");
//
// So, send the next message, Match Descriptor Response,
// cluster ID 0x8006, profile 0x0000, src and dest endpoints
// 0x0000; there's also a payload byte
//
// {00.02} gave clicks
uint8_t payload2[] = {0x00,0x00,0x00,0x00,0x01,02};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x8006, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
payload2, //payload
sizeof(payload2), //payload length
myFrameId++); // frame ID
xbee.send(tx);
// Serial.println();
Serial.print("sent Match Descriptor Response frame ID: ");
Serial.println(myFrameId-1);

//
// Odd hardware message #1. The next two messages are related
// to control of the hardware. The Iris device won't stay joined with
// the coordinator without both of these messages
//
uint8_t payload3[] = {0x11,0x01,0x01};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00f6, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload3, //payload
sizeof(payload3), //payload length
myFrameId++); // frame ID
xbee.send(tx);
// Serial.println();
Serial.print("sent funny hardware message #1 frame ID: ");
Serial.println(myFrameId-1);
//
// Odd hardware message #2
//
uint8_t payload4[] = {0x19,0x01,0xfa,0x00,0x01};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
2, //dest endpoint
0x00f0, //cluster ID
0xc216, //profile ID
0, //broadcast radius
0x00, //option
payload4, //payload
sizeof(payload4), //payload length
myFrameId++); // frame ID
xbee.send(tx);
// Serial.println();
Serial.print("sent funny hardware message #2 frame ID: ");
Serial.println(myFrameId-1);

}
else if (clusterId == 0xf6){
// This is The Range Test command response.
Serial.print("Cluster Cmd: ");
Serial.print(rx.getRFData()[2],HEX);
Serial.print("");
Serial.print("*** Cluster ID 0xf6 ");
if (rx.getRFData()[2] == 0xfd){
Serial.print("RSSI value: ");
print8Bits(rx.getRFData()[3]);
Serial.print("");
print8Bits(rx.getRFData()[4]);
Serial.print("");
Serial.print((int8_t)rx.getRFData()[3]);
Serial.println();
}
else if (rx.getRFData()[2] == 0xfe){
Serial.println("Version information");
// bytes 0 - 2 are the packet overhead
// This can be decoded to give the data from the switch,
// but frankly, I didn't know what I would do with it
// once decoded, so I just skip it.
}
// This is to catch anything that may pop up in testing
else{
Serial.print(rx.getRFData()[2],HEX);
Serial.print("");
for(int i=0; i < rx.getRFDataLength(); i++){
print8Bits(rx.getRFData()[i]);
Serial.print('');
}
Serial.println();
}
}
if (clusterId == 0x00f0){
// Serial.println("Most likely a temperature reading; useless");
// for(int i=0; i < rx.getRFDataLength(); i++){
// print8Bits(rx.getRFData()[i]);
// Serial.print('');
// }
// Serial.println();
Serial.print("Cluster Cmd: ");
Serial.print(rx.getRFData()[2],HEX);
Serial.print("");
uint16_t count = (uint8_t)rx.getRFData()[5] +
((uint8_t)rx.getRFData()[6] << 8);
Serial.print("Count: ");
Serial.print(count);
Serial.print("");
Serial.print(rx.getRFData()[12]);
Serial.print("");
Serial.print(rx.getRFData()[13]);
Serial.print("");
uint16_t temp = (uint8_t)rx.getRFData()[12] +
((uint8_t)rx.getRFData()[13] << 8);
Serial.println(temp);
// temp = (temp / 1000) * 9 / 5 + 32;
// Serial.println(temp);

}
if (clusterId == 0x00ef){
//
// This is a power report, there are two kinds, instant and summary
//
Serial.print("Cluster Cmd: ");
Serial.print(rx.getRFData()[2],HEX);
Serial.print("");
Serial.print("*** Power Data, ");
// The first byte is what Digi calls 'Frame Control'
// The second is 'Transaction Sequence Number'
// The third is 'Command ID'
if (rx.getRFData()[2] == 0x81){
// this is the place where instant power is sent
// but it's sent 'little endian' meaning backwards
int power = rx.getRFData()[3] + (rx.getRFData()[4] << 8);
Serial.print("Instantaneous Power is: ");
Serial.println(power);
}
else if (rx.getRFData()[2] == 0x82){
unsigned long minuteStat = (uint32_t)rx.getRFData()[3] +
((uint32_t)rx.getRFData()[4] << 8) +
((uint32_t)rx.getRFData()[5] << 16) +
((uint32_t)rx.getRFData()[6] << 24);
unsigned long uptime = (uint32_t)rx.getRFData()[7] +
((uint32_t)rx.getRFData()[8] << 8) +
((uint32_t)rx.getRFData()[9] << 16) +
((uint32_t)rx.getRFData()[10] << 24);
int resetInd = rx.getRFData()[11];
Serial.print("Minute Stat: ");
Serial.print(minuteStat);
Serial.print(" watt seconds, Uptime: ");
Serial.print(uptime);
Serial.print(" seconds, Reset Ind: ");
Serial.println(resetInd);
}
}
if (clusterId == 0x00ee){
//
// This is where the current status of the switch is reported
//
// If the 'cluster command' is 80, then it's a report, there
// are other cluster commands, but they are controls to change
// the switch. I'm only checking the low order bit of the first
// byte; I don't know what the other bits are yet.
if (rx.getRFData()[2] == 0x80){
Serial.print("Cluster Cmd: ");
Serial.print(rx.getRFData()[2],HEX);
Serial.print("");
if (rx.getRFData()[3] & 0x01)
Serial.println("Light switched on");
else
Serial.println("Light switched off");
}
}
}
else if (xbee.getResponse().getApiId() == ZB_TX_STATUS_RESPONSE) {
ZBTxStatusResponse txStatus;
xbee.getResponse().getZBTxStatusResponse(txStatus);
Serial.print("Status Response: ");
Serial.println(txStatus.getDeliveryStatus(), HEX);
Serial.print("To Frame ID: ");
Serial.println(txStatus.getFrameId());
}
else {
Serial.print("Got frame type: ");
Serial.println(xbee.getResponse().getApiId(), HEX);
}
}
else if (xbee.getResponse().isError()) {
// some kind of error happened, I put the stars in so
// it could easily be found
Serial.print("************************************* error code:");
Serial.println(xbee.getResponse().getErrorCode(),DEC);
}
else {
// I hate else statements that don't have some kind
// ending. This is where you handle other things
}
if (Serial.available() > 0) {
char incomingByte;

incomingByte = Serial.read();
Serial.print("Selection: ");
Serial.println(atoi(&incomingByte), DEC);
// This message set will turn the light off
if (atoi(&incomingByte) == 0){
uint8_t payload1[] = {0x01}; //
uint8_t payloadOff[] = {0x00,0x01};
sendSwitch(0x00, 0x02, 0x00ee, 0xc216, 0x01, payload1, sizeof(payload1));
sendSwitch(0x00, 0x02, 0x00ee, 0xc216, 0x02, payloadOff, sizeof(payloadOff));
}
// This pair of messages turns the light on
else if (atoi(&incomingByte) == 1){
uint8_t payload1[] = {0x01}; //
uint8_t payloadOn[] = {0x01,0x01};
sendSwitch(0x00, 0x02, 0x00ee, 0xc216, 0x01, payload1, sizeof(payload1));
sendSwitch(0x00, 0x02, 0x00ee, 0xc216, 0x02, payloadOn, sizeof(payloadOn));
}
// this goes down to the test routine for further hacking
else if (atoi(&incomingByte) == 2){
testCommand();
}
// This will get the Version Data, it's a combination of data and text
else if (atoi(&incomingByte) == 3){
uint8_t data[] = {0x00, 0x01};
sendSwitch(0x00, 0x02, 0x00f6, 0xc216, 0xfc, data, sizeof(data));
}
// This command causes a message return holding the state of the switch
else if (atoi(&incomingByte) == 4){
uint8_t data[] = {0x01};
sendSwitch(0x00, 0x02, 0x00ee, 0xc216, 0x01, data, sizeof(data));
}
// restore normal mode after one of the mode changess that follow
else if (atoi(&incomingByte) == 5){
uint8_t databytes[] = {0x00, 0x01};
sendSwitch(0, 0x02, 0x00f0, 0xc216, 0xfa, databytes, sizeof(databytes));
}
// range test - periodic double blink, no control, sends RSSI, no remote control
// remote control works
else if (atoi(&incomingByte) == 6){
uint8_t databytes[] = {0x01, 0x01};
sendSwitch(0, 0x02, 0x00f0, 0xc216, 0xfa, databytes, sizeof(databytes));
}
// locked mode - switch can't be controlled locally, no periodic data
else if (atoi(&incomingByte) == 7){
uint8_t databytes[] = {0x02, 0x01};
sendSwitch(0, 0x02, 0x00f0, 0xc216, 0xfa, databytes, sizeof(databytes));
}
// Silent mode, no periodic data, but switch is controllable locally
else if (atoi(&incomingByte) == 8){
uint8_t databytes[] = {0x03, 0x01};
sendSwitch(0, 0x02, 0x00f0, 0xc216, 0xfa, databytes, sizeof(databytes));
}
}
Alarm.delay(0); // Just for the alarm routines

}

uint8_t testValue = 0x00;

void testCommand(){
Serial.println("testing command");
return;
Serial.print("Trying value: ");
Serial.println(testValue,HEX);
uint8_t databytes[] = {};
sendSwitch(0, 0xf0, 0x0b7d, 0xc216, testValue++, databytes, sizeof(databytes));
if (testValue != 0xff)
Alarm.timerOnce(1,testCommand); // try it again in a second
}

/*
Because it got so cumbersome trying the various clusters for various commands,
I created this send routine to make things a little easier and less prone to
typing mistakes. It also made the code to implement the various commands I discovered
easier to read.
*/
void sendSwitch(uint8_t sEndpoint, uint8_t dEndpoint, uint16_t clusterId,
uint16_t profileId, uint8_t clusterCmd, uint8_t *databytes, int datalen){

uint8_t payload [10];
ZBExpCommand tx;
// Serial.println("Sending command");
//
// The payload in a ZigBee Command starts with a frame control field
// then a sequence number, cluster command, then databytes specific to
// the cluster command, so we have to build it up in stages
//
// first the frame control and sequence number
payload[0] = 0x11;
payload[1] = 0;
payload[2] = clusterCmd;
for (int i=0; i < datalen; i++){
payload[i + 3] = databytes[i];
}
int payloadLen = 3 + datalen;
// OK, now we have the ZigBee cluster specific piece constructed and ready to send

tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
sEndpoint, //src endpoint
dEndpoint, //dest endpoint
clusterId, //cluster ID
profileId, //profile ID
0, //broadcast radius
0x00, //option
payload, //payload
payloadLen, //payload length
myFrameId++); // frame ID

xbee.send(tx);
Serial.print("sent command: ");
Serial.print(payload[2], HEX);
Serial.print(" frame ID: ");
Serial.println(myFrameId-1);
}

/*-------------------------------------------------------*/
// these routines are just to print the data with
// leading zeros and allow formatting such that it
// will be easy to read.
void print32Bits(uint32_t dw){
print16Bits(dw >> 16);
print16Bits(dw & 0xFFFF);
}

void print16Bits(uint16_t w){
print8Bits(w >> 8);
print8Bits(w & 0x00FF);
}

void print8Bits(byte c){
uint8_t nibble = (c >> 4);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);

nibble = (uint8_t) (c & 0x0F);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);
}
Here's a chunk of the sample output:

00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 37753 213 159 40917
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 37873 214 159 40918
00EF: Cluster Cmd: 82 *** Power Data, Minute Stat: 0 watt seconds, Uptime: 58620 seconds, Reset Ind: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 37993 214 167 42966
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 38113 216 149 38360
00EF: Cluster Cmd: 82 *** Power Data, Minute Stat: 0 watt seconds, Uptime: 58680 seconds, Reset Ind: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 38233 216 146 37592
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 38353 217 147 37849
00EF: Cluster Cmd: 82 *** Power Data, Minute Stat: 0 watt seconds, Uptime: 58740 seconds, Reset Ind: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 38473 216 150 38616
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00F0: Cluster Cmd: FB Count: 38593 217 150 38617
00EF: Cluster Cmd: 82 *** Power Data, Minute Stat: 0 watt seconds, Uptime: 58800 seconds, Reset Ind: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0
00EF: Cluster Cmd: 81 *** Power Data, Instantaneous Power is: 0

I didn't have anything plugged into it at the time, but you an see what the general output is.

Have fun

Raspberry Pi and the Lowe's Iris Smart Switch

$
0
0
So, I got the Lowe's Iris Smart Switch working pretty well for the Arduino <link>.  The problem is that it isn't where I want the software to run.  I have a Raspberry Pi controlling the house <link> and this software should go there.  As I mentioned, I don't have a spare Pi right now, so I worked up the software for the Arduino with the full intention of moving it to the Pi as soon as it worked and I had time to mess with it.

Well, I decided the cool thing to do was to port the test software directly to the Pi in python and run it there.  I approached this with a little trepidation; taking things from an Arduino to another platform and language can drive you nuts.  First, I already have an XBee attached to the Pi; it uses the (only) serial port on the little device, so I got a Sparkfun XBee explorer <link> and plugged it into the USB port.  Fully expecting to have to jump through a bunch of hoops to get it to work, I did a simple 'cat /dev/ttyUSB0' command and actually got output to the screen on the very first try!

Sure it was garbage and didn't mean much, but I got output that corresponded to pushing the button on the switch.  Step one was done.  Next I put together a little code using the python XBee library to catch a packet and see what happened.  Right off the bat, I got this printed on the console of the Pi:

{'profile': '\xc2\x16', 'source_addr': '+\xd1', 'dest_endpoint': '\x02', 'rf_data': '\t\x00\x81T\x00', 'source_endpoint': '\x02', 'options': '\x01', 'source_addr_long': '\x00\ro\x00\x027\xb2Z', 'cluster': '\x00\xef', 'id': 'rx_explicit'}

Holy Cow, the library already had support for the ZigBee specific messages!  Notice that the fields have names already, and the ones that it took me so long to figure out are already there.  This means I can jump right in and start taking the messages from the switch apart.  It worked like a charm; there were some understanding problems in that the XBee library returns the data as character strings inside a dictionary of the various fields in a message, but these can be overcome once you catch on to what is happening.  Once I could decode the messages and print the power values and state of the switch, I implemented the commands from the Arduino code and they worked quite well.  So, here are the same capabilities that I presented for the Arduino implemented on the Raspberry Pi:

The Python Script
#! /usr/bin/python
# This is the an implementation of controlling the Lowe's Iris Smart
# Switch. It will join with a switch and allow you to control the switch
#
# Only ONE switch though. This implementation is a direct port of the
# work I did for an Arduino and illustrates what needs to be done for the
# basic operation of the switch. If you want more than one switch, you can
# adapt this code, or use the ideas in it to make your own control software.
#
# Have fun

from xbee import ZigBee
from apscheduler.scheduler import Scheduler
import logging
import datetime
import time
import serial
import sys
import shlex


#-------------------------------------------------
# the database where I'm storing stuff
DATABASE='/home/pi/database/desert-home'

# on the Raspberry Pi the serial port is ttyAMA0
XBEEPORT = '/dev/ttyUSB0'
XBEEBAUD_RATE = 9600

# The XBee addresses I'm dealing with
BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff'
UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address

switchLongAddr = '12'
switchShortAddr = '12'

#-------------------------------------------------
logging.basicConfig()

#------------ XBee Stuff -------------------------
# Open serial port for use by the XBee
ser = serial.Serial(XBEEPORT, XBEEBAUD_RATE)

# this is a call back function. When a message
# comes in this function will get the data
def messageReceived(data):
# print 'gotta packet'
# print data
# This is a test program, so use global variables and
# save the addresses so they can be used later
global switchLongAddr
global switchShortAddr
switchLongAddr = data['source_addr_long']
switchShortAddr = data['source_addr']
clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1])
print 'Cluster ID:', hex(clusterId),
if (clusterId == 0x13):
# This is the device announce message.
# due to timing problems with the switch itself, I don't
# respond to this message, I save the response for later after the
# Match Descriptor request comes in. You'll see it down below.
# if you want to see the data that came in with this message, just
# uncomment the 'print data' comment up above
print 'Device Announce Message'
elif (clusterId == 0x8005):
# this is the Active Endpoint Response This message tells you
# what the device can do, but it isn't constructed correctly to match
# what the switch can do according to the spec. This is another
# message that gets it's response after I receive the Match Descriptor
print 'Active Endpoint Response'
elif (clusterId == 0x0006):
# Match Descriptor Request; this is the point where I finally
# respond to the switch. Several messages are sent to cause the
# switch to join with the controller at a network level and to cause
# it to regard this controller as valid.
#
# First the Active Endpoint Request
payload1 = '\x00\x00'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x00\x05',
profile = '\x00\x00',
data = payload1
)
print 'sent Active Endpoint'
# Now the Match Descriptor Response
payload2 = '\x00\x00\x00\x00\x01\x02'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x80\x06',
profile = '\x00\x00',
data = payload2
)
print 'Sent Match Descriptor'
# Now there are two messages directed at the hardware
# code (rather than the network code. The switch has to
# receive both of these to stay joined.
payload3 = '\x11\x01\x01'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x02',
cluster = '\x00\xf6',
profile = '\xc2\x16',
data = payload2
)
payload4 = '\x19\x01\xfa\x00\x01'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x02',
cluster = '\x00\xf0',
profile = '\xc2\x16',
data = payload4
)
print 'Sent hardware join messages'

elif (clusterId == 0xef):
clusterCmd = ord(data['rf_data'][2])
if (clusterCmd == 0x81):
print 'Instantaneous Power',
print ord(data['rf_data'][3]) + (ord(data['rf_data'][4]) * 256)
elif (clusterCmd == 0x82):
print "Minute Stats:",
print 'Usage, ',
usage = (ord(data['rf_data'][3]) +
(ord(data['rf_data'][4]) * 256) +
(ord(data['rf_data'][5]) * 256 * 256) +
(ord(data['rf_data'][6]) * 256 * 256 * 256) )
print usage, 'Watt Seconds ',
print 'Up Time,',
upTime = (ord(data['rf_data'][7]) +
(ord(data['rf_data'][8]) * 256) +
(ord(data['rf_data'][9]) * 256 * 256) +
(ord(data['rf_data'][10]) * 256 * 256 * 256) )
print upTime, 'Seconds'
elif (clusterId == 0xf0):
clusterCmd = ord(data['rf_data'][2])
print "Cluster Cmd:", hex(clusterCmd),
if (clusterCmd == 0xfb):
print "Temperature ??"
else:
print "Unimplemented"
elif (clusterId == 0xf6):
clusterCmd = ord(data['rf_data'][2])
if (clusterCmd == 0xfd):
print "RSSI value:", ord(data['rf_data'][3])
elif (clusterCmd == 0xfe):
print "Version Information"
else:
print data['rf_data']
elif (clusterId == 0xee):
clusterCmd = ord(data['rf_data'][2])
if (clusterCmd == 0x80):
print "Switch is:",
if (ord(data['rf_data'][3]) & 0x01):
print "ON"
else:
print "OFF"
else:
print "Unimplemented Cluster ID", hex(clusterId)
print

def sendSwitch(whereLong, whereShort, srcEndpoint, destEndpoint,
clusterId, profileId, clusterCmd, databytes):

payload = '\x11\x00' + clusterCmd + databytes
# print 'payload',
# for c in payload:
# print hex(ord(c)),
# print
# print 'long address:',
# for c in whereLong:
# print hex(ord(c)),
# print

zb.send('tx_explicit',
dest_addr_long = whereLong,
dest_addr = whereShort,
src_endpoint = srcEndpoint,
dest_endpoint = destEndpoint,
cluster = clusterId,
profile = profileId,
data = payload
)

#------------------If you want to schedule something to happen -----
#scheditem = Scheduler()
#scheditem.start()

#scheditem.add_interval_job(something, seconds=sometime)

#-----------------------------------------------------------------

# Create XBee library API object, which spawns a new thread
zb = ZigBee(ser, callback=messageReceived)

print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
print "Enter a number from 0 through 8 to send a command"
while True:
try:
time.sleep(0.001)
str1 = raw_input("")
# Turn Switch Off
if(str1[0] == '0'):
print 'Turn switch off'
databytes1 = '\x01'
databytesOff = '\x00\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x01', databytes1)
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x02', databytesOff)
# Turn Switch On
if(str1[0] == '1'):
print 'Turn switch on'
databytes1 = '\x01'
databytesOn = '\x01\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x01', databytes1)
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x02', databytesOn)
# this goes down to the test routine for further hacking
elif (str1[0] == '2'):
#testCommand()
print 'Not Implemented'
# This will get the Version Data, it's a combination of data and text
elif (str1[0] == '3'):
print 'Version Data'
databytes = '\x00\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xf6', '\xc2\x16', '\xfc', databytes)
# This command causes a message return holding the state of the switch
elif (str1[0] == '4'):
print 'Switch Status'
databytes = '\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x01', databytes)
# restore normal mode after one of the mode changess that follow
elif (str1[0] == '5'):
print 'Restore Normal Mode'
databytes = '\x00\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xf0', '\xc2\x16', '\xfa', databytes)
# range test - periodic double blink, no control, sends RSSI, no remote control
# remote control works
elif (str1[0] == '6'):
print 'Range Test'
databytes = '\x01\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xf0', '\xc2\x16', '\xfa', databytes)
# locked mode - switch can't be controlled locally, no periodic data
elif (str1[0] == '7'):
print 'Locked Mode'
databytes = '\x02\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xf0', '\xc2\x16', '\xfa', databytes)
# Silent mode, no periodic data, but switch is controllable locally
elif (str1[0] == '8'):
print 'Silent Mode'
databytes = '\x03\x01'
sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xf0', '\xc2\x16', '\xfa', databytes)
# else:
# print 'Unknown Command'
except IndexError:
print "empty line"
except KeyboardInterrupt:
print "Keyboard interrupt"
break
except NameError as e:
print "NameError:",
print e.message.split("'")[1]
except:
print "Unexpected error:", sys.exc_info()[0]
break

print "After the while loop"
# halt() must be called before closing the serial
# port in order to ensure proper thread shutdown
zb.halt()
ser.close()


Just like the Arduino code, this will allow a switch to join and then it will constantly update based on messages from the switch.  Here's some sample output from a run of this code:

Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 60285 Watt Seconds Up Time, 1200 Seconds
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 65266 Watt Seconds Up Time, 1260 Seconds
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 84
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 70250 Watt Seconds Up Time, 1320 Seconds
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 75230 Watt Seconds Up Time, 1380 Seconds
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 80213 Watt Seconds Up Time, 1440 Seconds
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Minute Stats: Usage, 85194 Watt Seconds Up Time, 1500 Seconds
Cluster ID: 0xef Instantaneous Power 84
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xf0 Cluster Cmd: 0xfb Temperature ??
Cluster ID: 0xef Instantaneous Power 83
Cluster ID: 0xef Instantaneous Power 83


I had a little light hooked up to it that has two 40W incandescent bulbs in it so there was something to show.

Now I have the basics of reading the switch, and all I have to do now is hook it up with the rest of the software in the House Controller.  Then I can place these things wherever I want either, control of the power, or a measurement of how much power is being used.  Very nice little switch; I couldn't have built one for the price off the shelf at Lowe's.  And most importantly to me, I have control of it, not some cloud server or control device that I have to rely on a corporation's whim to change to fit my needs.

Have fun.

Using the Iris Smart Switch to Measure Appliance Power

$
0
0
Since I finished the example of how to operate the Iris Smart Switch with a Raspberry Pi, I couldn't wait to hook it to something and do something real with it.  I decided to measure the power usage of my refrigerator.  First, about the refrigerator:  It's a refrigerator only, it doesn't have a freezer section; I have a separate freezer in the kitchen also.  There's a reason for this besides showing off.  Out here in the sticks where a trip to the grocery store is a major undertaking, we don't go very often.  That means plenty of food storage; I have a pantry, an upright freezer and separate refrigerator in the house and a chest freezer in the garage.  Yes, I can go about two weeks before I even have to go for bread and milk.  This makes things like a shopping list that is up to date a valuable skill.  I keep it up to date and every time I'm forced into town, it goes with me.  I don't want to waste a trip.

Anyway, I put together some code and read the switch, recorded the readings in my database and updated Xively with the data.  Then I suck the accumulated data from Xively and graph it to see what the appliance is doing.  I didn't implement the ability to turn the switch on and off since who wants to shut their refrigerator off??  Here's a days worth of data on the refrigerator:



Notice how it's only on for a short amount of time and that, other than the spike when the compressor starts, it doesn't use much power?  It's on roughly 15 minutes, and off for about 30 minutes in repeating cycles.  Here is an expansion of a portion of it:


The large spike on the first 'hump' is where the compressor kicks on.  It takes a lot of power to get the compressor going, but it drops off rapidly.  The granularity of the readings is only a minute, and if I was graphing every few seconds, you would see spikes on all the humps.  However, this is an accurate representation of what is happening overall.  Notice that the 550W spike is short and that the current draw decreases over time.  The average usage is probably around 110W or so for the period where the compressor is on.  That's the amount of energy used by some incandescent bulbs !  Let's see what a door open does to the chart:



Notice that the power usage jumps (the hump on the right) to 300W when the door is open?  It's pretty sobering that the power used by the lights when the door is open is actually more than the power used to keep the darn thing cold.  Now see why they say to close the door to save energy?  They just didn't tell us that the power loss was due to the bulbs, not the fact that we're warming up the fridge.  I'll probably get arguments from the energy nerds out there, but for my appliance, this is true.  Or, at least it is right now; I plan on getting LED bulbs to take care of that silly problem in the next couple of weeks.

Of course, there's the compressor kicking on right after I closed the door because I left the door open for a couple of minutes and it had to recover the temperature.  That's why the hump for the lights is on the far right, I actually went over and opened the door to get this reading.

All in all though, the fridge is not a huge power consumer.  It's less than running a 100W bulb all the time.  Actually, it's probably close to a 40W bulb since it's off twice as long as it's on in any given period.

Let's review.  I hacked into the Lowe's smart switch using an arduino (it's what I had to play with)<link>, ported the code to python so I could run it on a Raspberry Pi <link>, and then hooked it up to an appliance and learned that it was actually doing a good job.  Actually better than I expected.  Was it worth it?  Of course it was.  Now folks out there can take my example and measure their own appliances to compare them to their own expectations and needs.

Soon, I'll get another Smart Switch and put it in the circuit for the freezer and see how much power it uses.  I expect it to be higher than the fridge because it has the same set of lights, a bigger compressor and the circuitry for auto defrost.  Auto defrost is a real power monger since it actually warms up some areas of the appliance and drains the water out to be evaporated away.  I also want to see how often it cycles since it could happen that the compressor on the freezer and the compressor on the fridge could overlap their cycles.  That would cause a spike in power usage for the overlap period, which in turn, could cause my demand usage to rise.  That's the kind of thing I want to keep track of.  Demand billing is a painful thing because a simple mistake in power usage could double your monthly power bill.  I have a long discussion about my experiences here <link>.

Now, go out and get your own system of monitors running.  Take control of your house.

Raspberry Pi, USB, and XBee

$
0
0
I have two XBees hooked to my Pi now, one to control the network I've been building for a long time hooked to the serial port, and the other acting as a controller for my Iris Smart Switch hooked into a USB port.  I like the Smart Switch, so I'll be using more of them and that meant I had to figure out a better way to hook two XBees to the Pi and make it work.

I really didn't want to get into the mess of trying to get another standard serial port working because I eventually want LEDs for indicators of things that are important and maybe even a display that I can put words on.  Heck, I could decide to put voice on it at some point.  So, since the Pi comes with two USB ports, I decided to move both of the XBees to USB.  I started out with the USB explorer that I have to program the XBees:


I've had this board a long time and it just works.  Never had a problem with it, so I just plugged it into one of the USB ports on the Pi and changed my code to point to it.  It worked after a tiny bit of messing around.  Now I could control my own stuff and read the Smart Switch as well.  That proved that it could be done so I got two of these:


I chose this board so I could use a short USB cable to connect it, avoiding the problem of the bigger board above not fitting.  With these plugged into the Pi, I could still control the house and have all my GPIO pins available for whatever else I decide to do someday.  But, as usual, there was a problem.  See, USB is designed to be plugged and unplugged so you can change devices on the Pi, that unfortunately means that the address of the XBee will change each time you unplug it from the USB port.  So, one time I would have the smart switch on /dev/ttyUSB0 and the next time it would show up on /dev/ttyUSB1.  Makes coding for it more complex, and I really don't want complex right now.

It turns out that there is another way to connect to it.  Under the directory /dev is another directory 'serial'; looking in /dev/serial, I found two other directories by-id and by-path.  The by-id directory holds the device ids of the USB to serial chip and that doesn't change as you plug and unplug it.  So, I changed my code to define the serial port to this:

XBEEPORT = '/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QLG3-if00-port0'

Yes, it's long and obscure, but that's what variables and comments are for.  It's also hard to type in, but that's what copy and paste are good for.  All in all, not a bad way to go as long as I don't reverse which little board I have the XBee plugged into.  Remember this little trick when your USB device seems to disappear.

So, now I have all the GPIO pins available; wonder what I'm going to plug into them.

I Want To Complain about 'Experts'

$
0
0
My refrigerator started giving me trouble.  No, it wasn't the Iris Smart Switch I installed in the power, it was that the darn thing started to get warm.  I opened the door and the little thermometer I have inside said it was 55 F inside.  It usually runs under 45, so something was up.  I checked all the obvious things, the compressor was going, the fan on the condenser coil was running; I couldn't see anything obvious.

Once again, a little bit about the refrigerator:  It's a refrigerator (only) that I've had for some years and it has given me trouble before; I even had to have the evaporator coil replaced.  I got this thing because I wanted it to be the last refrigerator I'd ever buy.  To meet that I got the very top of the line GE Monogram model ZIRS36 in stainless steel (ZIRS36NMRH).  It's a really, really old design that doesn't even have a frost free option.  That's because it has no freezer inside and just doesn't form frost.

Basically, the power comes in, goes to the temperature control, then to a temperature sensor on the evaporator coil (to prevent freezing up) and then to the compressor.  There's three fans: one on the top to cool the condenser and condenser coil, one inside to circulate air from the evaporator coil to the rest of the fridge, and one on the bottom to evaporate any water that drains out of the evaporator.  Then the lights are all that's left.  Not much to go bad, and not much to troubleshoot when it has problems.

However, I couldn't find anything wrong with any of those things, so I went to the web and found this discussion on one of those 'Ask an Expert' sites <link>; basically, the expert said the correct thing first, then corrected himself to the wrong thing.  This fridge is a middle level cooling system.  It doesn't cool below freezing anywhere inside except the actual surface of the evaporator coil.  It maintains a temperature of about 38 degrees and doesn't need a defrost timer or defrost heater at all.  The customer in the discussion tells the expert this, but that doesn't keep the expert from getting confused and messing it up.  I chuckled a little bit and then found this discussion <link>;  once again, the expert changes his mind midstream and messes it up.

The last link is especially interesting because a couple of other repairmen get into it and try to straighten things out.  By the very bottom of the discussion, the actual possible problems finally show up.

So, what was wrong with mine?  The door needed to be adjusted.  Yep, the door has sagged a little bit over the years and was far enough down that it didn't close a switch at the top that turns on the fan for the evaporator.  See, if the fan isn't on, the evaporator goes into freezing range and the temperature sensor I mentioned above, shuts off the compressor to prevent ice forming and breaking the evaporator coil.  I drug out the tools and modified the top hinge to be adjustable and reset the door.  Problem solved.

The reason I had to modify the hinge was because GE made provisions for raising and lowering the door, but not tilting it.  They even avoid the subject in their installation manual (yes, I read it) like a door will never, ever skew a little bit.  The experts out there will tell me that I should adjust the fridge to assure that it's square because that can cause the same problem; I did that first with a nice level, framing square, and all.  Nope, the door actually needed to be adjusted.

But the point of this is that after I found the two discussions listed above, I found several others where the expert just wouldn't listen to the customer, assumed he knew everything, and screwed it up.

I'll never, ever ask a question on one of those sites.

But, there were a few good things that came out of this fiasco.  I moved the Smart Switch back to the fridge (I had it on the freezer for a few days), and took a look at the power usage during the problem:


Notice the small spikes?  It seems the compress was turning on for a very short period, then shutting off for a very long period.  This was because the evaporator coil got too cold (no airflow) and the thermal protection kicked in and shut the compressor off.  This is very different from the graph I blogged about earlier <link>.  Another thing good about this was that I improved the performance of the appliance:


Now, the compressor is on longer, but off longer as well.  I turned the temperature control down and allowed it to stabilize at 38 F and all seems to be well.  Of course, something else can crop up and cause problems, but it looks like I fixed it.

Kissing Goodbye to DynDns

$
0
0
For a long time now I've been using Dyn's free service DynDns.  For those of you that don't understand what I'm talking about, most homes have DSL provided by one of the big names or cable modems provided by some of the same big names.  I happen to have CenturyLink (used to be Qwest).  With these services, your IP address changes periodically so it's hard to have a name linked to an IP address when it changes every time the provider decides they want to.  At one time CenturyLink was changing my IP address a couple of times a day.

So there are a number of dynamic dns services out there that will update the various name servers on the internet when your IP address changes.  Doing something like this saves you the extra fee for a static IP and you can still get to your home machine by some name or other.  These services used to be free, but over time, they've started to charge for the service.  Dyn just did this.  Over the weekend I got an email telling me that my free account with them would be cancelled.

So, what to do?  I simply signed up with a different service and changed the bookmark in my browser.

Now, a lot of you folk can't do it that easily.  You may have distributed the name of your site or machine to friends and relatives and now you would have to let them know it changed.  Or, perhaps wait until they try it and give you a call to see what happened.

Don't misunderstand, I don't mind paying for value.  If they had said that they were going to charge $5 a year for this service I wouldn't have even thought about it, I would have just signed up and paid the five bucks.  They wanted $25 a year (discounted the first year), and frankly, that's too much for such a simple thing as sending an update to the name servers that your IP address has changed.  To be fair, they offer a number of other services and have a good reputation, but I don't need the other services.  Perhaps someday when I have a huge corporation to worry about, I might consider them again.  For now, my one little blog and one little computer will do just fine without them.

So, if you want to visit my home control and monitoring system, use this link <link>.  Strangly, it took me about six minutes to get it working and another ten to find the software to keep it updated through IP address changes.  That's about 1/10th the time it took me to set it up originally with DynDns.



Why I'll Never Buy Another Motorola Cell Phone

$
0
0
A couple of years ago I pre-ordered a Motorola Razr Max cell phone.  It came right on time, the day it was released, and was SO COOL!  It was running Android and had all the bells and whistles that modern technology could create.  I specifically waited for this phone because they put a battery in it large enough to last a full day of normal use and because the screen was big enough to actually use the device.

I was so stoked.  I played with it for days, customizing every little tiny feature.  I carried it with me everywhere. It became a companion.  Don't get me wrong, I don't spend all my time chatting on the phone, text messages are mostly unused, and email can wait until I have a real keyboard to use.  But I could turn my lights on using a web browser from the Burger King in town.  This is a techie's dream.

Then one day I was taking a picture in the back yard and the screen cracked.  Yep, it cracked vertically from top to bottom when I touched the little icon on the screen.  I couldn't believe it.  Here was a device that had a Kevlar case and Gorilla Glass screen and was advertised as being able to handle 'real life' that broke under my index finger.

I called the cell phone provider about the problem and met with the, "physical damage is not covered under warranty" line.  I argued for almost an hour that taking a picture wasn't damage, it was normal use, but they didn't give an inch.  I took it to one of the corporate stores and they did the same thing. I was totally annoyed and actually thought about a little mayhem as a possible route to some degree of satisfaction.  I even contacted the manufacturer about it.  The result was exactly the same.  Their logic is that a cracked screen is a result of damage, not bad design or manufacture.  They couldn't possibly be at fault.

Having exactly zero success trying to get someone to acknowledge that there was a problem that needed correcting, I used the insurance I bought along with the phone.  A hundred bucks later, I had a new phone just like the old one, but the magic was gone.  The device was too fragile to trust.  I equipped it with an otterbox, which make it boxy and ugly, but it might survive being used and nursed it along.

During this time I noticed a slight bulge on the back of the phone; the darn battery was swelling.  Yep, the state-of-the-art lithium-ion battery was puffing up and distorting the case.  So much for embedded batteries that you can't change; it was failing in one year nine months and bending the back cover.  Again on the phone with the provider and they say that it's out of warranty and there's nothing to be done, but I could use the insurance again (another 100 bucks) and get a replacement.  I decided not to send good money chasing after bad and continued to treat the phone like a new girl friend until the contract ended.

My thinking was that I'd get a different manufacturer's device and maybe have better luck.  And, as luck would have it, Samsung announced that their Galaxy 5 would come out in a couple of months; last Friday (the release date) my new phone was delivered to the house (again on the day of release).  I'm in love again.

However, I still have a bitter taste in my mouth about putting up with a piece of crap for two years because Motorola created a hunk-o-junk, but I'll get over it in time.  Here's a couple of pictures of the damage done by the expanding battery:


The bulge is really obvious, but take a look at the next photo.


The case has actually split at a seam and you can see inside the phone if you get a flashlight.  This is the damage that Motorola and Verizon won't warranty because too much time has passed.  Just like the crack down the screen they wouldn't cover because of 'physical damage'.  The swelling actually broke the speaker on the back, so I couldn't hear a ring; I had to rely on vibrate for the last month.

What a racket.

Yes, I know that one should get a case for a cell phone.  But doesn't that eliminate the need to make it slim, attractive and stylish?  Take a piece of high tech jewelry and hide it inside a poly carbonate case with a silicon protector around it and it looks like a plastic box, not a cell phone.  Yes, I know that screens can crack, but when you touch it to take a picture??  What the heck is up with that?  That generation of Gorilla Glass doesn't deserve the name.  The good thing is that I tested the heck out of the Otterbox case; I'm certain that the case was the only reason I didn't have to replace the phone a couple more times.

So ends an annoying chapter of my life.  I would change carriers to get away, but no other carrier works out here.

Now, I get to play with all the features of a new phone.  I get to customize every little thing and put pictures of my dog on it to annoy other people with.

Motorola used to be a great company.  Maybe Google will bring some of the life back to them since they took over the mobile phone part, but I'm keeping my distance.

Android, Samsung Phone, My very own app, and a Raspberry Pi Home Automation Server

$
0
0
I did it; I finally took the plunge and created an app for Android that runs on my phone and controls the house.  I didn't want to go through the learning curve necessary to conquer the Android Development Environment, so I just relied on the web interface for all this time.  The problem is that I got a new phone and it was too great a temptation for me.  I had to do it.

I lucked out and ran across AppInventor 2 from MIT (it was once owned by Google) and it looked pretty cool, so I started trying it out.  The very first tutorial was enough to get me wound up.  Over several days, I experimented and coaxed an app out of it and then went totally nuts.  It's still a work in progress like all my projects, but it is SO COOL!

Here's the screen you see when you first tap on it


The first three buttons: Power, Temp, and Clock open the SteelSeries gauges I've already created and put them up so I can watch them.  The clock is bound to the time at my house so I can see exactly what time it is there wherever I am.  The data is supplied by my Raspberry Pi in JSON format so there is very little traffic to load it down.  All the formatting and such is done exclusively on the phone saving my little Pi for doing the monitoring tasks.  Those buttons were a pain.  Do you know how hard it is to find cool buttons out there?  That is without having to spend days editing them to get them to look OK.  Here's what it looks like with the power gauge showing:


Yes, this is the exact same display code I use for the web interface.  Appinventor 2 has provisions for a little browser that can handle javascript code.  I can't do the fancy graphs yet, because they are Flash based.  I'll have to look into that in the future.  Just for fun, here's the display when I put up the temperature gauge:


I stack the various controls and gauges one on top of the other so I can scroll between them if needed.  For example, Here's the screen when I have both thermostats showing:


I color the buttons yellow to show inactive and green for active which means they should be showing on the screen.  You can see how the two A/C unit selectors are green and below the controls and status of each of them.  One of them is idle and the other is recirculating the air; that's what the green fan is showing.  Yes, the fan turns; figuring out how to make it rotate was painful since the tool doesn't support animated GIFs yet.  The black fan doesn't turn silly, it means it isn't running.  There's also red and blue fans to show heating or cooling.  The tool has a 'picker' screen built into it, so to change the mode from Off to something else I pop up this display to choose from:


There's something similar for the fan also.  I got fancier on the temperature and put up a slider and set buttons:


When you slide it towards the red, the temperature setting goes up and similarly the other direction, then touch set and you have a new temperature setting.  When you touch set, the temperature controls go away and it leaves you with just the thermostat readings displayed.  This gives me darn good control over the thermostats and a nice flashy display.  I put pictures on the lights to show their state:


Those lights were a total pain in the butt.  I had to construct them from pictures that were waaay too big and didn't match the off representation.  Took a few hours using paint.net to get them right.  To turn off the lights, just touch the icon itself:


Notice how I managed to find a nice clear bulb and use it for both states.  No, I can't dim the bulbs, nor do I want to.  The garage doors are still pretty primitive:


But, they get the job done.  I want to fancy that display up some over time.  Similarly, the pool controls are still in the beginning stages:


Lots of refinements are needed here.  All of it works though, I can turn on the pool light anywhere I have data service on the phone.  Of course, no rural place would be complete without knowing the state of the septic tank:


When septic gets too full, the OK icon goes red.  I do the same thing when the acid pump goes low.  I show this to people and they invariably click on the septic tank:


I love their reaction when they have to select "I'm a Dummy" from the picker.  There's a couple of Easter Egg messages like that in other places as well.  Hey, gotta have fun with it too ... right?

Needless to say, I spent a heck of a lot of time just playing with the controls and watching things turn on and off around the house.  I actually had the front lights cycling in a kind of rhythm for a while, much to the amusement of the neighbors.  I guess the new will wear off in a few weeks, but then I'll just make something else and hook it into the system.  I'll also be looking for cute icons to include in various places in the app; it would be nice to have a cartoon garage door to show when it is open.  I'm thinking about using my old cell phone as a house control.  Take the battery that expanded out <link> and hook it to power.  Since everything will work except the actual phone, I should be able to turn it into a house controller that sets by the bed, or maybe mount it on the wall.  It would give me great pleasure to repurpose that device so it would actually get some use ... finally.

I can't wait to show this off.  So, people, what's holding you up from building something like this as well?  No, I won't port this to the iPhone.

Wemo Devices Are Getting Interesting Now

$
0
0
I'm not one of those guys that gathers tech news items and reposts them to increase my page views or hits, but this is going to be an exception.  I was prowling around looking for ideas for a particular problem I'm having with the Wemo devices I have and ran across a cool news item <link>.  Basically, it says that Belkin is expanding their line of WiFi controlled devices to include light bulbs, slow cookers, coffee makers, humidfiers, and who knows what else.

This is so cool.  Imagine, a coffee pot with an internet address.

I thought I'd capture a picture from a 'coming soon' article that really caught my eye:


I stole the picture and put it here because the advertisement will probably disappear soon.  The link is from the CrockPot web site <link>, and is for a Wemo based slow cooker that is going on my 'gottahavethatthing' list.  Being a widower, I use a slow cooker a lot and this would be great to have.

But look, space heaters, humidifiers, coffee makers, what will they think of next?  Sure, I use things like this to control or monitor a lot of devices and I could probably build my own remote controlled coffee maker, but the neighbors can't.  Now, they can just buy it.

So, it looks like Belkin is in this marketplace big time.  They actually published their API for Android (yes, I downloaded it) so other people can get into the arena.  With the library work in Python and several other languages, the home hacker can start building nice controls for their own use.  Normal folk can use the free phone app and do much the same with no work (or personal satisfaction).

Home automation is looking better all the time.

Hooking HighCharts Into My House

$
0
0
If you prowl around this blog very long you'll run into a bunch of charts.  I've experimented with several cloud services for storing data and shown examples of my own data using their charting provisions.  All of them are cool, but some of them leave a little bit to be desired.  For example, Xively uses Rickshaw <link>, and it's a nice package, but the Xively implementation is too darn complex for my taste.  If you look at the various cloud providers, they all have their favorites.

Up until now I've been using the Google graph API for my own stuff, but the problem is that Google Graph uses Flash.  Cell phones don't like Flash very much.  That means my home graphs don't work on my phone.  I absolutely can't let that continue any longer, and since my favorite cloud provider, Grovestreams <link>, uses HighCharts <link>, I stole their example and developed my own graph using the same library.

Hey, they stole my idea for the SteelSeries gauges, turn about is only fair play.

But, as usual, it was an exercise in patience.  The documentation on HighCharts is voluminous; it goes on forever and ever with tons of examples and links into JFiddle <link> so the ideas and stuff can be experimented with; talk about information overload.  It's also tough to figure out how the set the various properties in the right place to get things done, and none of the examples did exactly what I needed.  That's why patience was required.  I didn't want to get disgusted with it and just give up.

Here's the chart I came up with for my phone:


This is actually pretty nice.  If you put your pointer inside the chart, you can get the actual reading of any point.  If you click, drag across the chart, you can zoom in and see it closer.  If you have a touch screen, you can use the 'pinch' and 'expand' to zoom in and out; when your zoomed in, you can slide the chart right and left to see the data.  By clicking on the labels at the bottom, you can turn off one of the series to make the remaining one more clear.  I could put up a lot of things to examine and choose which one by turning the others off.  I'm not sure I like the colors yet, but they're easy to change.

The beauty of this chart is that it's totally javascript.  That means it works on the phone, so I can add it to the Android app I have:


OK, the chart looks silly.  But look what happens when you turn the phone into portrait mode:


Is that slick or what?  Using the touchscreen I can zoom around it and select points to my hearts content.

The data shown is taken from my legacy feed on Xively and was reasonably easy to grab.  The only real problems I had getting it going was prowling through the documentation on HighCharts; it was all there, but like finding a needle in a haystack.

If you want to steal my example and modify it for your own use, the first chart on this page is live, so just right click in the chart and 'view frame source'.  Doing it that way will get the very latest changes I might have made.  However, my ISP has expired my IP lease three times today already, so you may notice a problem while the DNS servers get updated.  I may have to look into a static IP address soon since they have been doing this more and more recently.

Jasper for voice commands

$
0
0
I'm rapidly getting to the point that I need to have a simple remote control that I can use around the house.  I've thought about several possibilities, but I ran across Jasper <link> and that would be so cool.  A house that I can talk to.

On their site, they have a great video of the microphone setting a few meters away and the authors gleefully giving it commands and listening to the responses.  I could make a few of these and scatter them around the house to control things by voice; that would really be nice.  Very slick site and seemingly well documented.

So, looking at the directions, they have a pretty complex system that installs voice recognition and speech capabilities on the Pi.  I decided to try it out.  Since their image file is most likely configured differently than what I use, I decided to do the full installation ... DON'T DO THIS.  Their instructions leave a lot of tiny little things out, like what directory to be in when you do things, how long the various steps can take, how freaking big it is.  Y'know things like that.

After 26 hours of installs, updates, compiles (some of which failed) and a whole lot of head scratching, I just gave up.  I still wanted to try it out, so I downloaded their disk image and installed it on my Pi.  After spending quite a while messing around, I still couldn't get it to respond to a command or hear what it was saying; obviously there was something wrong with the audio setup.  The authors used ALSA (google it) so I started digging into it to see how to test the audio.  After changing the terminal interface on Putty several times, I discovered that my alsamixer settings were all set to the minimum.  After jacking up the input gain and the output volume, I managed to get sound into and out of Jasper.

Then the disappointments really began.  I couldn't recognize my voice most of the time, when it did manage to get it, it was mostly wrong.  "What is the meaning of life," one of the built in commands, would check my mail.  Interesting, but not what I expected.  It would do the time pretty well, but most everything else would cause an exception in the python script that was running.  Sometimes, it would fail so bad the Pi actually crashed.  Nothing has done that to me before.  Remember, this is a disk image, I didn't change anything; it failed right out of the box.  The voice synthesis was really, really hard to understand, and playing over and over again didn't seem to help.  My ears just weren't training to the odd sound.  Sort of like the Pi had a really bad cold and couldn't quite get the words out.

I didn't expect it to work like the movies, but c'mon, it should at least be as good as the videos they produced to show it off.  I even used the exact same microphone they show on their site and some really good powered speakers.  I can't blame the hardware at all.

So, I know there are a number of folk out there that installed this and got it to work.  There always are, but obviously, they didn't build it on their Pi, because that simply doesn't work, and not many people have days to waste compiling thousands of source files.  So, they must have used the image, but what the heck did they do to get it to understand them?  Similarly, how the heck did they understand what was being said by the software.  A series of beeps would have been better.

A few hours spent looking at various things on the web didn't reveal any secrets to make this work.  Instead, I found a significant number of people that hadn't been able to make it perform.  There didn't seem to be any solutions either.  There were a lot of, "I think," or "Maybe you can try," and "Have you tried;" but I discount most of those since they are simply guesses.  I didn't find anyone that was bragging about their success; that is very telling.  I'm not willing to dig into it because the vagaries of ALSA are daunting enough without having to delve into the secret and mysterious world of PocketSphinx (the recognition system they use).  I'm not interested in a new career.

I'm burning my original software back on the Pi's card right now so I can try something else.  For you folk out there that want to try it, I certainly hope you have better luck than I did.
Viewing all 218 articles
Browse latest View live