Quantcast
Channel: Desert Home
Viewing all 218 articles
Browse latest View live

Voice Recognition on the Raspberry Pi - Reality Check

$
0
0
My recent experience with voice control on the Pi got me to thinking.  Why wasn't this a rising star and constantly being talked about?  The idea of talking to your house is so compelling that there must be hundreds of implementations out there.

Well, there are, and none of them work very well.

I described my experience with Jasper, it just didn't live up to the hype.  So I went looking and did some experimenting.  Everyone talks about how good Siri is.  My experience with it is far less than stellar; all the phones I've tried it on misunderstand me about 6 out of 10 times.  Google's implementation seems to work better and I get about an 80% success rate.  Both of these are stellar compared to several software techniques I tried out, with the absolute worst being CMU Sphinx that Jasper was based on.

Remember, I'm looking at this as a way to control my house with a little computer, not dictate letters wearing a headset, so let me talk a bit about methods.  No, I'm not going to bore the heck out of you with a dissertation on the theories of voice recognition, I want what everyone else wants: I want it to work.  There are basically two methods of doing speech recognition right now, local and distributed.  By local I mean totally on one machine, and distributed is when they send the sound over the internet and decode it somewhere else.  Google's voice API is an example of distributed and CMU Sphinx is an example of local.

What we all want is for it to operate like Star Trek:

"Computer."

Nice clear beep

"Turn on the porch lights"

Nice clear acknowledgement, and maybe a, "Porch light is now on."

I went through the entire process of bringing up CMU Sphinx <link>, and when I tried it, I saw something on the order of, "Burn under the blight." To be fair, Sphinx can be trained and its accuracy will shoot way up, but that takes considerable effort and time.  The default recognition files just don't cut it.  Especially when I tried the same thing with 100%, yes totally accurate results with Google's voice interface.  The problem with Google's interface is that it only works in the Chrome browser.  Yes, there are tools out there that use the Google voice API; notably VoiceCommand by Steve Hickson <link> , but expect it to quit working soon.  Google ended their offering of version 2 of the interface, and version three is limited in how many requests can be used and you have to have a special key to use it.  Thus will end a really cool possibility, I hope they bring it back soon.

So, the local possibilities are inaccurate and the distributed are accurate, but the one everyone was using is likely to disappear.  There are other distributed solutions, I brought up code taken from Nexiwave <link> and tested it.  There was darn near a 100% success rate.  The problem was delay.  Since I was using a free account, I was shuffled to the bottom of the queue (correctly and expectedly) so the response took maybe three seconds to come back.  Now, three seconds seem like a small price to pay, but try it out with a watch to see how uncomfortable that feels in real use.  This is not that Nexiwave is slow, it's that the dog gone internet takes time to send data and get back a response.  I didn't open a paid account to see if it was any better, this was just an experiment.

But, think about it a bit.  "Computer," one thousand and one, one thousand and two, one thousand and three, "Yes".  Then the command, "Turn on the porch light", etc.  It would be cool and fun to show off, but do you really want to do it that way?  Plus it would require that the software run continuously to catch the occasional, "Computer" command initiation.  Be real, if you're going to have to push a button to start a command sequence, you might as well push a button to do the entire action.  Remember, you have to have a command initiator or something like, "Hey Jeff, get your hand out of the garbagedisposal, it could turn on," could be a disaster.  A button somewhere labeled, "Garbage Disposal," would be much simpler and safer.

Don't talk to me about Dragon Naturally Speaking from Nuance <link>.  That tool is just unbelievable.  It is capable of taking dictation at full speed with totally amazing accuracy, but it only runs on machines much larger than a Pi, and not at all under Linux.  Even their development version is constructed for Windows server machines.  Microsoft has a good speech recognition system built right into the OS, and under Windows 8, it is incredible.  Especially at no additional cost at all.  But, there aren't many Raspberry Pi machines running Windows 8.

Thus, I don't have a solution.  The most compelling one was Nexiwave, but the delays are annoying and I don't think it would work out long term.  Here's the source I used to interface with it:

#!/usr/bin/python

# Copyright 2012 Nexiwave Canada. All rights reserved.
# Nexiwave Canada PROPRIETARY/CONFIDENTIAL. Use is subject to license terms.

import sys, os, json, urllib2, urllib, time

# You will need python-requests package. It makes things much easier.
import requests

# Change these:
# Login details:
USERNAME = "user@myemail.com"
PASSWORD = "XYZ"

def transcribe_audio_file(filename):
"""Transcribe an audio file using Nexiwave"""
url = 'https://api.nexiwave.com/SpeechIndexing/file/storage/' + USERNAME +'/recording/?authData.passwd=' + PASSWORD + '&auto-redirect=true&response=application/json'

# To receive transcript in plain text, instead of html format, comment this line out (for SMS, for example)
url = url + '&transcriptFormat=html'


# Ready to send:
sys.stderr.write("Send audio for transcript with " + url + "\n")
r = requests.post(url, files={'mediaFileData': open(filename,'rb')})
data = r.json()
transcript = data['text']

# Perform your magic here:
print "Transcript for "+filename+"=" + transcript


if __name__ == '__main__':
# Change this to your own
filename = "/data/audio/test.wav"

transcribe_audio_file(filename)
I took this directly from their site and posted it here because it is hard to find, and I don't think they care if I advertise for them.  All I did to make it work was to sign up for a free account and enter my particulars in the fields up at the top.  It worked first try; simple and easy interface.  It would be relatively easy to adapt this to a voice control system on my Pi if I decided to go that way.  Which I may do for control in the dark of my bedroom where I don't want to search for a remote that may be behind the side table.

The audio file I sent was my usual, "Porch light on," and it decoded it exactly first try.  I tried a few others and they all worked equally well.  Which brings up another item, sound on the raspberry Pi.  Frankly, unless you're dealing with digital files and streams, it sucks.  There isn't enough filtering on the Pi to keep audio hum out of things.  The amplified speakers I was using had a constant low level hum (regular ol' 60 hertz hum), and it would get into the audio captured from the USB microphone as well.  This could have been reduced by an expensive power supply with very good filtering, or maybe not; I didn't try.

To add insult to an already injurious process, ALSA (Advanced Linux Sound Architecture) is the single most confusing sound implementation I've ever seen.  It was constructed by sound purists and technology students so it is filled with special cases, odd syntax, devices that mostly work, etc.  The documentation is full of 'try this'.  What?  I love experimenting, but I sort of like to have documentation that actually has information in it.  Pulse audio is another possibility, but I'll approach that some other time.  Maybe a few weeks after hell freezes over, ALSA was bad enough.  But, if you're going to experiment with sound under Linux, you'll have to deal with ALSA at some point.  Especially if you actually want to turn the volume up or down.

I think I'm going to do some research on remote control ergonomics.  There's got to be a cool and actually useful way to turn on the porch lights.

Smoking Meat and Power Usage

$
0
0
I have an electric smoker.  Why electric?  Well, I got tired of running out of fuel during long smokes and decided to just break with tradition and increase the technology.   But, I've wondered for a long time how much power these things use.  Sure there's a label on them, and they're almost always wrong in my experience.

Wait though, I have this cool Iris switch that measures power, and I have charting tools, and I have cloud storage.  Let's take a look:


It uses around 800 watts when it's on and nothing when it's off.  After a start up period of about 20 minutes it settles into a regular cycle that varies around three minutes on and three off that expands to longer off times as the food inside rises in temperature.  If I had run it longer, I suspect the off times would have stabilized to periods approaching 10 minutes.

So, the device doesn't use a lot of power for something that keeps wood smoldering to cook and flavor food, but it could be death on my 'Demand' number for the month.  An extra 800 Watts added to my base usage during this time of year would probably push me up over 2kW and cost around $15 more on my power bill (I usually run around 1.6 kW this time of year).

For you folk that don't have demand billing, this looks like it would be .45 kW for each hour of usage on average if you ran it for 4 hours or more.  Not too bad for smoking food, your dryer uses a heck of a lot more than that.  Just to illustrate how this can combine with normal house electrical usage, take a look at my overall usage during this same period:


This makes it look like I'm a total power hog.  My usage tops out around 17 kW and jumps over 10kW several times during the morning.  That's what happens when you're doing laundry; the dryer is chewing up almost 8kW by itself when the element is on, the washing machine motor eats it fair share, the swimming pool filter is running on high, and the smoker is stinking up the area.  This is life though, we have to figure out how to deal with it and not let the power company take away our savings.

Notice that everything shuts off at noon?  Yep, I carefully shut things down to prevent the demand billing system from eating me alive.  Actually, that's what this site was started for; back before I wandered off into anti-siphon faucets, rattlesnakes, acid handling, battery maintenance and such.  Funny how things take on a life of their own.

It looks like smoking will be reserved for nights and weekends.

Apple's Big Home Automation Announcement

$
0
0
A couple of days ago I ran across an article that said Apple was going to announce a movement into home automation. Every time Apple coughs, the news services go orgasmic and start showing everything they can, the tech copycat articles start popping up using text they stole from some other site, and the news aggregators (huffington and the like) duplicate the articles all over the place.  I knew this was going to be fun.  Here's a link to the article I ran into <link>

So, today was the day that Apple was going to make their earth-shattering announcement.  The home automation part of it didn't really get out there until several hours latter, the copycats were too busy talking about the Beats purchase and maybe some new changes to iOS.  But, I found one here <link>, I'll bet the audience of Appleophiles were going all big eyed at what was possible.  Why, you can shut your garage door, set your thermostats, turn the lights on ... I actually laughed out loud.

For goodness sake, folks like me have been doing this for years now.  We control our pool, measure the weather (inside and out), and even measure the level of our septic tanks; we mix and match devices, services, and web pages to our hearts content.  We don't pay monthly subscriptions and are not at the mercy of a single supplier.

We are free to handle our house the way we want to.

I think it's a good thing that they are doing this because it won't be long before one of us hacks into the various devices and makes them work for us.  This means more devices we can play with, more things we can automate, more of Apple's research we can leverage to our own ends.  I don't think they think like I do though; they'll want a monthly fee for web services and get even richer.  I'll be chuckling as I take apart their devices to see how they work.

Another good thing, Apple's announcement will open people's eyes to what is possible.  Much more so than my little blog and the hundreds of others out there that do the same kind of thing.  There's a whole lot of us that are doing it ourselves, and this may enlist more of us.  Sure, we're outnumbered by the folk that just buy a solution, but where's the fun and satisfaction in that?

Meanwhile, I got a notice today that I'm going to get a smart meter on the house.  Finally, the powers-that-be are moving technology out into my neck of the cactus plants.  Now, I may have to build up a device to read the pulses from the meter and get power readings that are as accurate as theirs, but much more current than what they provide.  I suspect I won't be able to crack into their monitoring signal, but I don't really need that.  However, you know I'm going to try.

Just for Fun

$
0
0


I just answered an email from a nice person out there that was curious about my avatar that shows up in various places I post and my little home web site.  It's a squirrel:


Let me introduce you to Conker. He's the hero of the video game Conker's Bad Hair Day, the very best of the 'M' rated video games ... ever.  This game came out well over a decade ago for the Nintendo 64 and became a cult classic.  Microsoft bought out the company that produced it, Rare software, and totally destroyed the cleverness of the company's offerings.  Microsoft did re-release Conker as, Conker, Live and Reloaded, but they changed it a bit and used it as a gateway into XBox Live.  But, enough history.

I love this game.  I actually keep a Nintendo 64 around to play the game and occasionally drag it out to refresh my memory.  What could keep the attention of an old man that must have outgrown silly little games a long time ago?  Here's a link to one of the fun monsters in the game.  It's a rather long video, but you'll see what attracted me to the game.  <link>


I decided that Conker was my hero.  This led me to use other squirrels from time to time as avatars, or just to illustrate a point:






I guess I like squirrels that can fight back.  The game has other monsters to conquer, Dracula, Death, etc.  It's a fun game that kids today will never get to experience.

Unless they come to my house.

My Freezer and the Iris Smart Switch, Measuring Power

$
0
0
I went to Lowe's Home Improvement the other day to get supplies for jobs around the house that need to be done, you know caulk needing replacement, some spray foam, wire brush, the things we all have to get from time to time, but somehow, I wound up in front of the Iris display.  Shame on me.

When I got home with the stuff I needed, and my new Iris smart (which was NOT on the list) I decided to hook it to my freezer so I can monitor both big appliances in the kitchen.  Since I was lazy the last time I played with these devices, I had to modify the code to support more than one switch.  That wasn't too hard.  The switch connected first try and started reporting.  I was a bit surprised at what I got out:


I labeled a couple of interesting things; remember I have separate refrigerator and freezer, this is the freezer only.

The spike and drop at point A is the automatic defrost running.  The way these thing work is there is an actual heating element on the evaporator in the freezer.  A timer runs it a couple or more times a day.  The timer kicks on and the heater brings the evaporator up to 40°F when a thermostat on the evaporator coil opens and turns off the heater.  The rest of the defrost cycle runs and control goes back to the temperature control system.  Which usually means the compressor kicks on.  This period of time allows the water to drain away into a pan where a fan evaporates it.  My defrost cycle falls in the 'Peak' period of billing, so that's not good.  The timer also runs on a little less than a 24 hour cycle, so it doesn't kick in at the same time each day.

This all means that I have a heater kicking on during the peak period that could be run some other time.  Fortunately, there's a little slot on the defrost timer that allows it to be advanced such that the defrost time can be moved, but since it will eventually move back into the peak period, I'll have to readjust it from time to time.  Notice it runs twice a day, that'll just complicate matters a bit.  This little timer may be a good place to put an Arduino synced up to my house time for absolute control of the defrost cycle.  No, I'm not getting too anal about this ...

Point B is the lights inside the darn thing.  This freezer has 8 stinking lights in it; four on the top, and four on the bottom.  Time to see if I can get a deal on led lights to replace these incandescents along with the ones in the fridge.  I mean the lights actually use more power while they're on than the compressor does.  Granted, they aren't on very often or for very long, but crap, close to 300 watts to light up the freezer?

Point C is the automatic ice maker.  When ice is low, this thing kicks on periodically to roll the new cubes into the tray and opens a valve to let water in for more.  This seemingly simple operation uses over 150 watts.  This is also short lived, so it can be tolerated.

What surprised me was how long the compressor runs.  I was worried that there may be something wrong since low coolant pressure or an air leak can cause the compressor to run too much.  I went to the GE site and looked up their explanation of the compressor cycle and what I'm seeing is perfectly normal.  The compressor on a freezer can run from 75 to 90 percent of the time under normal operation.  This seems excessive, but the motor is designed to take this much use.  It only uses 130 watts, so that's not too bad, and it does cycle somewhat, especially in the late evening early morning period.

What's becoming more apparent over time is the insignificance of parasitic power devices.  Little things like cell phone chargers that are left plugged in, televisions that look for remote control signals, wireless phones, etc.  I'll start checking on these devices, but it appears that all of them are roughly equivalent to a 60 watt light bulb.  You'll get a lot of bang on the power bill by paying attention to the lights and ignoring the small devices around the house.

Now, I have to get the caulk gun and get back to what I'm supposed to be doing.

I Got My Smart Meter - North of Phoenix, AZ

$
0
0
Yep, today I got my smart meter.  I called the power company yesterday and asked them the questions I had and guess what ... they didn't have the answers.  Are you surprised (not!)?  No, I'm not a health nut; I realize that the frequency of the transmissions is non ionizing, and that the intensity decreases with the square of the distance, so this tiny radio on the side of my garage is about 1/1000th as dangerous as a wayward Lego on the floor.

No, what I wanted to know is:

Is it Zigbee?  No, they chose another transmission and protocol method.  I don't know what, because the tech didn't either, however it is a store forward mesh network that can use other nearby meters to forward the data to a central collector.

What frequency range does it use: It's in the 900mHz band.

How often does it report? The tech was unclear on that, but the total time is only seconds a day which leads me to believe that it's on an hourly basis.

Who made it?  The meter is the common Elster REX2.  Which negated a whole series of questions about what technique was used to measure the power coming into the house.  I'll talk more about this a little further down.

Were they any  good? He thought they were accurate and seemed to be holding up well in the heat, but the cover tended to glaze over from the sun and there seemed to be a higher replacement rate for not reporting than the old meter.  This is a little strange though since the old meter didn't report at all; they had to come out and actually read it.  I suspect he was talking about a seemingly unusual rate of not reporting for the meters he had installed.

Where was the collector for the meters?  It's down the road and around the corner roughly a mile from my house.  The meters are rated for six miles on their own, so the data may not have to be relayed at all if the range claims are true.

How does the data get to APS (the power company)?  At the collector it is forwarded to Verizon and uses their facilities to get it to the power company data center.  Apparently, they use both cellular and wired depending on where the collector is and what facilities are available.  Out where I am, they use cellular links to forward the data.

Did it have a pulse output? NO DARN IT.  I was looking forward to setting up hardware to read the pulses to see how accurate my measurements are over the long haul.  This was disappointing, but since I already measure the power myself, I don't have much to complain about.

So, I didn't gain much with the new meter.  However the power company will collect data by usage time and I can prowl around their data and compare it to mine.  If they show a significant difference, I might have to talk to them about it.  Although, if they record less than I read, they won't be hearing from me.

What's fun though is that people have taken these meters apart and looked inside <link>.  This article is absolutely great because it matches my meter exactly.  I stole a couple of pictures from the article because I can't be sure it won't disappear at some point and I want to point out a couple of things:


This is the bottom plate of the meter.  Notice the large wires to connect the supply to my house, they have a current transformer wrapped around them.  This is almost exactly the way I measure power with my power monitor <link>  The big difference is that I had to use two CTs to measure the power since the wires were too far apart to get one CT to go around them both.  Nice to have some validation after so long.  I've had to explain this a thousand times to various folk.  Here's a picture of the CT with the wires after the author removed them from the housing:


Clever weren't they?  The CT leads into a circuit board that measures the current and voltage to calculate the power usage.  Darn, this is really starting to sound familiar.

There's a ton of conspiracy folk out there that see smart meters as a threat.  They could become one at some point given an unrestricted power company and shoddy government oversight, but I'm not worried about that.  I'm excited about the technology.  This little guy can measure my power and send it up to APS (my power company) once it's there, I can grab it and do comparisons against my own measurements.

Heck, I'm using my power company as another cloud storage.

Taking apart the Iris Smart Switch

$
0
0
I decided to take the plunge and finish off the 120V appliances I want to monitor right now.  There will probably be others in the future, but for now the two freezers and refrigerator will be the devices I keep an eye on.  I also want another Iris Smart Switch to wander around the house with.  It was nice to be able to evaluate the new smoker <link>, and I'm sure I'll want to do that again with some other appliance (TV, wall warts, maybe a Raspberry Pi) from time to time.

This time though, I'll finish the software to monitor the appliances and take the switch apart to modify it to make sure the appliances can't be turned off accidentally.  It's been a concern of mine that the switch can be both locally and remotely controlled and my major appliances are fed through it.  I don't want a power surge or EMF pulse to shut down my freezers.  So, I got out the screwdriver and a knife.  I peeled the label covering one of the screws and took the thing apart.

I was expecting something like the X10 devices I've dismantled over the years, and was I ever surprised, this thing is really well made:


Notice that there's a well laid out circuit board, nice large wires to handle the load, and substantial connectors for the power in and out.   Right off I noticed that they had a resistor in the circuit for power.  Yes, they're using a real shunt resistor to measure power.  Talk about old school, tried and true, easy to work with design.  There's a pretty substantial set of filtering caps to keep noise out of the circuitry and a 16A Omron relay for control.  The point I wanted was where the power is broken by the relay; I just need to short across the relay connections and I'll be done.

For the folk out there that want to see the rest of it, here's the back side of the board:


The relay is a single pole, double throw and they use both sets of contacts.  I was impressed by the construction and may, someday, take the board out and use it for control of something else.  For now, having it at the plug is really great.

After I shorted across the relay pins --- go look up the relay to get the pins <link>, I don't want to be responsible for you messing it up and screwing up the switch --- I put it back together and started working on the code.  The way I did it is, if a new device shows up, I put an entry in my Sqlite3 database holding the fields I'm interested in (all I use) and assigned it the name of 'unknown'.  Then I go ahead and let the switch join.  It will immediately start sending data, and I save it to the new record in the database.  Then using the command line interface for Sqlite3, I just give it a name and it is now part of my network.  If you want to store it in some other fashion, just change the code to accommodate whatever technique you are using.  However, I really like the Sqlite3 database, it serves as storage for all my devices and has worked like a champ for months now.  If you're doing this on an Arduino, you can use the non-volatile storage capability.  I'm not going to port this code to the Arduino though, I have no need for it there.

The Python Script
#! /usr/bin/python
# This is the an implementation of monitoring the Lowe's Iris Smart
# Switch that I use. It will join with a switch and does NOT allow you
# to control the switch
#
# This version has been adapted to support more than one switch and will
# add a new record to my database to hold the data. Adapt it as you need
# to.
#
# Have fun

from xbee import ZigBee
from apscheduler.scheduler import Scheduler
import logging
import datetime
import time
import serial
import sys
import shlex
import sqlite3

#-------------------------------------------------
# the database where I'm storing stuff
DATABASE='/home/pi/database/desert-home'

# on the Raspberry Pi the serial port is ttyAMA0
XBEEPORT = '/dev/ttyUSB1'
XBEEBAUD_RATE = 9600

# The XBee addresses I'm dealing with
BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff'
UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address

#-------------------------------------------------
logging.basicConfig()

# this is the only way I could think of to get the address strings to store.
# I take the ord() to get a number, convert to hex, then take the 3 to end
# characters and pad them with zero and finally put the '0x' back on the front
# I put spaces in between each hex character to make it easier to read. This
# left an extra space at the end, so I slice it off in the return statement.
# I hope this makes it easier to grab it out of the database when needed
def addrToString(funnyAddrString):
hexified = ''
for i in funnyAddrString:
hexified += '0x' + hex(ord(i))[2:].zfill(2) + ''
return hexified[:-1]


#------------ XBee Stuff -------------------------
# Open serial port for use by the XBee
ser = serial.Serial(XBEEPORT, XBEEBAUD_RATE)

# this is a call back function. When a message
# comes in this function will get the data
def messageReceived(data):
#print 'gotta packet'
#print data
clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1])
#print 'Cluster ID:', hex(clusterId),
if (clusterId == 0x13):
# This is the device announce message.
# due to timing problems with the switch itself, I don't
# respond to this message, I save the response for later after the
# Match Descriptor request comes in. You'll see it down below.
# if you want to see the data that came in with this message, just
# uncomment the 'print data' comment up above
print 'Device Announce Message'
elif (clusterId == 0x8005):
# this is the Active Endpoint Response This message tells you
# what the device can do, but it isn't constructed correctly to match
# what the switch can do according to the spec. This is another
# message that gets it's response after I receive the Match Descriptor
print 'Active Endpoint Response'
elif (clusterId == 0x0006):
# Match Descriptor Request; this is the point where I finally
# respond to the switch. Several messages are sent to cause the
# switch to join with the controller at a network level and to cause
# it to regard this controller as valid.
#
# First the Active Endpoint Request
payload1 = '\x00\x00'
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x00\x05',
profile = '\x00\x00',
data = payload1
)
print 'sent Active Endpoint'
# Now the Match Descriptor Response
payload2 = '\x00\x00\x00\x00\x01\x02'
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x80\x06',
profile = '\x00\x00',
data = payload2
)
print 'Sent Match Descriptor'
# Now there are two messages directed at the hardware
# code (rather than the network code. The switch has to
# receive both of these to stay joined.
payload3 = '\x11\x01\x01'
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x02',
cluster = '\x00\xf6',
profile = '\xc2\x16',
data = payload2
)
payload4 = '\x19\x01\xfa\x00\x01'
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x02',
cluster = '\x00\xf0',
profile = '\xc2\x16',
data = payload4
)
print 'Sent hardware join messages'
# now that it should have joined, I'll add a record to the database to
# hold the status. I'll just name the device 'unknown' so it can
# be updated by hand using sqlite3 directly. If the device already exists,
# I'll leave the name alone and just use the existing record
# Yes, this means you'll have to go into the database and assign it a name
#
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# See if the device is already in the database
c.execute("select name from smartswitch "
"where longaddress = ?; ",
(addrToString(data['source_addr_long']),))
switchrecord = c.fetchone()
if switchrecord is not None:
print "Device %s is rejoining the network" %(switchrecord[0])
else:
print "Adding new device"
c.execute("insert into smartswitch(name,longaddress, shortaddress, status, watts, twatts, utime)"
"values (?, ?, ?, ?, ?, ?, ?);",
('unknown',
addrToString(data['source_addr_long']),
addrToString(data['source_addr']),
'unknown',
'0',
'0',
time.strftime("%A, %B, %d at %H:%M:%S")))
dbconn.commit()
dbconn.close()

elif (clusterId == 0xef):
clusterCmd = ord(data['rf_data'][2])
if (clusterCmd == 0x81):
usage = ord(data['rf_data'][3]) + (ord(data['rf_data'][4]) * 256)
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# This is commented out because I don't need the name
# unless I'm debugging.
# get device name from database
#c.execute("select name from smartswitch "
# "where longaddress = ?; ",
# (addrToString(data['source_addr_long']),))
#name = c.fetchone()[0].capitalize()
#print "%s Instaneous Power, %d Watts" %(name, usage)
# do database updates
c.execute("update smartswitch "
"set watts = ?, "
"shortaddress = ?, "
"utime = ? where longaddress = ?; ",
(usage, addrToString(data['source_addr']),
time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long'])))
dbconn.commit()
dbconn.close()
elif (clusterCmd == 0x82):
usage = (ord(data['rf_data'][3]) +
(ord(data['rf_data'][4]) * 256) +
(ord(data['rf_data'][5]) * 256 * 256) +
(ord(data['rf_data'][6]) * 256 * 256 * 256) )
upTime = (ord(data['rf_data'][7]) +
(ord(data['rf_data'][8]) * 256) +
(ord(data['rf_data'][9]) * 256 * 256) +
(ord(data['rf_data'][10]) * 256 * 256 * 256) )
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
c.execute("select name from smartswitch "
"where longaddress = ?; ",
(addrToString(data['source_addr_long']),))
name = c.fetchone()[0].capitalize()
print "%s Minute Stats: Usage, %d Watt Hours; Uptime, %d Seconds" %(name, usage/3600, upTime)
# update database stuff
c.execute("update smartswitch "
"set twatts = ?, "
"shortaddress = ?, "
"utime = ? where longaddress = ?; ",
(usage, addrToString(data['source_addr']),
time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long'])))
dbconn.commit()
dbconn.close()

elif (clusterId == 0xf0):
clusterCmd = ord(data['rf_data'][2])
# print "Cluster Cmd:", hex(clusterCmd),
# if (clusterCmd == 0xfb):
#print "Temperature ??"
# else:
#print "Unimplemented"
elif (clusterId == 0xf6):
clusterCmd = ord(data['rf_data'][2])
# if (clusterCmd == 0xfd):
# pass #print "RSSI value:", ord(data['rf_data'][3])
# elif (clusterCmd == 0xfe):
# pass #print "Version Information"
# else:
# pass #print data['rf_data']
elif (clusterId == 0xee):
clusterCmd = ord(data['rf_data'][2])
status = ''
if (clusterCmd == 0x80):
if (ord(data['rf_data'][3]) & 0x01):
status = "ON"
else:
status = "OFF"
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
c.execute("select name from smartswitch "
"where longaddress = ?; ",
(addrToString(data['source_addr_long']),))
print c.fetchone()[0].capitalize(),
print "Switch is", status
c.execute("update smartswitch "
"set status = ?, "
"shortaddress = ?, "
"utime = ? where longaddress = ?; ",
(status, addrToString(data['source_addr']),
time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long'])))
dbconn.commit()
dbconn.close()
else:
print "Unimplemented Cluster ID", hex(clusterId)
print

def sendSwitch(whereLong, whereShort, srcEndpoint, destEndpoint,
clusterId, profileId, clusterCmd, databytes):

payload = '\x11\x00' + clusterCmd + databytes
# print 'payload',
# for c in payload:
# print hex(ord(c)),
# print
# print 'long address:',
# for c in whereLong:
# print hex(ord(c)),
# print

zb.send('tx_explicit',
dest_addr_long = whereLong,
dest_addr = whereShort,
src_endpoint = srcEndpoint,
dest_endpoint = destEndpoint,
cluster = clusterId,
profile = profileId,
data = payload
)
# This just puts a time stamp in the log file for tracking
def timeInLog():
print time.strftime("%A, %B, %d at %H:%M:%S")

#------------------If you want to schedule something to happen -----
scheditem = Scheduler()
scheditem.start()

scheditem.add_interval_job(timeInLog, minutes=15)

#-----------------------------------------------------------------

# Create XBee library API object, which spawns a new thread
zb = ZigBee(ser, callback=messageReceived)

print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
while True:
try:
time.sleep(0.1)
sys.stdout.flush() # if you're running non interactive, do this

except KeyboardInterrupt:
print "Keyboard interrupt"
break
except:
print "Unexpected error:", sys.exc_info()[0]
break

print "After the while loop"
# halt() must be called before closing the serial
# port in order to ensure proper thread shutdown
zb.halt()
ser.close()

Of course, I have to update my storage out there to hold the new appliance and start charting it.  That means, for now, updating my legacy feeds on Xively and then adding the new feed to my appliance chart.  Then when I get (yet) another one of these, I'll have to make similar updates for it.  I know, I could spend a few days coming up with a way to automate this, but why bother?  It only takes an hour or so to add a new monitor device and I don't have any current plans for a new one.  Here's the latest chart, notice I can now monitor the freezer out in the garage.


Sure, the chart looks too busy to understand, but remember you can click on the legend items on the bottom and turn off whatever you don't want to look at right now.  It looks like my garage freezer is roughly equivalent to a 50W light bulb running all the time.  I guess I need to figure out what that costs me, but it takes a spreadsheet to do it.  I have seasonal rates and demand billing, that will take some thought.

So, I'll add one more device to float around the house measuring things that may catch my interest from time to time.  The kill-a-watt is cool, but you don't get a feel for how a particular device will behave over time.  The kill-a-watt didn't tell me that the freezer defrost timer was hitting during the peak period when it could just as well operate during off-peak periods.

Now I have to open the other two devices and install a jumper, have fun.

Cell Phone Charging and Power Usage.

$
0
0
One of the comments on this blog made me start wondering about the little parasitic devices we have all over the house.  I've always assumed that they drew so little power they wouldn't matter when compared to to the kilowatt guzzling motors and heating elements we have in the larger appliances.  One little device that annoys me is the cell phone charger.  Every time the phone gets to a full charge it tells me to unplug the charger to save energy.  Sheesh, leave me alone, let me worry about how much power I'm using.

However, the commenter said his measuring device recorded 15 watts for several of his devices.  If I can confirm that, I may have to think about doing something since I have a bunch of them around the house.  So, I'll take on an annoyance and see what the power usage really is for my cell phone charger.


This little phone sucks 12 watts when it first starts out, so I need a really good wall wart that can supply over two amps to get the quickest charge.  Since I used the 'genuine' Samsung charger for this test, I let the phone drain down to (approximately) the same point and tried one of those 'Samsung' chargers that are available on Ebay.


Don't let the scale confuse you, this charger never gets over 1 watt.  Since my granularity (using this monitor device) is 1 watt, it could have gone a little higher or lower and still read this way, so I plugged in an amp meter in series and watched a while and it didn't ever go over 500 mA.  It doesn't have the stair steps of the real Samsung charger and took much longer ( less than an hour compared to over 2.5).  It looks like buying a real brand name may result in much, much better performance.  However, how the heck do you tell if it's really from the manufacturer?  Both of them are labeled 'Samsung' and they are the same form factor.  The ratings on the side are the same, so how do us folk out here in the world tell the difference?

I don't have an answer to this, and I'm certainly not going to buy a few hundred different ones and try them out.  I'm seriously thinking about making a load to test these things before I try to depend on them for anything.  If they fail the test, I'll do some serious complaining to the supplier.  The other thing I'm thinking about is putting together a power supply that will give me 5V at three amps reliably.  This would be useful to see how the charge characteristics of the phone look when it has enough current available.

Just for fun, I weighed each of them.  The real Samsung charger weighed 37 grams and the other one came in at 25 grams.  The real one has 12 grams more stuff in it, or thicker plastic.  Since I plan on using the fake for other things, I'm not going to dismantle it ... yet.  But here's a picture of the two of them:


The fake is the black one.  

Just to let you folk know though, there are two differences between them. 1. The real one has a UL certification on it, the fake doesn't.  That's really easy to overcome, they simply add another certification stamp to it.  2. The fake says 5.0 volt at 2 amp, and the real one says 5.3 volt at 2 amp.  Once again, that's easy to overcome.  I've already mentioned the weight, but no supplier tells you the weight of the device.

Anyway, I started this as an investigation into parasitic power and wound up researching chargers and their capabilities.  Sigh.  At any rate, let the buyer beware.

I'll try to explain XBee API Mode and How to Set it Up

$
0
0
I've gotten hundreds of questions on XBee API mode.  API mode is necessary for setting up an XBee network of more than three devices because AT mode (also called transparent) will cause fragmentation and mixing of data over time.  I have a post somewhere about this that explains my experience when I finally exceeded what the XBee was capable of when using AT mode <link>.

So, what is API mode?  It's simply a protocol between the serial port on the XBee and your controller.  Be it Raspberry Pi, Arduino, Galileo, whatever, you talk to an XBee through a serial port and what goes over the serial port isn't what the XBees send amongst themselves.  First, a simple picture:
I'm not an artist, so use your imagination.  The term 'API mode' refers only to the two serial links that connect your controllers to the XBee.  What goes over the RF link is not affected and shouldn't be; that's something that Digi takes care of quite nicely.

In AT mode, what you send out the port is what you want to be sent to the other controller.  You want, "Hello World," that's exactly what you send.  The XBee will collect the characters and put them in an RF packet and send them to the other XBee.  If the other XBee is in AT mode as well, it will parse the characters out and send them through the serial port to the other controller.  Simple, easy to understand, and the way almost everyone starts out with the XBees.

Now a little bit about API mode.  API mode allows you to shape a packet complete with checksum, destination address, and other things that give you control over what goes on.  It also allows you confidence that what you send gets to the other end complete and in one piece.  See, AT mode will send some of it, wait, then send the rest of it.  This depends on traffic and RF noise, so it's possible to get part of a message, then the rest of it later.  If you have a number of XBees, you'll actually get messages that have been mixed together and made unusable.  You get all the stuff that was sent, but it may be jumbled up.  API mode is the way to go after you've played with AT mode some to get a feel for things.

So, how to you set up an XBee to transmit in API mode?  I only work with Series 2 and later XBees, so everything here applies to them, if you're working with series 1, you're going to have to look somewhere else for examples.  Remember that API mode only applies to the serial links; I've labeled them in the drawing, so there are two things you have to change to get API mode on the XBee: the software you load on the device, and the serial interface you choose.

I use modem setting 'XB24-ZB' and Function Set 'ZIGBEE COORDINATOR API'.  These are found in the Modem Configuration tab of XCTU up at the top.  I always choose that latest software version on the far right.  Here's a picture to help you find it:


This software, once you load it on the XBee will give you API mode.  The XBees come defaulted to AT mode, so you'll have to download XCTU and change them yourself.  Of course, if you're programming a router, choose 'ZIGBEE ROUTER API' as the 'Function Set.'

Now you have to change the various parameters to be specific for your usage.  Things like Pan ID, Channel, that kind of stuff.  I address these things in other posts, so look them up there, or use one of the jillion tutorials out there.  Once you get that stuff in, you can set which API mode you want to use.  On the Arduino using Andrew Rapp's XBee library, you'll need to use API mode 2.  For the Raspberry Pi using the XBee package, you'll need to use API mode 1.  Don't worry, they will work fine together.  Remember above where I told you API mode only affects the serial link to the controller and not the RF link between XBees?  This is the beauty of these little devices, the mode doesn't change how the XBees talk to each other, just how they talk to the controller.

API mode is set under 'Serial Interfacing' a little bit down the screen, so scroll down until you find it.  The values are (duh) 0, 1, or 2.  Zero means no API, 1 is for API mode 1, and 2 is for API mode 2.  What's the difference?  I talk about this and the interaction of the various modes here <link>.  Why do I need different API modes on the different controllers?  Because the authors of the libraries wrote them that way and we have to adjust ... or heaven forbid ... write our own code to handle it.

Write the parameters to the XBee and you have now set up the XBee to work in API mode using whichever mode ( 1 or 2 ) is appropriate.   In case you need to save this configuration, there is a profile save button on the screen.  Use it to keep the various profiles you try over time.  You can simply reload it and avoid all of this next time.

One other thing: now you have to change the settings on the first screen of XCTU to talk to it properly.


About half way down the screen there's a box 'API,' check the 'Enable API' setting, and depending on how you configured the XBee, check the 'Use escape characters' box.  In API mode 1, you don't check it; in API mode 2, you do.

Now you have to change the code you're running on the controllers to use the new mode.  It's worth it because you can check the checksum on a message to see if it got clobbered on its way to you, address specific XBees so you don't have unnecessary traffic.  Use end devices that don't have a controller attached, just use the native pins on the XBee.  It makes the little devices much more versatile.  For examples, I have posts about some of the libraries on this blog, Just type XBee in the search box on the upper left and look around.

Perils of Using Cloud Services

$
0
0
So far, I've been a fan of cloud services.  I say, "so far," because it's a great idea on the surface.  For free, or a small fee, web businesses provide storage, blogging, graphs, and many specialized services.  It seems like the perfect solution to problems that we little folk have.  Some of us want to investigate power (me), others want to share pictures, still others want long winded political diatribes, and there are sites out there that cater to us.

But (remember, there's always a 'but'), the darn providers want to constantly change what they provide and charge ever increasing amounts of money for the services.  I've talked about my disappointment with Xively (Cosm, Pachube) over the years as they got bought out by various companies and abandoned the ideals and price point they started with.  Some of the services start off with a bang and dwindle to a whimper decreasing services down to an almost unusable level.  Others unexpectedly change things and don't let you know in advance, or even after the fact ... you have to find out because things quit working (usually over the weekend), and it takes time and patience to ferret out what the heck they did.

In the last couple of weeks I've encountered this annoying activity in regards to my charts that are displayed on this site.  One day (a Monday naturally), all my charts based on Xively storage stopped working.  It took me a while to figure out what happened and fix it.  It seems that Xively stopped responding to the old Pachube URLs and didn't give any warning at all.  I checked their blog, site, twitter account and my spam folder, nothing.  So, I had to visit each use of the charts and change the word 'pachube' to 'xively' to make it work.  Not a bad thing, and it wouldn't have mattered if they dropped me a note or put something somewhere that this was going to happen.  But, it was really annoying to have it sneak up like that.

Then, the charts stopped working again.  Nope, it wasn't Xively messing around, this time it was dropbox.  They tightened up the security and forced their old http links into https.  This is normally considered a good thing, but when you have a number of embedded charts that all look at http, it is a pain to change all of them.  Especially when they didn't say a word about it.  Nope, it didn't come in the email, no mention anywhere I could find.

After chasing down the various references, I got them fixed and working again (I think).

This has happened to me before; actually several times, google dropped support for their .png charts, I had to scramble to find another way.  A cloud storage service disappeared, that's how I decided to move to dropbox.  One of the quicken services became Mint and the transition sucked for me; I just dropped that idea altogether.  Etc.

I still think Cloud Services are a good idea, but the specific implementations seem to be ridden with various problems we have to adapt to.  The biggest being some corporate know-it-all that decides to change things without communicating it.  True, it is their service and they can do what they want.  I have the choice of staying with them or leaving for something else that fits my needs or doesn't tick me off as much.

However, I also have the choice of grabbing one of the great new little computers, hanging a big ol' drive on it that I can pick up for a song these days, and rolling my own in-home cloud services.  I mean, how hard can it be to record the data I send up the wire locally?  A nice free database manager (there are dozens), a little front end code that listens to http, and away I go.  Sure, it'll expose a machine to the internet if I want to make it public (I do) and I'd have to actually invest in a real router to isolate the rest of the house, but I wouldn't be subject to the changing whims of some corporate MBA that will only hold the job for a few months before leaving for greener pastures.

I haven't decided to abandon the cloud yet, but it's getting more and more compelling all the time.

Playing with the HLK-RM04 Serial to Ethernet Convertor

$
0
0
I went on ebay and got me one of these:


This is a HLK-RM04 serial to ethernet device.  When I got it, I thought, why does it have two ethernet plugs?  Why does it have a wireless antenna plug there too?  Does it work with TTL serial like the Arduino, Raspberry Pi, and XBee do?  So, I started playing with it to see what it could do.

I found out that it is probably an incredible little device.  I couldn't get it to sign on to my wireless network using the built in web server, the darn thing would try, but it just wouldn't connect.  So, I looked around the web and noticed that it had a 'secret' web page I could play with.  A lot more messing around and I found out that the software that comes with it is incredibly buggy.  No, I'm not surprised, just disappointed.  Why do folk that come out with devices like this hate doing reasonable documentation?  Why do the leave out significant features that could sky rocket their sales to the Makers out there?  Do they need these things as something to lower their reputation?

Silly.

I managed to turn this little device into a wireless ethernet bridge that supports 802.11 b/g/n at full speed, which is a blessing to me because my old house DSL modem is much slower and starting to get a bit flaky.  Did you catch that?  I have a nice wireless interface to my network at 'n' speeds without spending a fortune.  Also, I can put one of these anywhere I have an ethernet plug and extend my wireless for a small fraction of what Linksys or Belkin would like me to pay.  It cost me less than $25 complete with wall wart and 2db antenna included. Free shipping too.

All I had to do was overcome the crappy software and disgustingly incomplete documentation.  There's even an article on how to load openwrt on the device, but I kind of want to avoid that if possible (enough eggs on the fire right now ... thank you).

First though, my device isn't exactly like the one pictured above; mine's a little different:


Notice that there's an RF shield over the components that handle the RF?  That's probably to maintain the various certifications.  It has a LAN plug as well as a WAN plug besides the two serial ports and wireless.  Talk about stuffing ten pounds in a five pound sock!  What I will eventually be able to do with this series of boards should be a lot of fun.

One of my readers suggested I use this board.  My original intent was to use the serial input to monitor my pool controller for more protocol decoding that I haven't gotten to.  He and I embarked on a project that I'll write up later, and I got really tired of wandering from the house outside with a laptop and a bunch of tools to hook up and gather some data.  The idea was that I'd put this board out there and just watch it from inside where the temperature isn't approaching 115F in the full sun.  I can't do that yet, I didn't conquer the serial input before I discovered some of it's other capabilities.

Just so you folk don't think I'm just teasing you, the 'secret' web page for the settings that the manufacturer wants to keep us from playing with is http://192.168.15.254/home.asp  See, this configurable router is complete.  It can do all the encryptions, routing, tunneling, UPNP, etc that the expensive devices can do.  Sure, it's not a gigabit device, but neither is my house wiring.  You could put this behind the DSL modem at your house and totally stop the phone company from looking around your network.  I know, that sounds paranoid, but it is the 21st century and they ARE out to get me.

So, go to the 'secret' web page on one of these devices and experiment away.  You could have a cheap working Wifi access point in about 45 minutes.  You'd have a second one in about 2 minutes; there's a little learning curve to it.  Setting it up as a bridge is really simple.  The hardest part was finding the place where I could change the SSID.  There's a couple of different places for this depending on what you configure the device to be.

Now I can replace the old DSL modem I'm using in the garage to extend my wireless as well as the flaky wireless on my connected DSL up in the attic.

I have a letter to the vendor complaining about how the serial configuration failed; maybe he can get me some better documentation or something.  Meanwhile, I may just hunt for a couple more of these little things.  The darn ethernet plugs and antenna would cost me as much as the entire assembly did.

OpenWrt on the HLK-RM04, Reality Check

$
0
0
There's a ton of pixels out there about OpenWrt.  It's open source router software that can do an excellent job of expanding the capabilities of some routers way beyond what the manufacturers provide.  I had an OpenWrt router for a while, it eventually went up in smoke and I've been using what the DSL provider recommends ever since.  However, I ran into the HLK-RM04 <link> through the suggestion of a reader and decided to look into expanding the capabilities of this little device and OpenWrt seemed like a good choice to use as software for it.

There's a few threads on the OpenWrt forum and a couple of interesting repositories on github that relate to the device and they look reasonably promising,  So, I read the threads, and all of them suggested either picking up a binary they created or building my own.  There were involved instructions on the build, so I thought, "Why not?" Thus began my misadventures with building OpenWrt for the HLK-RM04.

First, I tried the binary that was created by one of the guys that frequents the OpenWrt forum.  This binary seemed to have great possibilities since, once you have a version of OpenWrt running, you can add to it and update it pretty easily.  So I followed the instruction to install the uboot boot loader and then installed the image that the guy put in github.  This process went pretty well, but very little worked on the router.  I could get an ethernet connection, but wireless and any outside access wouldn't work.  When I tried to enable wireless, the rm04 ran out of memory.  Fine, I'll build my own and learn about the configuration of the software while going through the process.

So, I thought to myself, I really should make sure I could restore the rm04 to factory settings.  Oops, too late.  Once you install uboot on the device, the factory software is gone and can't be recovered.  Even though it's in a different part of the device, there just isn't any way to get to it.  Sure, I could have spent about a month learning the ins and outs of the JTAG interface and bit banged myself a backup, but that didn't seem like a valuable skill to pursue.  There were two reasons I was tardy in looking at a way to back it up.  The first was because there really aren't any useful instructions out there on how to backup.  Sure there's a OpenWrt <link> wiki page on it, but I couldn't get enough information out of it to even attempt anything.  The 'How To' was full of 'could', 'maybe', and 'beware', with no real examples.  Like I said, I'd have to learn the intricacies of JTAG to have a chance.  The second was because there isn't a serial interface to control the chip.  There's a serial line, but you can't log into it and do anything.  I really don't understand why manufacturers do this, do they want to sell these things?

I guess I better make OpenWrt work.

I downloaded OpenWrt, configured it the way the instructions said and started make.  Six hours later, it failed.  Following the instructions, I used the V=s option to see what was failing.  It turns out that OpenWrt installation only installs part of the code, the makefile goes to various servers and their mirrors and downloads more.  The file it failed on was a little over two meg, so I tried it again.  Of course, it failed again.  I set up a little script to restart the make if it died and went to bed.

The next morning, it was still trying to load files.  Yes, I got the two meg and several others, but the files were getting enormous.  I watched it fail five times on an eight meg file and decide this wasn't working, so I hunted down the retry count and timeout amount in the scripts and changed them to 25 retries and a full minute timeout.  Ran it again and it seemed to stumble through some of the downloads.  However, when I watched the log, it was failing quite often with a 404 error on a file that it had been previously reading; what?  So, I changed the retry to 50 to allow for server errors and let it go again.

It chugged along for many hours and then I noticed that the file size was getting larger and larger.  There were files over 90Mb that were trying to be loaded.  The internet, my little machine, and bad luck were really conspiring because at one time I noticed the load speed was two bytes (yes bytes) a second.  Obviously, this wasn't going to work, at least in the time I have left on the planet.

Time to try another tactic.  I saw three possibilities 1) Configure various things out of the binary version I had previously tried until I got the bare minimum and see what I could get running.  2) Download and install a version I ran across that said that they had already stripped it to almost nothing.  3) Try to get the original software from somewhere and put it back, then start off with some other tactic.

I did all three.  I went to the OpenWrt image I had running, but ran out of memory, and started cutting things out.  Problem with this was that when I cut something out that I shouldn't, the little board died and I had to start over cutting things out.  This got tiring, but I got really good at loading software on the board and backing up the OpenWrt config files such that it was only tedious to restore it, not a real pain.  However, I never did get enough out of the distribution of OpenWrt to allow wireless to run without it failing from running out of memory.

So I went to step 2 above and loaded a bare minimum system on the board.  This one held some promise, but I couldn't get past the configuration changes necessary to make the board do something useful.

Step three was now the remaining option.  I dropped a note to the manufacturer asking for the very latest version of the boards software.  I hoped they would send it then I could try loading it on the board with uboot to see if that would get me back to the starting point.  I also dropped a note to a person that had posted that he had the manufacturer's files and had used them.  I fully expected to get no response from either of them, ... but:

The other customer that had the files posted them to github so folk could pick them up there.  So, the last two versions of the board's OS are out there to be tried.  The manufacturer actually answered me and sent a lot of documentation that appears to be correct.  They said they would send me the files as well, but I haven't received them yet.

This was totally unexpected.  Two total strangers responded extremely rapidly and there are possible solutions that I can try.  Gathering up enough cables and wires to make a huge mess, I tried loading the factory files to the board with uboot.  No luck there, I got the horrible message, "Bad Magic Number," and the boot failed.  I tried to look into the magic number, thinking that I could change it and get the file to execute.  No such luck.  One (get this only one) person had fought the battle against this message and posted about his trials <link>.  Turns out that it wasn't just a magic number, it's a 60 byte header that includes compression type, CRC checksum, and a bunch of other stuff I didn't know and couldn't think of a way to find out.

I'm out of options.

It's not as bad as it sounds though.  Remember, I bought this board as a tool that I could use to monitor devices and possibly make changes with.  It was truly compelling because it's relatively small and cheaper than most solutions out there.  Wait, cheap?  I ordered another chip for the board.  Since the board is just interfaces and mounts, a new chip will give me a second chance to try and get everything I need working, and I have loads of experience with it now to keep from making some of the mistakes that got me into this mess.

Back when I started experimenting with the board I managed to make a reasonable 802.11 b/g/n gateway with it.  That was a nice bonus because it meant that I definitely could get some use out of it as a wifi device.  If I can get the serial port to send data over wireless, that would just be icing on the cake.  That's worth another 10-12 bucks investment.  If I'm lucky, the newer software, better documentation, and experience could lead to a nice versatile addition to my network.

The problem with OpenWrt is that it is too versatile.  Over the years many people have contributed to the project and it has grown to the point where one needs a quad processsor running at light speed to build it the first time.  Of course that also means a quantum link to the ethernet or access to the buss of the server.  If you can get it loaded and built, you've actually got something that you can do something with because the ability to update it is really nice.  It's that first time that will test your patience.  When I finally gave up, it was over 2G and still growing; allow for this in your plans.

Now I'm waiting for the new chip and holding on to the updated manufacturer's software.  I have notes on how to do things that shouldn't kill the board and understand a whole lot more about the device.

C'mon mail.

Arduino, XBee, and a Freezer Defroster

$
0
0
I have an upright freezer.  Living a ways from town and the climate make this necessary.  I load a cooler in the car, go to the store get my frozen goods and some ice, pack the food in the cooler with the ice and come home.  When it's over 114F outside, you have to take some special precautions to keep the food from becoming a puddle in the back of the car.

A while back I discusssed how my freezer uses power; it's relatively efficient except for one item, the defroster <link>.  A freezer's defroster is a combination of a timer, a heater, and a thermostat to monitor the temperature of the evaporator.  What bothered me was the timing of the defrost cycle.  Every 11 hours or so, it would defrost, and this meant that the heater would be on during peak usage period.  Since I pay a lot for peak usage it would be nice to have better control of the timing of the defrost cycle.  So, an examination of the defrost circuitry showed a clear opportunity for an accurate timer and a simple SPDT relay as a replacement for the conventional method.

So, since I had a couple of arduinos that weren't doing anything, I got one of these:
This shield has both the relays and the XBee socket I wanted to use, perfect.  I have a few XBees that I picked up, so I configured one and plugged it in.  A little code later I was reading my house clock transmissions and setting off a timer every 12 hours.  I chose 08:15 and 20:15, times that are outside the peak period, now all I had to do was wire it in and test it.  Here's an image of the schematic of the circuitry:


I circled the defrost timer to make it obvious.  Notice that it simply switches between the compressor circuitry and the defrost heating assembly.  This makes it simple to wire into the circuitry, so I took out the timer and used the plug that went to it to wire into the relay of the arduino shield.  It was ready to test, and I hooked up a wall wart power supply and plugged the arduino into the same power monitor that I use on the freezer.  It worked like a charm.  Now my freezer goes into a defrost cycle at the programmed times and runs for 30 minutes.  I checked the evaporator pretty often to make sure it was running the heater long enough and everything seems fine.

While I was programming the device I threw in some code to allow me to set off the defrost cycle any time I want to as well as having it report through the XBee the status of the defroster.  This leaves a clear opportunity for installing a temperature sensor, compressor sensor, door sensor, etc.  Over time I may well do some of these things.  I could go nuts and use the arduino to control the entire freezer; these appliances are relatively simple devices and a little arduino and some relays could take over the entire operation.  I'm not sure there's any real point to that since it works well already, but I may get bored some hot summer day.

A temperature sensor would be pretty nice though.  I could send the temp to my house controller <link> and set up a piece of code to check for the temperature getting too high.  A too-high situation could easily send me mail or light up an LED somewhere to alert me to a problem.  Or both.   

Here's a chart of my freezer power usage over a 24 hour period:


The freezer is the black line.  Notice the two humps, one at 08:15 and the other at 20:15?  That's the increased power usage from the heating units (one on the evaporator and the other on the drain tube).  Now I have this power using mode scheduled for the lower rate periods.  With the new LED lights I installed in the freezer to replace the incandescent ones, this device is getting cheaper to run all the time.

Before you tell me that the wall wart and arduino probably use 5 watts continuously, remember my goal was to move the higher usage away from peak billing periods.  I'd rather have 5 watts continuous than 400 watts for thirty minutes during peak.  Peak usage is really expensive in my area.

No, it's not a major savings, but every little bit helps.  Heck, I'll probably get back the money I spent on the shield and arduino in ... what ... 10 years or so.

Controlling the Hayward Ecostar Pump

$
0
0
Quite some time ago I got fed up with the huge amount of power a pool pump uses and got one of those cool permanent magnet, variable speed pumps.  It's really cool, it can run from a few RPM up to way over 3000.  All of the speeds use less power than a conventional motor and you can actually circulate water with 200 watts of power usage.  This is what they look like:

There's three pieces to these things, the controller, the actual motor and the filter-impeller.  Hayward calls the controller the 'drive'; I don't have any idea why except maybe because they make pool controllers and they don't want customers to get confused.  The drive is this part here:


Yes, I stole the image off the web.  It was too hot to go out and take my own picture.  The drive can be separated from the motor and remotely set up.  This makes it convenient for ease of use, and nice if you have the rest of the stuff inside an enclosure.

The drive has its own controls, but it can also be controlled by my Goldline P4 controller.  The problem is that the darn controller only supports two speeds: low and high.  Sure, I can set the speed for each of them to anything I want, but I only have two speeds for my variable speed motor.  That has always ticked me off.  Hayward decided that I only get two speeds for the motor that are easily done.  To do anything else I have to futz around with settings each time.  Really ??

A few weeks ago one of my readers asked if I had ever pursued breaking the protocol between my controller and the drive.  I hadn't, but it sounded like a cool project to take on.  So, armed with my motor, his motor, his skills, my lack of skills, we embarked upon a project to get control of this motor.  Naturally, there is zero documentation on the protocol, but we had some hints.

It's RS485 half duplex because it works on the same line as the other controls for the P4.  It's 19.2Kbaud for the same reason.  That meant that we could set up monitors and code to listen to the conversations between the devices as we changed things and might stand a chance to get a hint on how to control it.

Guess what?  After a few false starts, we beat it.  We can now control the motor and change the speed to anything we want to.  It turns out the protocol is relatively simple; you send a command to the motor periodically telling it the speed you want it to run.  If you stop sending, the motor stops.  If you send a speed of zero, the motor stops.  If you send a specific speed, the motor runs at that speed.

The motor protocol works in percentage.  If you send 100%, it will run at the top speed.  If you send 0%, it stops.  You can choose any percentage in between and the motor will go to that speed.  You can't send 1000 RPM, you have to send a percentage instead.  Still, it works quite nicely.  The protocol looks like this:

Off     0x10, 0x02, 0x0C, 0x01, 0x00, 0x00, 0x00, 0x1F, 0x10, 0x03

100% 0x10, 0x02, 0x0C, 0x01, 0x00, 0x64, 0x00, 0x83, 0x10, 0x03

45%   0x10, 0x02, 0x0C, 0x01, 0x00, 0x2D, 0x00, 0x4C, 0x10, 0x03

The 0x10, 0x02 is the start of packet indicator.  The 0x0C, 0x01 means from device 1 to device 12.  The 0x00 is just a zero, never saw it become anything else.  the 0x00 or 0x64 or 0x2D is the percentage in hex (0, 100, 45) the next two bytes are the checksum and the 0x10, 0x03 is the end of packet indicator.  So, to change the speed, change the percentage, recalculate the checksum and send it.  The motor will change speed.  You have to send this at half second intervals to keep the motor running.  Actually, I've gone up to a second and a half and still had the motor run, but better safe than sorry.

The checksum is a matter of adding the byte values and sticking them in place.  In the first packet above, 
10 + 2 + C + 1 + zeros = 1F.  Remember, it's hexadecimal and the checksum is two bytes long, so it's 0x00, 0x1F.  Similarly, the 100% packet is 10 + 2 + C + 1 + 0 + 64 = 0x00, 0x83.  The last two bytes, 0x10, 0x03 are not used in the checksum.

The code for something like this was relatively simple to put together, but it gets really complex when you add in whatever method you use to control it.  I have an XBee network that I send commands through, but most folk out there want wireless ethernet or wires.  It can be controlled from a board as simple as an Arduino (like me and my colleague), or any other device that can either provide RS485 or handle an adapter to convert TTL serial.  My setup is described here <link>, I just added code to it to handle the motor separately and turned off control of the motor in the Goldline P4 I have.

The physical connection is with two wires from your RS485 device to the motor drive unit on the communications port.  You may have to use shielded cable and connect the ground up as well; these motors are extremely noisy on both the power line and RF near the unit.  So, grab some of that shielded CAT5 cable and go for it.


I cut this right out of the installation manual for the Ecostar motor.  Pin 7 is the plus (+) side of the RS485 link and they clearly label the comm connection.  This image also shows how the connection is made to a pool controller like my P4.

If you do this to control your motor, there is something that will help you.  When you send something to the motor that it can't understand, it locks up the drive unit.  I sent many a packet that was constructed incorrectly and the drive would go nuts and not respond after that.  You have to actually power down the motor and then turn it back on to continue.  Not the most robust software in the world.

There is another protocol here that I will research in more depth later.  Not only is the 'drive' controlled by the pool controller, the motor itself is controlled by the 'drive'.  Yes, the motor itself has a protocol all it's own that the drive supplies.  That's why you can separate the drive from the motor and mount it somewhere else with some wires running between.  This is also RS485 and it runs at 1200 baud.  It has a different checksum scheme and there are bits in there that have to be figured out.  We haven't made much progress on this protocol, but we can make the motor turn on without the drive unit even being there.  This may come in handy at some point in the future when the drive dies and the warranty has expired.  Those drives cost almost as much as the entire assembly ... jerks.

There you go folk, you now have control of that expensive motor Haward has been keeping secret. That's two of their secret protocols I've put on the web now.

Maybe I should think about hiding for a while...

Interaction of Appliances

$
0
0
One of my readers (thanks Andreas) commented about how controlling the appliances could help me.  For example, not letting the compressors of the refrigerator and freezer go on at the same time could lower my peak demand.  I answered that I was a bit leery about that because it could lead to spoiled food.  But, I decided an exercise in looking at it wouldn't take too much time and maybe it would tell me if such a thing would save me money.

For those of you just stumbling upon this site, I have demand billing from my wonderful power company.  There's a long discussion of this here <link>, but suffice it to say that from noon to 7PM I carefully control the usage of power around the house to reduce the power bill.  A single mistake that lasts for 15 minutes during this period could cost me a LOT of money.  Enough to buy several little computers to control things around the house.

So, I looked at adding the usages and creating a new graph to show the total use from my two freezers and refrigerator over a period of time, but had a brainstorm along the way.  I could simply use HighCharts ability to stack graphs.  It worked, and now I have a better understanding of my usage that I want to illustrate for people out there that are considering (or doing) the same kind of thing I do.

First though, a recap of the appliances and what their power usage profile is.  Here's my refrigerator graph as an area chart:


This appliance runs for short bursts to cool down to around 38-40 degrees.  It gets the most activity, but is pretty efficient.  Now, my freezer:


This thing runs a lot.  It has a good compressor that doesn't use much power, but it runs a lot more than I expected.  Still, it doesn't actually chew up much power.  And last, the garage freezer:


This thing runs about half the time during the summer because the garage is hot.  I don't cool the garage and it's on the south side.  

The little spikes in the charts are an artifact of the way I graph the data.  Yes, I could have hunted down the problem and fixed it, but I wanted to study the data, not spend a couple of hours chasing down a bug and fixing it.  So, just ignore the spikes, they only mean I'm lazy.

And finally, the stacked composite graph:


Spikes aside, this shows that my peak usage for all three devices is less than 500 watts.  Sure, it would be more energy efficient to stop one to allow a different one to run, but it wouldn't change my peak usage much.  The way they (the power company) company calculate this is to take a moving average of 15 minutes over the entire peak period for a month and then they bill me for the highest period.  So, any 500 watt period will cost me for the entire month.  I once messed up and used a couple of kW, so I had the freedom to really eat the power for the rest of the month.  Interesting result of peak demand billing, I wonder if they realize it.

At any rate, it doesn't look like I have to worry much about the combination of appliances getting my usage out of control, but it could be useful to keep this in mind if I add an appliance, or need to modify my lifestyle in the future.  Most people don't want to put up with their A/C units being shut off during the hottest part of the day, or their pool motor not running in the afternoon or early evening when they want to use it.  Fortunately, when relatives visit me, they understand my OCD about keeping appliances shut off, and just giggle about me when they get home, so I don't have to worry about them.

See folk, I actually take your suggestions seriously and even act on them from time to time.  It may take months, but I eventually get there.  Now I still have to think about getting a temperature sensor inside the freezer (thanks badhairday).


Controlling Wemo Light Switches, Another Look

$
0
0
No, I'm not dead, and yes, I still work on electronic projects.  I've been busy with projects that are much more mundane and normal.  Things like trying to build a pad for my motorcycle trailer, fighting with my chlorine generator for the pool, and recovering from my house flooding during the monsoon rains here in AZ.  Yep, I had water come in one side of the house and go out the other.  I'm still working on the damage from that episode.

But people don't read my posts to hear about that kind of thing, they probably have worse problems around the house than I do ... I don't get snow or tornadoes.

For months now I've had Wemo light switches running.  They're a nice little device that Belkin puts out and seem to be gaining in popularity.  Heck, Home Depot sells them in the electrical section now.  However, they rely on an app on the cell phone and a cloud server.  That may not be a concern for most people, but I personally hate having things I depend on subject to the whim of a company that might just decide to start charging me for the service in the future.  That's why I put together some software to run the switches from my Raspberry Pi based house controller <link><link><link>.

However the library and toolset I used, while a great effort and really classy, was too complex and kept failing.  Let me be perfectly clear, the ouimeaux library I used was good, the problem is that the wemo switches don't always work like Universal Plug and Play (upnp) devices are supposed to.  I don't want to get into a lengthy description of upnp here, there are a bunch of those out there, but suffice it to say that the subscription capabilities of the switches fails.

So, you discover the devices, set up a subscription where the devices let you know when the state changes happen, and then sometime later, the switch quits responding.  That means sometime or other, the light doesn't turn on or off and you have to restart the process that controls them to get the switch working again.  Additionally, the ouimeaux library uses a LOT of other tools and provides features that I'm just not interested in.  It uses gevent to handle asynchronous operations, has a little web server, client software, enough stuff for a major project.  I just want to turn them on and off and look to see which way they are now.  Using something large like this to do a simple job leads to huge problems when something fails.  There's just so much there that you can go nuts chasing a problem.

You guessed it, I wrote my own.  Well, that's actually an exaggeration.  I stole some stuff and used it would be closer to the truth.  I started out by writing a handler to discover upnp devices on my network.  If you've never done that, I recommend you look into it just for fun.  I discovered my satellite receiver, my Roku, the DSL modem, all the Wemo switches, and several other devices that I didn't know had plug and play interfaces.  That was a royal pain.  I had to learn about SSDP, SOAP, multicast, XML, and the list was growing.  Shoot, all I want to do was control the darn switches, not create a dissertation on half the jargon on the web.  I did learn a lot about discovering the devices on my network, but that wasn't my goal.

That led me to a few days of searching on the web for upnp libraries.  They're out there, and some of the are really good, but they're written in something besides python.  Sure a couple  have interfaces, but the documentation went totally over my head.  There were a some in python, but a couple had disappeared off the web, and one just didn't work (the samples even had syntax errors).  What should I do?  I remembered reading somewhere that python can import modules, and that the python tool Miranda worked reasonably well to experiment with upnp devices ... hmmm.

I got the latest Miranda and combined it with my own code and came up with a way to read and control the switches that uses only the basic python libraries.  It was a challenge hooking into someone else's code that was designed to run as a standalone tool, but it worked.  That way I leveraged the huge amount of work the author of Miranda did to make my job easier.  The big bonus was that Miranda is also really good as a upnp learning tool.  Ha, I have a debugger and a controller all in one.

Since I had spent a lot of time debugging the switch operation to determine why they would quit, I didn't want to use the subscription service.  The subscription facility is asynchronous, so the switch will send an XML message over UDP to a registered URL any time the state changes (off to on); too bad it doesn't work all the time.  That means that I would have to poll the switches to see if something or someone other than my code on the controller had changed their state.  Let me elaborate on that a bit.  These are wall switches; I can walk over and push the button and turn off the lights.  Since my software can't see that, it has to ask the switch what state it is in right now.

That's fine, I can poll it every couple of seconds and see what is happening, BUT there's a problem with that.  I love APScheduler a great timer tool in python; I have it in almost everything I do, but it uses threads to control when things happen.  Using threads and database updates is a path to disaster.  When I tried my new code out, about half the time it would leave the database locked because something jerked execution away at an inopportune time.  I even updated my code and libraries to the latest version of APScheduler which allows jobs to be paused, and still had the problem.  That meant that I had to figure out how to schedule polling of the switches within a single thread.

After a few magnificent failures I came up with a pretty simple way to do it; my method is in the code below.  I may have to use this in some of the previous work I've done, it's really small and simple, so it could decrease the load on the poor little Pi.

Even though this took an inordinate amount of time and study, it's still pretty darn nice.  I haven't ran it for months, but it's held up for a short time pretty well.  I could well develop problems with sockets hanging or something, but the code is simple and straight forward, so I should be able to isolate whatever problems turn up and fix them.  I don't have eight libraries of very complex code written by someone else to step through trying to find out how something works.  It's me and Miranda here, ... well there are the system libraries, but nothing terribly exotic.

Here's the code.  It's written in my usual style: variable names that are too darn long, comments for practically every darn line, over indentation, etc.  Basically everything I need to pick it up a year from now to fix a bug or add a feature.

#! /usr/bin/python
from miranda import upnp
from miranda import msearch
from miranda import set
import sys
#from apscheduler.schedulers.blocking import BlockingScheduler
import datetime
from datetime import timedelta
from datetime import datetime
import time
import sysv_ipc
import logging
import sqlite3
import pdb #yes, I had trouble and had to use this !!
#-------------------------------------------------
# the database where I'm storing stuff
DATABASE='/home/pi/database/desert-home'

def lprint(text):
print time.strftime("%A, %B, %d at %H:%M:%S"),text
sys.stdout.flush()

def _send(action, whichone, args):
if not args:
args = {}
entry = (item for item in lightSwitches if item["name"] == whichone).next()
index =entry['index']
host_info = conn.ENUM_HOSTS[index]
device_name = 'lightswitch'
service_name = 'basicevent'
controlURL = host_info['proto'] + host_info['name']
controlURL2 = host_info['deviceList'][device_name]['services'][service_name]['controlURL']
if not controlURL.endswith('/') and not controlURL2.startswith('/'):
controlURL += '/'
controlURL += controlURL2

resp = conn.sendSOAP(
host_info['name'],
'urn:Belkin:service:basicevent:1',
controlURL,
action,
args
)
return resp

def get(whichone):
'''
Returns True if the light is on and False if not
'''
resp = _send('GetBinaryState', whichone, {})
tagValue = conn.extractSingleTag(resp, 'BinaryState')
return 'On' if tagValue == '1' else 'Off'

def handleUpdate(whichone, status):
for i in lightSwitches:
if i['name'] == whichone:
i['status'] = status
updateDatabase(whichone, status)

def on(whichone):
"""
BinaryState is set to 'Error' in the case that it was already on.
"""
resp = _send('SetBinaryState', whichone, {'BinaryState': (1, 'Boolean')})
tagValue = conn.extractSingleTag(resp, 'BinaryState')
status = 'On' if tagValue in ['1', 'Error'] else 'Off'
handleUpdate(whichone, status)
lprint("turned %s on"%(whichone))
return status

def off(whichone):
"""
BinaryState is set to 'Error' in the case that it was already off.
"""
resp = _send('SetBinaryState', whichone, {'BinaryState': (0, 'Boolean')})
tagValue = conn.extractSingleTag(resp, 'BinaryState')
status = 'Off' if tagValue in ['0', 'Error'] else 'On'
handleUpdate(whichone, status)
lprint("turned %s off"%(whichone))
return status

def doLights():
for switch in lightSwitches:
thisOne = switch['name']
updateDatabase(thisOne,get(thisOne))

def doComm():
global firstTime
#global scheditem

try:
if (firstTime):
while(True):
try:
# commands could have piled up while this was
# not running. Clear them out.
junk = Cqueue.receive(block=False, type=0)
print "purging leftover commands", str(junk)
except sysv_ipc.BusyError:
break
firstTime=False
while(True):
newCommand = Cqueue.receive(block=False, type=0)
# type=0 above means suck every message off the
# queue. If I used a number above that, I'd
# have to worry about the type in other ways.
# note, I'm reserving type 1 messages for
# test messages I may send from time to
# time. Type 2 are messages that are
# sent by the php code in the web interface.
# Type 3 are from the event handler. This is just like
# the house monitor code in that respect.
# I haven't decided on any others yet.
handleCommand(newCommand)
except sysv_ipc.BusyError:
pass # Only means there wasn't anything there


def handleCommand(command):
#lprint("" + str(command))
# the command comes in from php as something like
# ('s:17:"AcidPump, pumpOff";', 2)
# so command[0] is 's:17:"AcidPump, pumpOff'
# then split it at the " and take the second item
try:
c = str(command[0].split('\"')[1]).split(',')
except IndexError:
c = str(command[0]).split('') #this is for something I sent from another process
#lprint(c)
if (c[0] == 'OutsideLightsOn'):
outsideLightsOn()
elif (c[0] == 'OutsideLightsOff'):
outsideLightsOff()
elif (c[0] == 'fPorchToggle'):
toggle("frontporch")
elif(c[0] == 'garageToggle'):
toggle("outsidegarage")
elif (c[0] == 'cactusToggle'):
toggle("cactusspot")
elif (c[0] == 'patioToggle'):
toggle("patio")
else:
lprint(" Weird command = " + str(c))

def outsideLightsOn():
lprint (" Outside lights on")
on("outsidegarage")
on("frontporch")
on("cactusspot")

def outsideLightsOff():
lprint (" Outside lights off")
off("outsidegarage")
off("frontporch")
off("cactusspot")

def toggle(whichOne):
if (get(whichOne) == 'On'):
off(whichOne)
else:
on(whichOne)

def keepAlive():
'''
For my own purposes, I update the database periodically with the time
so I can check to see if things are holding together. I currently use the
time in the light switch records for this.
'''
lprint(" keep alive")
for switch in lightSwitches:
thisOne = switch['name']
updateDatabase(thisOne, get(thisOne), force=True)

def updateDatabase(whichone, status, force=False):
'''
This is running on a Pi and is not event driven, so polling like
this will result in considerable wear to the SD card. So, I'm going to
read the database to see if it needs to be changed before I change it.
According to everything I've read, reads are free, it's the writes that
eat up the card.
'''
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
c.execute("select status from lights where name = ?;",
(whichone,))
oldstatus = c.fetchone()
if oldstatus[0] != status or force == True:
lprint ("Had to update database %s, %s"%(whichone, status))
c.execute("update lights "
"set status = ?, utime = ? where name = ?;",
(status, time.strftime("%A, %B, %d at %H:%M:%S"), whichone))
dbconn.commit()
dbconn.close()

if __name__ == "__main__":
#When looking at a log, this will tell me when it is restarted
lprint (" started")
firstTime = True
debug = False
if not debug:
conn = upnp(False,False,None,0)
'''
I don't want the search for devices to run forever
So, I set the timeout for miranda to some number of seconds
to limit it.
'''
set(3, ["set","timeout", "10"], conn)
'''
This looks at the devices that responded and gathers more data about
them by sending them a request to itemize their capabilities.

Sometimes a upnp device goes nuts and responds way out of
proportion. You can get the same device in the tables
many times, so set the uniq to True

Also, the Wemo switches don't always respond to a discover specific to
them. That means I have to do a general discover and get all the devices
on the network. This sucks because it slows things down, so if anyone
overcomes this problem, let me know how.
'''
set(3, ["set","uniq", True], conn)
''' This is the actual search '''
msearch(1,[msearch],conn)
''' and now do the interaction '''
for index, hostInfo in conn.ENUM_HOSTS.iteritems():
#print "************** ", index, " of ", len(conn.ENUM_HOSTS) - 1
''' on my network, I have a rogue device that reports badly '''
if hostInfo['name'].find('192.168.16.254') == 0:
print "Odd device, ignoring"
continue
''' if you want to see them as they come in, uncomment this '''
#print hostInfo
if hostInfo['dataComplete'] == False:
xmlHeaders, xmlData = conn.getXML(hostInfo['xmlFile'])
conn.getHostInfo(xmlData,xmlHeaders,index)

'''
now to select only the light switches from the various devices
that responded
'''
lightSwitches=[]
for index, host_info in conn.ENUM_HOSTS.iteritems():
if "deviceList" in host_info:
if "lightswitch" in host_info["deviceList"]:
name = host_info["deviceList"]["lightswitch"]["friendlyName"]
lightSwitches.append({"name": name, "index": index, "status" : 'unknown'})
'''
OK, now I have the list of Wemo light switches that are around the
house, so print it and show the state of each one
'''
print "this is the list of the", len(lightSwitches), "Wemo switches found."
for switch in lightSwitches:
switch['status'] = get(switch['name'])
print switch

# Create the message queue where commands can be read
# I just chose an identifier of 13 because the house monitor
# already took the number 12.
Cqueue = sysv_ipc.MessageQueue(13, sysv_ipc.IPC_CREAT,mode=0666)
'''
This is a poor man's timer for task control. I may put this in a class
after I've run it for a while. The reason I did it this way is that
APSscheduler creates a separate thread to handle timers and I don't
want the contention to the database of separate threads doing things
that way.

To use it, just put another entry into the table.
'''
lprint (" Setting up timed items")
startTime = int(time.time())
tasks = [{'whichOne': doLights,'interval' : 2, 'next': startTime+10},
{'whichOne': keepAlive, 'interval' : (4*60), 'next' : startTime+15}]
lprint ("going into the processing loop")
print startTime
while True:

#pdb.set_trace()
now = int(time.time())
for task in tasks:
if task['next'] <= now:
task['next'] = now + task['interval']
#print task['whichOne']
task['whichOne']()
doComm()
'''
doing a sleep here releases the cpu for longer than the program runs
That way I reduce the load on the machine so it can do more stuff
'''
time.sleep(0.25)
pass

sys.exit("done");


It's only fair to thank Ian McCracken who wrote the Ouimeaux library, Isaac Kelly who did much of the initial discovery work, and Craig Heffner the author of Miranda.  My work is built on theirs.

No, I don't have support in here for the Wemo motion sensor, crock pot, or other items they may come up with.  If you want to grab it and extend it to support those devices, feel free; just let me know so I can get the improvements as well.

There's also support in here for the sysv messages I use to communicate between processes on the machine.  Feel free to rip that out and put in anything you want.  I wanted to give you an example that I'm actually running right now, not something I cut up to look pretty.

Just for fun, here's the output of the code that is captured to my log:

Sunday, October, 05 at 16:22:06  started
action timeout
action uniq
Show unique hosts set to: True
argc 1
argv [<function msearch at 0xb69d94f0>]
Entering discovery mode for 'upnp:rootdevice', Ctl+C to stop...

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.34:49312
XML file is located at http://192.168.0.34:49312/device.xml
Device is running Linux/2.6.37.6-4.0, UPnP/1.0, Portable SDK for UPnP devices/1.
6.17
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.34:49200
XML file is located at http://192.168.0.34:49200/device.xml
Device is running Linux/2.6.37.6-4.0, UPnP/1.0, Portable SDK for UPnP devices/1.
6.17
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.28:49154
XML file is located at http://192.168.0.28:49154/setup.xml
Device is running Unspecified, UPnP/1.0, Unspecified
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.29:49153
XML file is located at http://192.168.0.29:49153/setup.xml
Device is running Unspecified, UPnP/1.0, Unspecified
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.43:49154
XML file is located at http://192.168.0.43:49154/setup.xml
Device is running Unspecified, UPnP/1.0, Unspecified
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.26:49153
XML file is located at http://192.168.0.26:49153/setup.xml
Device is running Unspecified, UPnP/1.0, Unspecified
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.22:2869
XML file is located at http://192.168.0.22:2869/upnphost/udhisapi.dll?content=uu
id:3a682b82-e803-4105-bfeb-114ed775cab1
Device is running Microsoft-Windows/6.3 UPnP/1.0 UPnP-Device-Host/1.0
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.14:8060
XML file is located at http://192.168.0.14:8060/
Device is running Roku UPnP/1.0 MiniUPnPd/1.4
****************************************************************

Error updating command completer structure; some command completion features mig
ht not work...
****************************************************************
SSDP reply message from 192.168.0.1:80
XML file is located at http://192.168.0.1:80/DeviceDescription.xml
Device is running ZyXEL-RomPlug/4.51 UPnP/1.0 IGD/1.00
****************************************************************


Discover mode halted...
Failed to get argument name for OpenInstaAP
Failed to get argument name for CloseInstaAP
Failed to get argument name for OpenInstaAP
Failed to get argument name for CloseInstaAP
Failed to get argument name for OpenInstaAP
Failed to get argument name for CloseInstaAP
Failed to get argument name for OpenInstaAP
Failed to get argument name for CloseInstaAP
Caught exception while parsing device service list: list index out of range
At index 7
Caught exception while parsing device service list: list index out of range
At index 8
this is the list of the 4 Wemo switches found.
{'status': 'Off', 'index': 2, 'name': 'cactusspot'}
{'status': 'Off', 'index': 3, 'name': 'frontporch'}
{'status': 'Off', 'index': 4, 'name': 'patio'}
{'status': 'Off', 'index': 5, 'name': 'outsidegarage'}
Sunday, October, 05 at 16:22:42 Setting up timed items
Sunday, October, 05 at 16:22:42 going into the processing loop
1412551362
Sunday, October, 05 at 16:22:56 Had to update database patio, On
Sunday, October, 05 at 16:22:56 turned patio on
Sunday, October, 05 at 16:22:57 keep alive
Sunday, October, 05 at 16:22:57 Had to update database cactusspot, Off
Sunday, October, 05 at 16:22:57 Had to update database frontporch, Off
Sunday, October, 05 at 16:22:57 Had to update database patio, On
Sunday, October, 05 at 16:22:58 Had to update database outsidegarage, Off
Sunday, October, 05 at 16:23:04 Had to update database patio, Off
Sunday, October, 05 at 16:23:04 turned patio off
Sunday, October, 05 at 16:23:12 Outside lights on
Sunday, October, 05 at 16:23:12 Had to update database outsidegarage, On
Sunday, October, 05 at 16:23:12 turned outsidegarage on
Sunday, October, 05 at 16:23:12 Had to update database frontporch, On
Sunday, October, 05 at 16:23:12 turned frontporch on
Sunday, October, 05 at 16:23:12 Had to update database cactusspot, On
Sunday, October, 05 at 16:23:12 turned cactusspot on
Sunday, October, 05 at 16:23:17 Outside lights off
Sunday, October, 05 at 16:23:17 Had to update database outsidegarage, Off
Sunday, October, 05 at 16:23:17 turned outsidegarage off
Sunday, October, 05 at 16:23:17 Had to update database frontporch, Off
Sunday, October, 05 at 16:23:17 turned frontporch off
Sunday, October, 05 at 16:23:17 Had to update database cactusspot, Off
Sunday, October, 05 at 16:23:17 turned cactusspot off

At the top is where I set some parameters in Miranda to control what I discover.  Then the discovery process prints out a summary of the devices it finds. The errors there are caused by the Wemo switches not responding properly.  Next are messages from the conversation with the switches.  Once again, the switches don't behave.  Finally the last step of initialization where the status and names of the switches show up.  The rest of the log is normal operation.  I poll the switches every two seconds to see if they're on or off. and every four minutes to make sure the database is kept current.  I use database currency to monitor the health of the controller.

If you have these switches, or are thinking about using them, grab the code and start playing.

My Tiny Timer Class

$
0
0
Just yesterday I posted about creating a new way of controlling and monitoring my Wemo light switches.  In the post I talked about having to develop a way of timing operations.  It was pretty simple and has worked well overnight, so I decided to put it in a python class and use it in the other parts of my system.

So you understand, I hate classes.  When c++ came around, and object oriented programming was all the rage, I resisted.  No, it wasn't that I couldn't accept change, it was that the crap they produced was incredibly hard to read.  Four pages of class definitions and sixteen lines of code may have been cool, but just try and debug it.  Especially with the tools available back then.  Well, it's been some years and objects have matured as well as the folk that created the mess back then.  Now, they're not as big a mess, maybe I should join the 21st century.  Sure, I use classes all the time; I just avoid writing them if at all possible; this is an exception.

So, here's my basic timer class.  Simple right?  Remember, the reason I did this was to keep control within a single thread thus avoiding deadlocks in code and locked up databases.

import time

class timer:
_things = []

def __init__(self, callback, seconds=1, minutes=0, hours=0):
interval = (hours*60*60) + (minutes*60) + seconds
actionTime = time.time() + interval
self._things.append({"callback":callback,"interval":interval,"actionTime":actionTime})

def tick(self):
now = time.time()
for i in self._things:
if i["callback"] == None:
continue
if now >= i["actionTime"]:
i["callback"]()
i["actionTime"] += i["interval"]

checkTimer = timer(None)

''' This is code to test and illustrate the usage '''
def printSecond():
print "second"

def printTwoSeconds():
print "twoseconds"

def printMinute():
print "minute"

if __name__ == "__main__":
# First create any timers you need
oneSecond = timer(printSecond, seconds=1)
twoSeconds = timer(printTwoSeconds, seconds=2)
minute = timer(printMinute, minutes=1)

# now once in a while call tick to let them happen
while True:
checkTimer.tick()
# a sleep lets the cpu have a rest to do other things.
time.sleep(0.5)

It just keeps a list of the various timers I create to do things and steps through them to see if something needs to be done.  I included an instance by default so, when I use it, I don't have to create one.  It's easy to use and I may be extending it to handle other things that may need to be timed differently like: Every day at 3:00pm, feed the fish.  That takes more work parsing various parameters and such, but it's easy to extend.

Within the module is an example of how to use it.  This can be directly run under python, to see how it works, and how to use it.

I've already modified my module for the Wemo switches to use it, now I'm looking at the other modules to see if this would be appropriate there as well.  

Using an 'rc' File in My System

$
0
0
First though, I've been away from my house control system for a while because of other projects, and I decided to do some major updates to portions of it.  Over the months, I've learned a lot and some of that new knowledge needs to be put into controlling the house.  So, yes, I'm posting more often.

Often times, I want to post a new update to one of the processes, but I have keys and other stuff in the modules that have to be removed before I can put it on the web.  I'm also going to move the code into GitHub at some point so folk don't have to copy and paste things for their own use.  They can just grab what they want from there.  That means I have to get the keys out of the code.

Unix has long had a customary way of doing things like this, an rc file.  The 'rc' stands for run command and has a long history, look it up.  There's a lot of files out that go into the user's home directory and have a name like .bashrc, .cshrc, etc.  These are configuration commands and aliases.  This is just the tool I need to use.  I can put an rc file in the home directory and all the keys, database names, etc can go in there to be read at runtime by the applications.  Cool, that way I can make something public without having to worry about somebody updating my data feeds by mistake.

But, I hate parsing data out of a file.  I was looking at parsers and ways of storing data when I got an email about decoding a JSON file.  While I was answering the mail I realized how easy it is in python to parse through JSON, so you guessed it, my rc file is JSON.  That means that it's a tiny bit harder to create and incredibly easy to use in a program.  Here's my .houserc file (with the secrets removed):

{
"database":"/home/pi/database/database",

"xbeeport": "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QLZ1-if00-port0",
"zigbeeport":"/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QQ2F-if00-port0",

"oldxively":{
"key":"secretkeythingieforlegacyxivelygoeshere",
"feed":"1234"},

"xively":{
"key":"secretkeythingieforthenewinterfacehere",
"feed":"1234567890"},

"emoncms":{
"key":"theyuseashorterkey"},

"grovestreams":{
"org":"grovestream-has-even-stranger-one",
"apiKey":"with-a-strange-key-as-well"},

"thingspeak":{
"key":"ASIMPLEKEYEASYTOUSE"},

"oreo":"double stuff"
}


It's just a simple JSON string that can be easily taken apart.  By using meaningful (to me anyway) names, it's pretty easy to read and edit as well.  To get the data back out in a python program, all you have to do is:

def getHouseValues():
json_data=open("/home/pi/.houserc").read()
return json.loads(json_data)
hv = getHouseValues()
FEED_ID = hv["oldxively"]["feed"]
API_KEY = hv["oldxively"]["key"]
# the database where I'm storing stuff
cookie = hv["oreo"]

And, since I don't want to type the directory name over and over again, I put the getHouseValues() in a python file and imported it.

No parsing, no silly obscure things to learn, just things I already had to work with doing this project.  Admittedly, it was a pain going to each module and modifying it to use the rc file, but it was worth it.  Notice I included the weird USB ports I use for the two XBees I have attached?  I can do that with other things that may get changed from time to time.  Just add them to the JSON string and I'll be able to get them back at run-time without changing code in any module.

Heck, it's not even hard to read.

OK, Back to the ZigBee protocol and XBees ... AGAIN

$
0
0
I managed to hack into the Iris Smart Switch from Lowe's and they've been working fine, but there's always been this nagging little annoyance bothering me.  The Alertme Switch that Lowe's sells is NOT ZigBee compliant no matter what they may tell you.  In hacking at it I pointed out a few things that were not according to spec (yes, I've actually read that massive spec related to home automation), and I've been wondering what it would be like to work with an actual ZigBee device.

A reader set me up with one of these:


This is a Centralite 4256050-ZHAC and has an impressive set of capabilities.  They claim that it will work with any controller that is compliant with the ZigBee HA (Home Automation) specification.  That sounds like a challenge to me.  If it is compliant, I should be able to figure out how to work it using an XBee; so away I went.

After almost six DAYS of poking messages at the switch, I was only a tiny bit further along than I was when I started.  This silly thing simply wouldn't join with the XBee so I could see anything it did.  Then I stumbled across a note that said it had a special key.  Key?  It needs a key?  OK, I can try this.  It started working.  I was able to send something and get an answer back; sure the answer was an error, but it was an answer.  Thus began my exploration of the ZigBee protocol in earnest; I was going to make this switch work.

Once again, this isn't one of those posts where I tried for weeks and finally gave up because there just wasn't enough information out there, the machine was too slow, or someone kept something secret; I made it work and will give you the code and XBee configuration to follow in my footsteps down below.  But first I want to talk about the ZigBee protocol and its relationship to XBees a bit.

First, this is an incredibly complex protocol and not for the faint of heart.  Just learning some of the jargon is daunting, much less trying to put it to use.  Sure, there are libraries out there, but have you looked at the prices of those things?  I simply can't afford to mess with them at that price.  Also, the libraries are as hard to understand as the protocol, AND it has the overhead of the library that has to be learned also.  I kept wondering if the XBee somehow could help with this.  Turns out the XBee really can do ZigBee, there just aren't may people that have tried.  Actually, I couldn't find anyone besides me that actually had.

There are lots of pictures and explanations out there about the ideas behind ZigBee, and some of them are even correct, but it was still hard for me to understand.  Let me give you some basics.  The protocol has profiles, these are collections of specs and suggestions for the operation of a system.  Things like Home Automation, Electric Lights (that Hue thingie), Smart Energy, and a device can support one or more of these things.  I'm interested in Home Automation, they call it HA, and that's where I concentrated.  Within this they separate data that you read or change and call them attributes.  These attributes are put within clusters.  Don't get confused, this isn't really that tough.

Within the HA profile, there are some defined clusters and they have numbers that identify them.  Let's take cluster 0x0006, the on-off cluster.  This will have an attribute, the state of the device, and it is numbered 0x0000 and has a datatype of boolean; it tells you if the switch is on or off.  To read this attribute you send a command to the cluster asking for it and the cluster returns the number identifier, datatype and value of the attribute.  See, clusters have commands to operate on them and attributes that you can use.

To tell if the switch is on, send a cluster command 0x00 (read attribute) to cluster 0x0006 (on/off) and the device will send back a command 0x01 (read attribute response) to cluster 0x0006 with the attribute identifier, datatype, value.  Cool.

In the message you send, you also specify the endpoint you want the reply to be sent to and the endpoint you are sending to.  What's an endpoint?  An endpoint is simply a collection of clusters.  On the centralite switch, it has endpoints 0, the general one, and 1, specific to this device.  The general endpoint is where stuff that you want to deal with of a general nature goes and endpoint 1 is where you send stuff that deals with turning the light on and off.

Thus, you have to worry about profiles, endpoints, clusters, clusters commands, and attributes.  Actually it's not that bad, it's just hard to ferret out of the thousands of pages of documentation.  But, you ask, how does the XBee help me?  The XBee eliminates about half of the documentation from being necessary for us to mess with.  It handles all the interactions to set up a network, keep track of device routing, radio initialization, that stuff.  It also gives us a simpler (not simple) message format to use so we don't have to worry about the six or seven layers of protocol, we work totally at the application level and just let it do the rest.  Heck, it even handles the encryption for us.

Combine an XBee to handle the low level link stuff and our own code to handle the application, and you have a reasonable way of controlling these switches that are on the market.  Let me show you the command above in python:

zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x00'+'\xaa'+'\x00'+'\x00'+'\x00'
)

There's the long address, it's 32 bits long and doesn't change, ever.  The short address, it's 16 bits long and changes every time the switch joins with the controller; yes, even after a power failure.  The source endpoint.  This is zero because I didn't want to deal with more than one in my code; all the responses come back to endpoint zero.  The destination endpoint which is one on this switch. The cluster id of six as I mentioned above.  The profile 0x0104 which is the number for the HA protocol. And, some data.  The data is one byte of control bits, a byte transaction sequence number that I set to 0xaa so it would be easy recognize, the cluster command 0x00, and the attribute id of 0x0000.  The reason it is shown as ascii characters is a characteristic of the python XBee library implementation.

This may be confusing at first, but trust me, it actually makes sense once you get into it a bit.

This message will send a response to profile id 0x104, endpoint 00, cluster 0x0006, with a payload of 0x00, 0x00, 0x10, 01 if the light is on.  The first two bytes are the attribute id, the next byte is the datatype (0x10 means boolean) and the last byte is 1, meaning the switch is closed.

Are you getting an idea of how this works?  Now, I can hear you asking, "How the heck do I find out these values?" They're documented in the Cluster Specification document, and there are messages that will itemize the endpoints and clusters within them that the device supports.  So, you send a message to the device to get the endpoints, it tells you what they are, then for each endpoint you ask what the attributes are and it responds.  You look at this stuff, see what you need and use it.

Actually makes sense in a deranged computer scientist sort of way.  But, let's talk about the setup for an XBee specifically to support the Home Automation profile.  That's what I wanted, to be able to turn this switch on and off.  First, it's different from the setup used on the Iris switch so don't think about that, take this as new.

Software Zigbee API Coordinator
Zigbee Stack Profile  (ZS) 2
Encryption Enable  (EE) 1
Encryption Options (EO) 0
Encryption Key  (KY) 5a6967426565416c6c69616e63653039
Network Encryption Key (NK) 0
API Enable (AP) 2
API Output Mode (AO) 3

Yes, you have to use the key.  That part took me the first week of messing with this to find out.  Of course, now that I know what to look for, it would take me about a minute to get it, but that's how we learn.  The difference in the encryption setup is what prevents this switch and the Iris switch from working with the same controller.  You can't have it both ways at once.  If anyone thinks of a way around this, let me know.

Once you have the XBee setup like this you can take the Centralite switch, press the button and hold it, then plug it in the wall.  When the led turns on, let go of the switch and it will join with the XBee automatically.  Yes, that's all there is to joining.  The two devices take care of it themselves and all you have to do is discover the device and start using it.  This is very different from the Alertme devices where we have to mess around with special secret commands to get it to work.  This device actually complies with the specification.

In the code below, I send a message asking for route information and grab the switch's address out of the response.  Then, I send it a command to set up reporting for the light and just wait for someone to tell the code what to do with the light. The commands are:

0 - Turn the switch off
1 - Turn the switch on
2 - Toggle the switch
3 - Dim the switch
4 - Brighten the switch
5 - Tell me the status of the switch
6 - Send messages and print responses about the switch.

Yes, the switch is capable of dimming a light.  The last command goes through a portion of the Zigbee discovery process to find out which endpoints are supported and what clusters and attributes are in them.  It's voluminous, but it's the first time I was actually able to see what the various buzz words actually represented.  This is the kind of thing I did to conquer the way the switch works.

#! /usr/bin/python

'''
This is an examination of a REAL ZigBee device. The CentraLite 4256050-ZHAC

It has an impressive array of capabilities that I don't delve into in depth in
this examination, but it responds properly to the various ZigBee commands and holds
the clusters necessary to control a switch.

Nice little device
'''

# This is the super secret home automation key that is needed to
# implement the HA profile.
# KY parameter on XBee = 5a6967426565416c6c69616e63653039
# Have fun

from xbee import ZigBee
import logging
import datetime
import time
import serial
import sys, traceback
import shlex
from struct import *
'''
Before we get started there's a piece of this that drove me nuts. Each message to a
Zigbee cluster has a transaction sequence number and a header. The transaction sequence
number isn't talked about at all in the Zigbee documentation (that I could find) and
the header byte is drawn backwards to everything I've ever dealt with. So, I redrew
the header byte so I could understand and use it:

7 6 5 4 3 2 1 0
X Disable Default Response 1 = don't return default message
X Direction 1 = server to client, 0 = client to server
X Manufacturer Specific
X Frame Type 1 = cluster specific, 0 = entire profile

So, to send a cluster command, set bit zero. If you want to be sure you get a reply, clearthe default response.  I haven't needed the manufacturer specific bit yet.
'''
switchLongAddr = '12'
switchShortAddr = '12'

'''
This routine will print the data received so you can follow along if necessary
'''
def printData(data):
for d in data:
print d, ' : ',
for e in data[d]:
print "{0:02x}".format(ord(e)),
if (d =='id'):
print "({})".format(data[d]),
print

def getAttributes(data, thisOne):
''' OK, now that I've listed the clusters, I'm going to see about
getting the attributes for one of them by sending a Discover
attributes command. This is not a ZDO command, it's a ZCL command.
ZDO = ZigBee device object - the actual device
ZCL = Zigbee cluster - the collection of routines to control it.

frame control bits = 0b00 (this means a BINARY 00)
manufacturer specific bit = 0, for normal, or one for manufacturer
So, the frame control will be 0000000
discover attributes command identifier = 0x0c

then a zero to indicate the first attribute to be returned
and a 0x0f to indicate the maximum number of attributes to
return.
'''
print "Sending Discover Attributes, Cluster:", repr(thisOne)
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = thisOne, # cluster I want to know about
profile = '\x01\x04', # home automation profile
# means: frame control 0, sequence number 0xaa, command 0c,
# start at 0x0000 for a length of 0x0f
data = '\x00' + '\xaa' + '\x0c'+ '\x00' + '\x00'+ '\x0f'
)

# this is a call back function. When a message
# comes in this function will get the data
def messageReceived(data):
global switchLongAddr
global switchShortAddr

try:
#print 'gotta packet',
#printData(data) # uncomment this to see the data returned

# Since this is a test program, it will only support one switch
# go get the long and short address out of the incoming packet
# for more than one switch, this won't work
switchLongAddr = data['source_addr_long']
switchShortAddr = data['source_addr']

if (data['id'] == 'rx_explicit'):
#print "RF Explicit"
#printData(data)
clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1])
print 'Cluster ID:', hex(clusterId),
print "profile id:", repr(data['profile']),
if (data['profile']=='\x01\x04'): # Home Automation Profile
# This has to be handled differently than the general profile
# each response if from a cluster that is actually doing something
# so there are attributes and commands to think about.
#
# Since this is a ZCL message; which actually means this message is
# is supposed to use the ZigBee cluster library to actually do something
# like turn on a light or check to see if it's on, the command way down
# in the rf_data is important. So, the commands may be repeated in
# each cluster and do slightly different things
#
# I'm going to grab the cluster command out of the rf_data first so
# I don't have to code it into each cluster
#print "take this apart"
#print repr(data['rf_data'])
if (data['rf_data'][0] == '\x08'): # was it successful?
#should have a bit check to see if manufacturer data is here
cCommand = data['rf_data'][2]
print "Cluster command: ", hex(ord(cCommand))
else:
print "Cluster command failed"
return
# grab the payload data to make it easier to work with
payload = data['rf_data'][3:] #from index 3 on is the payload for the command
datatypes={'\x00':'no data',
'\x10':'boolean',
'\x18':'8 bit bitmap',
'\x20':'unsigned 8 bit integer',
'\x21':'unsigned 24 bit integer',
'\x30':'8 bit enumeration',
'\x42':'character string'}
#print "Raw payload:",repr(payload)
# handle these first commands in a general way
if (cCommand == '\x0d'): # Discover Attributes
# This tells you all the attributes for a particular cluster
# and their datatypes
print "Discover attributes response"
if (payload[0] == '\x01'):
print "All attributes returned"
else:
print "Didn't get all the attributes on one try"
i = 1
if (len(payload) == 1): # no actual attributes returned
print "No attributes"
return
while (i < len(payload)-1):
print " Attribute = ", hex(ord(payload[i+1])) , hex(ord(payload[i])),
try:
print datatypes[payload[i+2]]
i += 3
except:
print "I don't have an entry for datatype:", hex(ord(payload[i+2]))
return

if (clusterId == 0x0000): # Under HA this is the 'Basic' Cluster
pass
elif (clusterId == 0x0003): # 'identify' should make it flash a light or something
pass
elif (clusterId == 0x0004): # 'Groups'
pass
elif (clusterId == 0x0005): # 'Scenes'
pass
elif (clusterId == 0x0006): # 'On/Off' this is for switching or checking on and off
#print "inside cluster 6"
if cCommand in ['\x0a','\x01']:
# The very last byte tells me if the light is on.
if (payload[-1] == '\x00'):
print "Light is OFF"
else:
print "Light is ON"
pass
elif (clusterId == 0x0008): # 'Level'
pass
else:
print("Haven't implemented this yet")
elif (data['profile']=='\x00\x00'): # The General Profile
if (clusterId == 0x0000):
print ("Network (16-bit) Address Request")
#printData(data)
elif (clusterId == 0x0008):
# I couldn't find a definition for this
print("This was probably sent to the wrong profile")
elif (clusterId == 0x0004):
# Simple Descriptor Request,
print("Simple Descriptor Request")
print("I don't respond to this")
#printData(data)
elif (clusterId == 0x0013):
# This is the device announce message.
print 'Device Announce Message'
#printData(data)
# This is a newly found device, so I'm going to tell it to
# report changes to the switch. There are better ways of
# doing this, but this is a test and demonstration
print "sending 'configure reporting'"
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x00' + '\xaa' + '\x06' + '\x00' + '\x00' + '\x00' + '\x10' + '\x00' + '\x00' + '\x00' + '\x40' + '\x00' + '\x00'
)
elif (clusterId == 0x8000):
print("Network (16-bit) Address Response")
#printData(data)
elif (clusterId == 0x8032):
print "Route Record Response"
elif (clusterId == 0x8038):
print("Management Network Update Request");
elif (clusterId == 0x8005):
# this is the Active Endpoint Response This message tells you
# what the device can do
print 'Active Endpoint Response'
printData(data)
if (ord(data['rf_data'][1]) == 0): # this means success
print "Active Endpoint reported back is: {0:02x}".format(ord(data['rf_data'][5]))
print("Now trying simple descriptor request on endpoint 01")
zb.send('tx_explicit',
dest_addr_long = data['source_addr_long'],
dest_addr = data['source_addr'],
src_endpoint = '\x00',
dest_endpoint = '\x00', # This has to go to endpoint 0 !
cluster = '\x00\x04', #simple descriptor request'
profile = '\x00\x00',
data = '\x13' + data['source_addr'][1] + data['source_addr'][0] + '\x01'
)
elif (clusterId == 0x8004):
print "simple descriptor response"
try:
clustersFound = []
r = data['rf_data']
if (ord(r[1]) == 0): # means success
#take apart the simple descriptor returned
endpoint, profileId, deviceId, version, inCount = \
unpack('<BHHBB',r[5:12])
print " endpoint reported is: {0:02x}".format(endpoint)
print " profile id: {0:04x}".format(profileId)
print " device id: {0:04x}".format(deviceId)
print " device version: {0:02x}".format(version)
print " input cluster count: {0:02x}".format(inCount)
position = 12
# input cluster list (16 bit words)
for x in range (0,inCount):
thisOne, = unpack("<H",r[position : position+2])
clustersFound.append(r[position+1] + r[position])
position += 2
print " input cluster {0:04x}".format(thisOne)
outCount, = unpack("<B",r[position])
position += 1
print " output cluster count: {0:02x}".format(outCount)
#output cluster list (16 bit words)
for x in range (0,outCount):
thisOne, = unpack("<H",r[position : position+2])
clustersFound.append(r[position+1] + r[position])
position += 2
print " output cluster {0:04x}".format(thisOne)
clustersFound.append('\x0b\x04')
print "added special cluster"
print "Completed Cluster List"
except:
print "error parsing Simple Descriptor"
printData(data)
print repr(clustersFound)
for c in clustersFound:
getAttributes(data, c) # Now, go get the attribute list for the cluster
elif (clusterId == 0x0006):
#print "Match Descriptor Request"
# Match Descriptor Request
#printData(data)
pass
else:
print ("Unimplemented Cluster ID", hex(clusterId))
print
else:
print ("Unimplemented Profile ID")
elif(data['id'] == 'route_record_indicator'):
#print("Route Record Indicator")
pass
else:
print("some other type of packet")
print(data)
except:
print "I didn't expect this error:", sys.exc_info()[0]
traceback.print_exc()

if __name__ == "__main__":
#------------ XBee Stuff -------------------------
# this is the /dev/serial/by-id device for the USB card that holds the XBee
ZIGBEEPORT = "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A600eDiR-if00-port0"
ZIGBEEBAUD_RATE = 9600
# Open serial port for use by the XBee
ser = serial.Serial(ZIGBEEPORT, ZIGBEEBAUD_RATE)


# The XBee addresses I'm dealing with
BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff'
UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address

#-------------------------------------------------
logging.basicConfig()


# Create XBee library API object, which spawns a new thread
zb = ZigBee(ser, callback=messageReceived)
print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
notYet = True;
firstTime = True;
while True:
try:
if (firstTime):
print("Wait while I locate the device")
time.sleep(1)
# First send a route record request so when the switch responds
# I can get the addresses out of it
print "Broadcasting route record request "
zb.send('tx_explicit',
dest_addr_long = BROADCAST,
dest_addr = UNKNOWN,
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x00\x32',
profile = '\x00\x00',
data = '\x12'+'\x01'
)
# if the device is already properly joined, ten seconds should be
# enough time for it to have responded. So, configure it to
# report that light has changed state.
# If it hasn't joined, this will be ignored.
time.sleep(5)
print "sending 'configure reporting'"
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x00' + '\xaa' + '\x06' + '\x00' + '\x00' + '\x00' + '\x10' + '\x00' + '\x00' + '\x00' + '\x40' + '\x00' + '\x00'
)
firstTime = False
print "Enter a number from 0 through 8 to send a command"
str1 = raw_input("")
# Turn Switch Off
if(str1[0] == '0'):
print 'Turn switch off'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x01' + '\x01' + '\x00'
)
# Turn Switch On
if(str1[0] == '1'):
print 'Turn switch on'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x01' + '\x01' + '\x01'
)
# Toggle Switch
elif (str1[0] == '2'):
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x01' + '\x01' + '\x02'
)
# This will dim it to 20/256 over 5 seconds
elif (str1[0] == '3'):
print 'Dim it'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x08', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x01'+'\xaa'+'\x00'+'\x25'+'\x32'+'\x00'
)
# This will brighten it up to 100% over 5 seconds
elif (str1[0] == '4'):
print 'Bright'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x08', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x01'+'\xaa'+'\x00'+'\xff'+'\x32'+'\x00'
)
elif (str1[0] == '5'):
print 'Report Switch Status'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x01',
cluster = '\x00\x06', # cluster I want to deal with
profile = '\x01\x04', # home automation profile
data = '\x00'+'\xaa'+'\x00'+'\x00'+'\x00'
)
elif (str1[0] == '6'):
print 'Get Report from Switch'
zb.send('tx_explicit',
dest_addr_long = switchLongAddr,
dest_addr = switchShortAddr,
src_endpoint = '\x00',
dest_endpoint = '\x00',
cluster = '\x00\x05', # cluster I want to deal with
profile = '\x00\x00', # home automation profile
data = switchShortAddr[1]+switchShortAddr[0]
)
except IndexError:
print "empty line, try again"
except KeyboardInterrupt:
print "Keyboard interrupt"
break
except NameError as e:
print "NameError:",
print e.message.split("'")[1]
traceback.print_exc(file=sys.stdout)
except:
print "Unexpected error:", sys.exc_info()[0]
traceback.print_exc(file=sys.stdout)
break

sys.stdout.flush() # if you're running non interactive, do this

print ("After the while loop")
# halt() must be called before closing the serial
# port in order to ensure proper thread shutdown
zb.halt()
ser.close()

No, it isn't pretty, but it has comments.  It should be easy for folk to read and try out, and is the first example of a direct interface to a ZigBee compliant device I've ever seen.  This should take some of the mystery out of the protocol and the controllers that use it.  This code could be expanded to work one of the thermostats, receive from a panic button, or even one of those simple alarm switches.  The XBee does all the really hard stuff and saves us from worrying about it.

Have fun

ZIgBee Protocol, XBee, but this time an Arduino !

$
0
0
A couple of days ago I posted how to get a switch using the ZigBee protocol running with an XBee <link>; I'm pretty proud of that, but there's lots of folk out there with an Arduino already.  What about them?  Since I love those little board, I ported the code over to it as well.  See, the protocol is huge and extensive, but once you know how to operate a device that uses the protocol, you don't need much code to keep doing it.  That makes the little Arduino a cool device for controlling a ZigBee switch.

This code is the rough equivalent of the code I posted previously for the Raspberry Pi, same selections, except I left out the one that interrogates the switch, and let it report when the switch joins with the Arduino.  So, go look at the explanation I've already done to see how it works and how to use it.  Here's the Arduino Sketch:

/**
This is an implementation of Zigbee device communication using an XBee
and a Centralite Smart Switch 4256050-ZHAC

dave (www.desert-home.com)
*/

// This code will handle both a Uno and a Mega2560 by careful use of
// the defines. I tried it on both of them, and the only problem is that
// SoftwareSerial sometimes loses characters because the input buffer
// is too small. If you have this problem, see the SoftwareSerial
// documentation to see how to change it.
#include <XBee.h>
//#include <SoftwareSerial.h>
#include <Time.h>
#include <TimeAlarms.h>

// create reusable objects for messages we expect to handle
// using them over and over saves memory instead of sucking it off the
// stack every time we need to send or receive a message by creating
// a new object
XBee xbee = XBee();
XBeeResponse response = XBeeResponse();
ZBExpRxResponse rx = ZBExpRxResponse();
ZBExpCommand tx;
XBeeAddress64 Broadcast = XBeeAddress64(0x00000000, 0x0000ffff);

// Define the hardware serial port for the XBee (mega board)
#define ssRX 2
#define ssTX 3
// Or define NewSoftSerial TX/RX pins
// Connect Arduino pin 2 to Tx and 3 to Rx of the XBee
// I know this sounds backwards, but remember that output
// from the Arduino is input to the Xbee
//SoftwareSerial nss(ssRX, ssTX);

XBeeAddress64 switchLongAddress;
uint16_t switchShortAddress;
uint16_t payload[50];
uint16_t myFrameId=1;

void setup() {
// start serial
Serial.begin(9600);
// and the software serial port
//nss.begin(9600);
// Or the hardware serial port
Serial1.begin(9600);
// now that they are started, hook the XBee into
// whichever one you chose
//xbee.setSerial(nss);
xbee.setSerial(Serial1);
setTime(0,0,0,1,1,14); // just so alarms work well, I don't really need the time.
Serial.println("started");
}

boolean firstTime = true;

void loop() {
// Since this test code doesn't have the switch address, I'll
// send a message to get the routes to the devices on the network
// All devices are supposed to respond to this, and even the
// XBee we're hooked to will respond for us automatically
// The second message in will be the switch we want to work
// with. Thus, giving us the address we need to do things with
if (firstTime){
Serial.println(F("Wait while I locate the device"));
// First broadcast a route record request so when the switch responds
// I can get the addresses out of it
Serial.println(F("Sending Route Record Request"));
uint8_t rrrPayload[] = {0x12,0x01};
tx = ZBExpCommand(Broadcast, //This will be broadcast to all devices
0xfffe,
0, //src endpoint
0, //dest endpoint
0x0032, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
rrrPayload, //payload
sizeof(rrrPayload), //payload length
0x00); // frame ID
xbee.send(tx);
firstTime = false;
}

// First, go check the XBee, This is non-blocking, so
// if nothing is there, it will just return. This also allows
// any message to come in at any time so thing can happen
// automatically.
handleXbee();
// After checking the XBee for data, look at the serial port
// This is non blocking also.
handleSerial();
// Now, update the timer and do it all over again.
// This code tries to not wait for anything. It keeps it
// from hanging up unexpectedly. This way we can implement a
// watchdog timer to take care of the occasional problem.
Alarm.delay(0); // Just for the alarm routines
}

void handleSerial(){
if (Serial.available() > 0) {
char incomingByte;

incomingByte = Serial.read();
// Originally, I had a routine to send messages, but it tended to hide
// the way the messages were constructed from new folk. I changed it
// back to a verbose construction of each message sent to control the
// switch so people could more easily understand what they needed to do
if (isdigit(incomingByte)){
Serial.print("Selection: ");
int selection = atoi(&incomingByte);
Serial.print(selection, DEC);
switch(selection){
case 0: { // switch off
Serial.println(F(" Turn switch off"));
// In these outgoing messages I set the transaction sequence
// number to 0xaa so it could be easily seen if I was dumping
// messages as they went out.
uint8_t offPayload[] = {0x11,0xaa,0x00};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
offPayload, //payload
sizeof(offPayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}
case 1: { // switch on
Serial.println(F(" Turn switch on"));
uint8_t onPayload[] = {0x11,0xaa,0x01};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
onPayload, //payload
sizeof(onPayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}
case 2: { // switch toggle
Serial.println(F(" Toggle switch"));
uint8_t togglePayload[] = {0x11,0xaa,0x02};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
togglePayload, //payload
sizeof(togglePayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}
case 3: {
Serial.println(F(" Dim"));
uint8_t dimPayload[] = {0x11,0xaa,0x00,25,0x32,0x00};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0008, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
dimPayload, //payload
sizeof(dimPayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}
case 4: {
Serial.println(F(" Bright"));
uint8_t brightPayload[] = {0x11,0xaa,0x00,255,0x32,0x00};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0008, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
brightPayload, //payload
sizeof(brightPayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}
case 5: {
Serial.println(F(" Get State of Light "));
uint8_t ssPayload[] = {0x00,0xaa,0x00,0x00,0x00};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
ssPayload, //payload
sizeof(ssPayload), //payload length
0x00); // frame ID
xbee.send(tx);
break;
}

default:
Serial.println(F(" Try again"));
break;
}
// Now a short delay combined with a character read
// to empty the input buffer. The IDE developers removed
// the input flush that used to work for this.
while(Serial.available() > 0){
char t = Serial.read();
delay(25);
}
}
}
}
void handleXbee(){
// doing the read without a timer makes it non-blocking, so
// you can do other stuff in loop() as well. Things like
// looking at the console for something to turn the switch on
// or off
xbee.readPacket();
// the read above will set the available up to
// work when you check it.
if (xbee.getResponse().isAvailable()) {
// got something
//Serial.println();
//Serial.print("Frame Type is ");
// Andrew called the XBee frame type ApiId, it's the first byte
// of the frame specific data in the packet.
int frameType = xbee.getResponse().getApiId();
//Serial.println(frameType, HEX);
//
// All ZigBee device interaction is handled by the two XBee message type
// ZB_EXPLICIT_RX_RESPONSE (ZigBee Explicit Rx Indicator Type 91)
// ZB_EXPLICIT_TX_REQUEST (Explicit Addressing ZigBee Command Frame Type 11)
// This test code only uses these and the Transmit Status message
//
if (frameType == ZB_EXPLICIT_RX_RESPONSE) {
// now that you know it's a Zigbee receive packet
// fill in the values
xbee.getResponse().getZBExpRxResponse(rx);
int senderProfileId = rx.getProfileId();
// For this code, I decided to switch based on the profile ID.
// The interaction is based on profile 0, the general one and
// profile 0x0104, the Home Automation profile
//Serial.print(F(" Profile ID: "));
//Serial.print(senderProfileId, HEX);

// get the 64 bit address out of the incoming packet so you know
// which device it came from
//Serial.print(" from: ");
XBeeAddress64 senderLongAddress = rx.getRemoteAddress64();
//print32Bits(senderLongAddress.getMsb());
//Serial.print("");
//print32Bits(senderLongAddress.getLsb());

// this is how to get the sender's
// 16 bit address and show it
uint16_t senderShortAddress = rx.getRemoteAddress16();
//Serial.print(" (");
//print16Bits(senderShortAddress);
//Serial.println(")");

// for right now, since I'm only working with one switch
// save the addresses globally for the entire test module
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();

uint8_t* frameData = rx.getFrameData();
// We're working with a message specifically designed for the
// ZigBee protocol, see the XBee documentation to get the layout
// of the message.
//
// I have the message and it's from a ZigBee device
// so I have to deal with things like cluster ID, Profile ID
// and the other strangely named fields that these devices use
// for information and control
//
// I grab the cluster id out of the message to make the code
// below simpler.
//Serial.print(F(" Cluster ID: "));
uint16_t clusterId = (rx.getClusterId());
//print16Bits(clusterId);
//
// Note that cluster IDs have different uses under different profiles
// First I'll deal with the general profile.
if (senderProfileId == 0x0000){ // This is the general profile
if (clusterId == 0x00){
//Serial.println(F(" Basic Cluster"));
pass();
}
else if (clusterId == 0x0006){ // Match Descriptor
//Serial.println(F(" Match Descriptor"));
/*************************************/
// I don't actually need this message, but it comes in as soon as
// a device is plugged in. I answer it with a messsage that says I
// have an input cluster 0x19, since that's what it's looking for.
// Ignoring this message doesn't seem to hurt anything either.
uint8_t mdPayload[] = {0xAA,0x00,0x00,0x00,0x01,0x19};
mdPayload[2] = switchShortAddress & 0x00ff;
mdPayload[3] = switchShortAddress >> 8;
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x8006, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
mdPayload, //payload
sizeof(mdPayload), //payload length
0x00); // frame ID
xbee.send(tx);
// if you unplug a device, and then plug it back in, it loses the
// configuration for reporting on/off changes. So, send the configuration
// to get the switch working the way I want it to after the match
// descriptor message.
Serial.println (F("sending cluster command Configure Reporting "));
uint8_t crPayload[] = {0x00,0xaa,0x06,0x00,0x00,0x00,0x10,0x00,0x00,0x00,0x40,0x00,0x00};
tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
crPayload, //payload
sizeof(crPayload), //payload length
0x00); // frame ID
xbee.send(tx);

}
else if (clusterId == 0x0013){ //device announce message
// any time a new device joins a network, it's supposed to send this
// message to tell everyone its there. Once you get this message,
// you can interogate the new device to find out what it is, and
// what it can do.
Serial.println(F(" Device Announce Message"));
switchLongAddress = rx.getRemoteAddress64();
switchShortAddress = rx.getRemoteAddress16();
// Ok we saw the switch, now just for fun, get it to tell us
// what profile it is using and some other stuff.
// We'll send an Acttive Endpoint Request to do this
Serial.println (F("sending Active Endpoint Request "));
// The active endpoint request needs the short address of the device
// in the payload. Remember, it needs to be little endian (backwards)
// The first byte in the payload is simply a number to identify the message
// the response will have the same number in it.
uint8_t aePayload[] = {0xAA,0x00,0x00};
aePayload[1] = switchShortAddress & 0x00ff;
aePayload[2] = switchShortAddress >> 8;
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x0005, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
aePayload, //payload
sizeof(aePayload), //payload length
0xaa); // frame ID
xbee.send(tx);
}
else if (clusterId == 0x8004){
Serial.println(F(" Simple Descriptor Response "));
// Since I've been through this switch a few times, I already know
// what to expect out of it. This response is how you get the actual
// clusters that it has code for, and the profile ID that it supports.
// Since this is a light switch, it will support profile 0x104 and have
// clusters that support things like on/off and reporting.
// The items of interest are in the rf_data payload, and this is one way
// to get them out.
unsigned char *data = rx.getRFData(); // first get a pointer to the data
Serial.print(F(" Transaction ID: "));
print16Bits(data[0]); // remember the number that starts the payload?
Serial.println();
Serial.print(F(" Endpoint Reported: "));
print8Bits(data[5]);
Serial.println();
Serial.print(F(" Profile ID: "));
print8Bits(data[7]); // Profile ID is 2 bytes long little endian (backwards)
print8Bits(data[6]);
Serial.println();
Serial.print(F(" Device ID: "));
print8Bits(data[9]); // Device ID is 2 bytes long little endian (backwards)
print8Bits(data[8]);
Serial.println();
Serial.print(F(" Device Version: "));
print8Bits(data[10]); // Device ID is 1 byte long
Serial.println();
Serial.print(F(" Number of input clusters: "));
print8Bits(data[11]); // Input cluster count
Serial.print(F(", Clusters: "));
Serial.println();
for (int i = 0; i < data[11]; i++){
Serial.print(F(""));
print8Bits(data[i*2+13]); // some more of that little endian crap
print8Bits(data[i*2+12]);
Serial.println();
}
int outidx = 11 + 1 + 2*data[11];
Serial.print(F(" Number of output clusters: "));
print8Bits(data[outidx]); // Input cluster count
Serial.print(F(", Clusters: "));
Serial.println();
for (int i = 0; i < data[outidx]; i++){
Serial.print(F(""));
print8Bits(data[i*2 + outidx + 2]); // some more of that little endian crap
print8Bits(data[i*2 + outidx + 1]);
Serial.println();
}
Serial.println (F("sending cluster command Configure Reporting "));
// OK, for illustration purposes, this is enough to actually do something
// First though, let's set up the switch so that it reports when it has
// changed states in the on/off cluster (cluster 0006). This will require we
// send a message to the on/off cluster with the "Configure Reporting" command
// (0x06) with a bunch of parameters to specify things.
uint8_t crPayload[] = {0x00,0xaa,0x06,0x00,0x00,0x00,0x10,0x00,0x00,0x00,0x40,0x00,0x00};
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
1, //dest endpoint
0x0006, //cluster ID
0x0104, //profile ID
0, //broadcast radius
0x00, //option
crPayload, //payload
sizeof(crPayload), //payload length
0x00); // frame ID
xbee.send(tx);

}
else if (clusterId == 0x8005){
Serial.println(F(" Active Endpoints Response"));
// This message tells us which endpoint to use
// when controlling the switch. Since this is only a switch,
// it will give us back one endpoint. I should really have a loop
// in here to handle multiple endpoints, but ...
Serial.print(F(" Active Endpoint Count reported: "));
Serial.println(rx.getRFData()[4]);
Serial.print(F(" Active Endpoint: "));
Serial.println(rx.getRFData()[5]);
// Now we know that it has an endpoint, but we don't know what profile
// the endpoint is under. So, we send a Simple Descriptor Request to get
// that.
Serial.println (F("sending Simple Descriptor Request "));
// The request needs the short address of the device
// in the payload. Remember, it needs to be little endian (backwards)
// The first byte in the payload is simply a number to identify the message
// the response will have the same number in it. The last number is the
// endpoint we got back in the Active Endpoint Response.
// Also note that we're still dealing with profile 0 here, we haven't gotten
// to the device we actually want to play with yet.
uint8_t sdPayload[] = {0xAA,0x00,0x00,01};
sdPayload[1] = switchShortAddress & 0x00ff;
sdPayload[2] = switchShortAddress >> 8;
sdPayload[3] = rx.getRFData()[5];
ZBExpCommand tx = ZBExpCommand(switchLongAddress,
switchShortAddress,
0, //src endpoint
0, //dest endpoint
0x0004, //cluster ID
0x0000, //profile ID
0, //broadcast radius
0x00, //option
sdPayload, //payload
sizeof(sdPayload), //payload length
0xaa); // frame ID
xbee.send(tx);
}
else if (clusterId == 0x8032){
Serial.print(" Response from: ");
print16Bits(senderShortAddress);
Serial.println();
if(switchShortAddress != 0x0000){
Serial.print(F("Got switch address "));
Serial.println(F("Ready"));
}
}
else{
Serial.print(F(" Haven't implemented this cluster yet: "));
Serial.println(clusterId,HEX);
}
}
else if(senderProfileId == 0x0104){ // This is the Home Automation profile
// Since these are all ZCL (ZigBee Cluster Library) messages, I'll suck out
// the cluster command, and payload so they can be used easily.
//Serial.println();
//Serial.print(" RF Data Received: ");
//for(int i=0; i < rx.getRFDataLength(); i++){
//print8Bits(rx.getRFData()[i]);
//Serial.print('');
//}
//Serial.println();
if (clusterId == 0x0000){
//Serial.print(F(" Basic Cluster"));
pass();
}
else if (clusterId == 0x0006){ // Switch on/off
// Serial.println(F(" Switch on/off"));
// with the Centralite switch, we don't have to respond
// A message to this cluster tells us that the switch changed state
// However, if the response hasn't been configured, it will give back
// default response (cluster command 0b)
// so let's dig in and see what's going on.
//
// The first two bytes of the rfdata are the ZCL header, the rest of
// the data is a three field indicator of the attribute that changed
// two bytes of attribute identifier, a byte of datatype, and some bytes
// of the new value of the attribute. Since this particular attribute is a
// boolean (on or off), there will only be one byte. So
if(rx.getRFData()[2] == 0x0b){ // default response (usually means error)
Serial.println(F(" Default Response: "));
Serial.print(F(" Command: "));
print8Bits(rx.getRFData()[3]);
Serial.println();
Serial.print(F(" Status: "));
print8Bits(rx.getRFData()[4]);
Serial.println();
}
else if (rx.getRFData()[2] == 0x0a || rx.getRFData()[2] == 0x01){
// This is what we really want to know
Serial.print(F("Light "));
// The very last byte is the status
if (rx.getRFData()[rx.getRFDataLength()-1] == 0x01){
Serial.println(F("On"));
}
else{
Serial.println(F("Off"));
}
}
else{ // for now, the ones above were the only clusters I needed.
//Serial.println(F(" I don't know what this is"));
pass();
}
}
else if (clusterId == 0x0008){ // This is the switch level cluster
// right now I don't do anything with it, but it's where
// the switch lets you know about level changes
}
else{
Serial.print(F(" Haven't implemented this cluster yet: "));
Serial.println(clusterId,HEX);
}
}
}
else {
if (frameType == 0xa1){
//Serial.println(F(" Route Record Request"));
pass();
}
else if (frameType == ZB_TX_STATUS_RESPONSE){
//Serial.print(F(" Transmit Status Response"));
pass();
}
else{
Serial.print(F("Got frame type: "));
Serial.print(frameType, HEX);
Serial.println(F(" I didn't implement this frame type for this experiment"));
}
}
}
else if (xbee.getResponse().isError()) {
// some kind of error happened, I put the stars in so
// it could easily be found
Serial.print("************************************* error code:");
Serial.println(xbee.getResponse().getErrorCode(),DEC);
}
else {
// If you get here it only means you haven't collected enough bytes from
// the XBee to compose a packet.
}
}

/*-------------------------------------------------------*/
// null routine to avoid some syntax errors when debugging
void pass(){
return;
}
// these routines are just to print the data with
// leading zeros and allow formatting such that it
// will be easy to read.
void print32Bits(uint32_t dw){
print16Bits(dw >> 16);
print16Bits(dw & 0xFFFF);
}

void print16Bits(uint16_t w){
print8Bits(w >> 8);
print8Bits(w & 0x00FF);
}

void print8Bits(byte c){
uint8_t nibble = (c >> 4);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);

nibble = (uint8_t) (c & 0x0F);
if (nibble <= 9)
Serial.write(nibble + 0x30);
else
Serial.write(nibble + 0x37);
}

Have fun with it.

Viewing all 218 articles
Browse latest View live