In the previous post <link> I described how I set up a separate process for logging to the cloud servers at ThingSpeak and Xively. This removes the code from my house monitor process and makes changing or adding things much easier. So, I had two more to do, my legacy feeds on Xively and the datastore on emoncms. I just finished the emoncms process last night. It was relatively easy except for one tiny item. The emoncms server almost never returns an error. I messed up the feed over and over again getting it to work and the server never returned an error. So, I had to run the process, watch the site for updates and hope it updated fast enough for me to tell if it was getting through.
I got it working and ran it over night to be sure it was going to hold up. The code looks very, very much like the code for updating ThingSpeak, so there aren't too many surprises. Once again, this is a standalone process that takes items from a simple database and forwards them to emoncms for archival.
As usual, it's way over commented. I think there's a lot more lines of comment than lines of code in these things.
Now, I have only one more of these to build, the one to keep my legacy feeds at Xively up to date. I want to keep them running when I turn off the old controller so I can transfer the data from them to the new Xively interface. Then, I'm going to take a look at a new service that just came on line a little while back to see how they work.
Cloud services are the way of the future IMHO.
I got it working and ran it over night to be sure it was going to hold up. The code looks very, very much like the code for updating ThingSpeak, so there aren't too many surprises. Once again, this is a standalone process that takes items from a simple database and forwards them to emoncms for archival.
The Python Script
#! /usr/bin/python
from apscheduler.scheduler import Scheduler
import datetime
import logging
import time
import sqlite3
import httplib, urllib
# The API key that is needed
EMONKEY = "PUTSOMETHINGINHERE"
# the database where I'm storing stuff
DATABASE='/home/pi/database/desert-home'
# This is where the update to ThingSpeak happens
def updateEmonCms():
#print "Updating emoncms ", time.strftime("%A, %B %d at %H:%M:%S")
# open the database
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# Getting it out of the database a field at a time
# is probably less efficient than getting the whole record,
# but it works.
#
# On ThingSpeak I only update real power, power factor, voltage,
# frequency, outside temperature and inside temperature
# so I'm simply going to put the values in variables instead of
# some complex (and probably faster) compound statement.
outsideTemp = c.execute(
"select currenttemp from xbeetemp").fetchone()[0]
# This a really cool thing about some languages
# the variable types are dynamic, so I can just change it
# from a string to a int on the fly.
outsideTemp = int(float(outsideTemp) +.5)
power = c.execute(
"select rpower from power").fetchone()[0]
power = int(float(power)+.5)
voltage = c.execute(
"select voltage from power").fetchone()[0]
voltage = int(float(voltage)+.5)
apparentPower = c.execute(
"select apower from power").fetchone()[0]
apparentPower = float(apparentPower)
current = c.execute(
"select current from power").fetchone()[0]
current = int(float(current)+.5)
frequency = c.execute(
"select frequency from power").fetchone()[0]
frequency = float(frequency)
powerFactor = c.execute(
"select pfactor from power").fetchone()[0]
powerFactor = float(powerFactor)
insideTemp = c.execute(
"select avg(\"temp-reading\") from thermostats").fetchone()[0]
insideTemp = int(float(insideTemp)+.5)
# OK, got all the stuff I want to update
dbconn.close() # close the data base
#
# This is a debug statement that I put in to show
# not only what the values were, but also how they
# can be formatted.
# print ("Power = %d \nVoltage = %d \nApparent Power = %d "
# "\nCurrent = %d \nFrequency %.2f \nPower Factor = %.2f "
# "\nOutside Temp = %d \nInside Temp = %d" %
# (power, voltage, apparentPower, current,
# frequency, powerFactor, outsideTemp, insideTemp))
# OK, now I've got all the data I want to record on emoncms
# so I have to put it in json form. json isn't that hard if you
# don't have multiple levels, so I'll just do it with a string
# format. It's just a set of ordered pairs for this.
params = ("RealPower:%d,PowerFactor:%.2f,"
"PowerVoltage:%d,PowerFrequency:%.2f,"
"InsideTemp:%d,OutsideTemp:%d" %
(power,powerFactor,voltage,frequency,insideTemp,
outsideTemp))
# if you want to see the result of the formatting, just uncomment
# the line below. This stuff gets confusing, so give it a try
#print params
#
# Now, just send it off to emoncms
conn = httplib.HTTPConnection("emoncms.org:80")
request = "/input/post?apikey=" + EMONKEY + "&" + "json=" + params
#print request
# emoncms uses a GET not a POST
conn.request("GET", request)
response = conn.getresponse()
#print "emoncms Response:", response.status, response.reason
# I only check for the 'OK' in the reason field. That's
# so I can print a failure to any log file I happen to set
# up. I don't want to print a lot of stuff that I have to
# manage somehow. However, emoncms seldom returns an error,
# I messed this interaction up a number of times and never got
# an error.
if (response.reason != 'OK'):
print "Problem, ", response.status, response.reason
# conn.close
# This is where the main code begins. Notice how basically nothing
# happens here? I simply show a sign on message, set up logging, and
# start a scheduled task to actually do the work.
print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
logging.basicConfig()
#------------------Stuff I schedule to happen -----
scheditem = Scheduler()
scheditem.start()
# every minute update the data store on ThingSpeak
scheditem.add_interval_job(updateEmonCms, seconds=60)
while True:
time.sleep(20) #This doesn't matter much since it is schedule driven
from apscheduler.scheduler import Scheduler
import datetime
import logging
import time
import sqlite3
import httplib, urllib
# The API key that is needed
EMONKEY = "PUTSOMETHINGINHERE"
# the database where I'm storing stuff
DATABASE='/home/pi/database/desert-home'
# This is where the update to ThingSpeak happens
def updateEmonCms():
#print "Updating emoncms ", time.strftime("%A, %B %d at %H:%M:%S")
# open the database
dbconn = sqlite3.connect(DATABASE)
c = dbconn.cursor()
# Getting it out of the database a field at a time
# is probably less efficient than getting the whole record,
# but it works.
#
# On ThingSpeak I only update real power, power factor, voltage,
# frequency, outside temperature and inside temperature
# so I'm simply going to put the values in variables instead of
# some complex (and probably faster) compound statement.
outsideTemp = c.execute(
"select currenttemp from xbeetemp").fetchone()[0]
# This a really cool thing about some languages
# the variable types are dynamic, so I can just change it
# from a string to a int on the fly.
outsideTemp = int(float(outsideTemp) +.5)
power = c.execute(
"select rpower from power").fetchone()[0]
power = int(float(power)+.5)
voltage = c.execute(
"select voltage from power").fetchone()[0]
voltage = int(float(voltage)+.5)
apparentPower = c.execute(
"select apower from power").fetchone()[0]
apparentPower = float(apparentPower)
current = c.execute(
"select current from power").fetchone()[0]
current = int(float(current)+.5)
frequency = c.execute(
"select frequency from power").fetchone()[0]
frequency = float(frequency)
powerFactor = c.execute(
"select pfactor from power").fetchone()[0]
powerFactor = float(powerFactor)
insideTemp = c.execute(
"select avg(\"temp-reading\") from thermostats").fetchone()[0]
insideTemp = int(float(insideTemp)+.5)
# OK, got all the stuff I want to update
dbconn.close() # close the data base
#
# This is a debug statement that I put in to show
# not only what the values were, but also how they
# can be formatted.
# print ("Power = %d \nVoltage = %d \nApparent Power = %d "
# "\nCurrent = %d \nFrequency %.2f \nPower Factor = %.2f "
# "\nOutside Temp = %d \nInside Temp = %d" %
# (power, voltage, apparentPower, current,
# frequency, powerFactor, outsideTemp, insideTemp))
# OK, now I've got all the data I want to record on emoncms
# so I have to put it in json form. json isn't that hard if you
# don't have multiple levels, so I'll just do it with a string
# format. It's just a set of ordered pairs for this.
params = ("RealPower:%d,PowerFactor:%.2f,"
"PowerVoltage:%d,PowerFrequency:%.2f,"
"InsideTemp:%d,OutsideTemp:%d" %
(power,powerFactor,voltage,frequency,insideTemp,
outsideTemp))
# if you want to see the result of the formatting, just uncomment
# the line below. This stuff gets confusing, so give it a try
#print params
#
# Now, just send it off to emoncms
conn = httplib.HTTPConnection("emoncms.org:80")
request = "/input/post?apikey=" + EMONKEY + "&" + "json=" + params
#print request
# emoncms uses a GET not a POST
conn.request("GET", request)
response = conn.getresponse()
#print "emoncms Response:", response.status, response.reason
# I only check for the 'OK' in the reason field. That's
# so I can print a failure to any log file I happen to set
# up. I don't want to print a lot of stuff that I have to
# manage somehow. However, emoncms seldom returns an error,
# I messed this interaction up a number of times and never got
# an error.
if (response.reason != 'OK'):
print "Problem, ", response.status, response.reason
# conn.close
# This is where the main code begins. Notice how basically nothing
# happens here? I simply show a sign on message, set up logging, and
# start a scheduled task to actually do the work.
print "started at ", time.strftime("%A, %B, %d at %H:%M:%S")
logging.basicConfig()
#------------------Stuff I schedule to happen -----
scheditem = Scheduler()
scheditem.start()
# every minute update the data store on ThingSpeak
scheditem.add_interval_job(updateEmonCms, seconds=60)
while True:
time.sleep(20) #This doesn't matter much since it is schedule driven
As usual, it's way over commented. I think there's a lot more lines of comment than lines of code in these things.
Now, I have only one more of these to build, the one to keep my legacy feeds at Xively up to date. I want to keep them running when I turn off the old controller so I can transfer the data from them to the new Xively interface. Then, I'm going to take a look at a new service that just came on line a little while back to see how they work.
Cloud services are the way of the future IMHO.