Polling Plugwise data

[Radu]

Hello readers

I got last week a functional network of 16 circles. (I use a dedicated XP machine, low cost, no fan, small hd, etc).

The only way to obtain data on my linux box (heart of my ha tinkering) is polling the Source object (?) (wget http://user:pass@192.168.1.34:8080/xml/appliances.xml). I interpret the result in Perl and stuff an Oracle db (I have to, it is my job ). As a first approach, I display all modules information and all appliances information.

Question zero: Very often, the powerusage in the appliance is different from the powerusage dispayed in the module list. (for heavy consumers, 4 to 8 watts) - see http://www.tafora.com/mod_perl/gate/infoplugwise.pl, chances are you notice this too. (Left column module, right column appliance) (496.442 vs 492.1309). I suppose this is maybe because modules and appliances are just objects polled by the source itself at different times)

All the questions below are XML poll related.
Question 1) Is there a way to be notified immediately after an on/off action ? (Or should I simply poll very frequently the source base ?).
(I see on the splendid bwired list of devices that they are frequently polled (i infere that because of the constant time of the date): (LASTRECEIVED column)
Question 2) Is a very frequent poll really reflecting the reality ? I mean very often, the source stuff is displaying new, data, the xml stuf freflects it later (minutes ?)

Last but one question for today. Is polling the mdb file more accurate/fast than the xml poll ? Because if it is, i will have to check some perl related modules in order to get data from that ? (I discoverd a new hidden mdb file in MY profile ). I am learning windows too.

Are there any other polling methods ? One, xml, two mdb, three ... Maybe reading directly the serial port corresponding to the usb dongle?

Thank you for your attention.



RC

[Darwusch]

Hi Radu,

Sorry, I don't have the answer to your question.
However, it brings a related question to my mind that I have myself. So I guess I shouldn't start a separate threat.

How do you poll at all? I know how to see the "real time" data by using an xml-script. But how do I put this in a mysql database automatically, let's say every 5 minutes?
I already have some mysql databases running on my server, so I know a little how to setup a database. But do I need to run a certain script in the background as a service or something to put it in the database?

Any help would be appreciated!
Thanks.
Darwusch

[Radu]

Hi Darwusch

I poll now in two ways:
a) with the xml stuff. (and soon the mdb file)
1) I have a shell script which reads from the Source web server. The result is a plain file.
2) the plain file is read by a perl script which dumps the xml result (actual values and enters in the db those values). So the script reads a text file, connects to a db and writes. My db is Oracle, but you may use any other DB (in perl, you use a DBD::Something). part of snippet is here:
# use module
use XML::Simple;
use Data::Dumper;
$xml = new XML::Simple;
$data = $xml->XMLin($filename);
($data is the xml generated from Source server)
...
# HoH is a hash of hashes, data read from the file, acual values
%HoH=%{$data};
...
For each line of data, it writes to the db :
my $ordre="Insert into ha.$Subitem "."(".join(",", @sqlordera).")";
$ordre=$ordre." values ". "(".join(",", @sqlorderb).")";
(insert into WhichTable (c1, c2, c3 ...) values (val1, val2, val3...)

For a quick and dirty solution (I do not know if you master perl stuff), you could dump the real sql orders from the xml interpreted data to a text file and then run another script every x minutes which commands Mysql (I do not know mysql, but In Oracle I may do something like
sqlplus user/pass@db @myscript.sql (meaning running the script myscript.sql as user user/pass against the db database.

There might be much cleaner solutions

Regards


RC

[Darwusch]

Thank you for your quick reply.
So what part is running all the time and containing the update frequency, is that the shell script? And does the shell script activate the perl script to dump the data in the database?

[Digit]

<blockquote id="quote"><font size="1" face="Verdana, Arial, Helvetica" id="quote">quote:<hr height="1" noshade id="quote"><i>Originally posted by Radu</i>
<br />Maybe reading directly the serial port corresponding to the usb dongle?
<hr height="1" noshade id="quote"></font id="quote"></blockquote id="quote">Yes, that is an option too: http://www.domoticaforum.eu/topic.asp?TOPIC_ID=1771
But i can't imagine you guys missing this topic

[Bwired]

Hi Radu,

You need to run the XML two times to get the current usage, so you are always a little behind. If you run the XML, you get the current value and an update is started to get the new value from the circle which is placed in grid off the source. Pol again and you get that value!
So I have my doubts about polling the webserver with the XML, this is the same what the Source software is doing for filling the grid and MDB.
And if you poll like every 30 seconds you are putting an extra load on the zigbee plugwise network.
I poll like every 60 seconds now but considering getting my data form the mdb now, I will just make a function which is updating my MYSQL database with all the data form the tables. Just open one of the tables and you see them updating realtime and frequently!

Polling the devices yourself I think is only good when you have a direct API to the Plugwise dongle so not using the Plugwise Source software.
I'm still experimenting with it and checking if I'm right.
Check this forum because Maarten has made a Linux setup and getting data directly from the Dongle.

[Radu]

Thank you Pieter. I saw in another forum topic the idea of polling twice. You answerd also the question about what is better: poll XML or mdb. (As you want to use the mdb, I infere it is better)

For Digit's (parent to son phrase) "But I can't imagine you guys missing the topic" : as a matter of fact, I read it more than once, I even tried the Linux port, but I was stopped by some system library missing, and of course I might have missed a lot of things, because of the excitement of discovery of new things. anyway, I know I will use the windows stuff as short as possible

Regards to all

Radu



RC

[Rene]

You will have a problem with accessing the MDB directly as soon as Plugwise changes the schema of the database. Then you have to modify your code. While a good API, like the webserver hopefully will offer in time, will still support 'old style' API calls. Something to consider;-)


Rene.

[Bwired]

Also true!
Also not all thing about the Plugwise webserver are completely clear on how it gets its data etc.
We will talk to Plugwise about this.

[Darwusch]

Hi,

After lots of trying I could not write a script which puts the data in my new mysql database.
I have a new empty database ready with a table per devicem with these fields:
time - powerusage

I would like to have a script run in the background which polls every 5 minutes (from de access database from plugwise) and stores the data in the mysql database.

Does anybody have a script that I can use for this?

[Jfn]

My domotica system is running on Linux too.

I am working on a xpl-perl based implementation for Plugwise that allows me to bypass the Source software and read data from the plugs directly (See this thread: http://www.domoticaforum.eu/topic.asp?TOPIC_ID=2939)

Another excellent piece of info on supporting plugs directly under Linux (By forummember Maarten) can be found here: http://www.domoticaforum.eu/topic.asp?TOPIC_ID=1771

The part that extracts (usage) data from the plugs is more or less finished and I am about to start working on a (perl-based) script that fetches this data from the plugs and stores the data in a MySQL database.

Fetching realtime energyconsumption from the plugs (using xpl-perl) is very simple:

Code: Select all

$msg->send_from_string ("-m xpl-cmnd -c plugwise.basic command=powerinfo device='37b4a7'")

Which gives:

[xpl-trig/plugwise.basic: bnz-plugwise.debian -> * - powerinfo 37B4A7]
 plugwise.basic
 {
  command=powerinfo
  device=37B4A7
  pulse1sec=001E    (Current energy consumption - 1 second average)
  pulse8sec=001F    (Current energy consumption - 8 second average)
  unknown=000132C3  (Meaning yet unknown)
 }

This data combined with some other data you need to fetch from the plugs allows for easy calculation of energyconsumption (In the example above the 1s average equals to about 60W, which is the load generated by my UPS, Domotica PC, cable modem and router).

The plugs also store energy-consumption on an hourly basis in buffers which can be read very easily.

Unfortunately there is nog native Linux support (yet) for Plugwise, but I am sure there will be in the future.