Toon app: Zon op Toon using PVoutput

Forum about forum-provided applications on Toon

Moderators: marcelr, TheHogNL, Toonz

Ierlandfan
Member
Member
Posts: 151
Joined: Thu Oct 03, 2013 7:53 pm

Re: Toon app: Zon op Toon using PVoutput

Post by Ierlandfan »

Yes, I think bxt is the way to go. I was thinking the same. I also wrote my own script to dump the values from a Steca 1500 Solar converter that dumps it to PVoutput.
I don\t know which package is responsible for the Solar writing part. IF we know the package I can create the bxt part.
Since my laptop decided to retire I only have a tablet with physical keyboard to investigate atm so it takes a little longer now.
Then it's just a matter of writing some simple bash script that runs every x minutes and updates the values.
marcelr
Global Moderator
Global Moderator
Posts: 1153
Joined: Thu May 10, 2012 10:58 pm
Location: Ehv

Re: Toon app: Zon op Toon using PVoutput

Post by marcelr »

Hi Ierlandfan,

I think it's the hcb_rrd executable that does all that work. From /HCBv2/config/config_hcb_rrd.xml at least, it looks like that. Not sure how (yet), but maybe I can squeeze more info from that executable with strace, similar to our fishing expedition in the OT protocol implementation. No guarantees, though.

What exactly are we looking for? The bxt protocol, or the shape of the data as extracted from the zwave interface, or something else?
bwbboot
Starting Member
Starting Member
Posts: 28
Joined: Fri Jul 28, 2017 2:02 am

Re: Toon app: Zon op Toon using PVoutput

Post by bwbboot »

Do you have strace available as an ipk? Possibly tcpdump as wel?
marcelr
Global Moderator
Global Moderator
Posts: 1153
Joined: Thu May 10, 2012 10:58 pm
Location: Ehv

Re: Toon app: Zon op Toon using PVoutput

Post by marcelr »

Yes, you can find them here:

http://files.domoticaforum.eu/uploads/T ... r1_qb2.ipk
http://files.domoticaforum.eu/uploads/T ... r1_qb2.ipk

tcpdump depends on this one:

http://files.domoticaforum.eu/uploads/T ... r1_qb2.ipk

Neither are the latest versions, they were built from the openembedded tree for toon.
bwbboot
Starting Member
Starting Member
Posts: 28
Joined: Fri Jul 28, 2017 2:02 am

Re: Toon app: Zon op Toon using PVoutput

Post by bwbboot »

Many thanks, works without any issues so far!
marcelr
Global Moderator
Global Moderator
Posts: 1153
Joined: Thu May 10, 2012 10:58 pm
Location: Ehv

Re: Toon app: Zon op Toon using PVoutput

Post by marcelr »

You're welcome. Just my curiosity: What are you trying to do with it?
bwbboot
Starting Member
Starting Member
Posts: 28
Joined: Fri Jul 28, 2017 2:02 am

Re: Toon app: Zon op Toon using PVoutput

Post by bwbboot »

Well, i figured since TOON can be updated from the mothership (like activate subscription) i want to run tcpdump on the TAP0 interface for a while to see whats coming in and out.
Interestingly my first findings confirm that communication is done by using the /bxtproxy. I even wrote a quick/dirty script to test if I could have it accept data. And it seems to work. Now trying to figure out if it can be used to update other stuff as well.

Code: Select all

#!/bin/bash
#
/usr/bin/curl -v -H "X-BoxTalk-Class: JS-Pavel" -H "X-BoxTalk-UUID: BoxtalkGatewayServlet" -H "Cache-Control: no-cache" -H "Pragma: no-cache" -H "Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2" -H "Connection: keep-alive" -H "Content-type: application/x-www-form-urlencoded" -A "User-Agent: Java/1.8.0_131" -X POST -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<action class="invoke" uuid="a0b6a2690-f343-11e1-a21f-0800200c9a66" destuuid="eneco-001-XXXXXX:happ_thermstat" serviceid="urn:hcb-hae-com:serviceId:specific1" commonname="eneco-001-XXXXXX" xmlns:ns5="urn:hcb-hae-com:service:ConfigProvider:1" xmlns:u="urn:hcb-hae-com:service:specific1:1" xmlns:ns4="urn:hcb-hae-com:service:SwitchPower:1" xmlns:ns3="urn:hcb-hae-com:service:Benchmark:1">
    <u:UpdateDataSetSubscription>
        <updateAction>extend</updateAction>
        <dataSet>thermostatInfo thermostatStates</dataSet>
    </u:UpdateDataSetSubscription>
</action>' http://192.168.2.180/bxtproxy

echo -e "\n"
The result:

Code: Select all

*   Trying 192.168.2.180...
* Connected to 192.168.2.180 (192.168.2.180) port 80 (#0)
> POST /bxtproxy HTTP/1.1
> Host: 192.168.2.180
> User-Agent: Java/1.8.0_131
> X-BoxTalk-Class: JS-Pavel
> X-BoxTalk-UUID: BoxtalkGatewayServlet
> Cache-Control: no-cache
> Pragma: no-cache
> Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
> Connection: keep-alive
> Content-type: application/x-www-form-urlencoded
> Content-Length: 624
> 
* upload completely sent off: 624 out of 624 bytes
< HTTP/1.1 200 OK
< Content-Length: 12
< Date: Tue, 08 Aug 2017 12:28:22 GMT
< Server: lighttpd/1.4.33-devel-17
< 
* Connection #0 to host 192.168.2.180 left intact
message sent
The /HCVv2/sbin/hcb_bxtproxy -v process responds with the following message:

Code: Select all

[hbxt]\--------------------------------/
[hcb_bxtproxy:../src/hcb_bxtproxy.c@http_post():444]ERROR: Not a valid BoxTalk message, not sent to hcb_comm!
[hcb_bxtproxy:../src/hcb_bxtproxy.c@updateMsgStatuses():908]Messages  SENT Ok:23 Retries:0 Error:0  RECV Ok:1 Err:22  QUEUE:0 env:PRD
[hcom]Received HBXT_ACTION_RESPONSE from eneco-001-xxxxxx:happ_thermstat to a0b6a2690-f343-11e1-a21f-0800200c9a66/specific1: n=UpdateDataSetSubscriptionResponse result=ok
[hcb_bxtproxy:../src/hcb_bxtproxy.c@handler():1051]ERROR: Retry this msg 3 times | send anonymous: 0
[hcb_bxtproxy:../src/hcb_bxtproxy.c@enqueuePacket_l():112]enqueing for uuid: eneco-001-XXXXXX:happ_thermstat to url: http://feed.hae.int:31080/toonMobileBackendWeb/display/receiveXml data: <action class="response" uuid="eneco-001-xxxxxx:happ_thermstat" destuuid="a0b6a2690-f343-11e1-a21f-0800200c9a66" serviceid="urn:hcb-hae-com:serviceId:specific1" commonname="eneco-001-XXXXXX">
    <u:UpdateDataSetSubscriptionResponse xmlns:u="urn:hcb-hae-com:service:specific1:1"><result>ok</result>
    </u:UpdateDataSetSubscriptionResponse>
</action> retries: 3
[hcb_bxtproxy:../src/hcb_bxtproxy.c@transmitMsgQueueHead_l():926]Sending to url: http://feed.hae.int:31080/toonMobileBackendWeb/display/receiveXml
[hcb_bxtproxy:../src/hcb_bxtproxy.c@http_post():304]ERROR: overrideUrl: http://172.18.96.1:31080/toonMobileBackendWeb/display/receiveXml
[hbxt]/--------------------------------\
Think we can safely ignore the first line since it seems to appear frequently without me interfering:

Code: Select all

"[hcb_bxtproxy:../src/hcb_bxtproxy.c@http_post():444]ERROR: Not a valid BoxTalk message, not sent to hcb_comm!"


Now onto constructing an update message and see what happens.
marcelr
Global Moderator
Global Moderator
Posts: 1153
Joined: Thu May 10, 2012 10:58 pm
Location: Ehv

Re: Toon app: Zon op Toon using PVoutput

Post by marcelr »

Nice, I think ...

Not sure yet what's possible with this, but please carry on!

The master thesis of Jurgen Braam (developer at Quby) describes the boxtalk protocol al little. Maybe it's helpful:
http://files.domoticaforum.eu/uploads/H ... _Braam.pdf
Ierlandfan
Member
Member
Posts: 151
Joined: Thu Oct 03, 2013 7:53 pm

Re: Toon app: Zon op Toon using PVoutput

Post by Ierlandfan »

You can use strace or just invoke the binary with -vvvvvvv
Then you'll see the commands that are being send. Maybe faster than strace. You want to look for the xml request that updates the solar values so I guess a ./hcb_rrd -vvvvvvv |grep solar " or something similar will do. So I would say as long as the z-wave registers a solar value (any value will do) hcb_rrd will try to update that value by an xml request to Boxtalk.


Edit...try to send the commands to localhost:1337, (or open the port in in the iptables for remote access) you're sending things to or through the Quby servers now.
bwbboot
Starting Member
Starting Member
Posts: 28
Joined: Fri Jul 28, 2017 2:02 am

Re: Toon app: Zon op Toon using PVoutput

Post by bwbboot »

no luck so far, any constructed XML posts to /bxtproxy are accepted but totally ignored by the hcb_bxtproxy which seems responsible for processing incoming requests.

@Ierlandfan, i don't have solar generation data so the rrd graphs will never have to process any updates there.
TerrorSource
Administrator
Administrator
Posts: 494
Joined: Thu May 04, 2017 9:28 pm

Re: Toon app: Zon op Toon using PVoutput

Post by TerrorSource »

In what format does the data needs to be delivered? Maybe i can give you some information as i do have 3yrs of history data from my solar panels.
bwbboot
Starting Member
Starting Member
Posts: 28
Joined: Fri Jul 28, 2017 2:02 am

Re: Toon app: Zon op Toon using PVoutput

Post by bwbboot »

This is one of the last tests that i did. A strace on for example HCBv2 happ_pwrusage reveils the following messages being sent:

Code: Select all

read(3, "<action class=\"response\" uuid=\"eneco-002-XXXXXX:hcb_config_61127789807\" destuuid=\"eneco-002-XXXXXX:happ_pwrusage\" serviceid=\"urn:hcb-hae-com:serviceId:specific1\" requestid=\"1295-1266\">\n    <u:GetRraDataResponse xmlns:u=\"urn:hcb-hae-com:service:specific1:1\"><requestId>1295-1266</requestId>\n   <rraData>{ \"1502218800\": NaN}</rraData>\n   <firstSample>1344430800</firstSample>\n   <lastSample>1502218800</lastSample>\n   <tzOffset>2</tzOffset>\n    </u:GetRraDataResponse>\n</action>\0", 16384) = 477
gettimeofday({1502218412, 425384}, NULL) = 0
read(3, "<notify uuid=\"5f195db2-8ae7-4e75-a7ba-d227f556c166\" serviceid=\"urn:hcb-hae-com:serviceId:ElectricityQuantityMeter\">\n    <CurrentElectricityQuantity>10798938</CurrentElectricityQuantity>\n</notify>\0", 16384) = 196
read(3, "<notify uuid=\"713aa639-1f74-42fd-a978-4558f50d1159\" serviceid=\"urn:hcb-hae-com:serviceId:ElectricityQuantityMeter\">\n    <CurrentElectricityQuantity>10857074</CurrentElectricityQuantity>\n</notify>\0", 16384) = 196
read(3, "<notify uuid=\"5f195db2-8ae7-4e75-a7ba-d227f556c166\" serviceid=\"urn:hcb-hae-com:serviceId:ElectricityFlowMeter\">\n    <CurrentElectricityFlow>890</CurrentElectricityFlow>\n</notify>\0", 16384) = 179
write(2, "\33[01;34m[\33[01;36mhdrv\33[0m\33[01;34m]\33[0m\33[01;35mDS: \33[0mdatasetUpdate powerUsage\n", 79) = 79
write(2, "\33[01;34m[\33[01;36mhdrv\33[0m\33[01;34m]\33[0m\33[01;35mDS: \33[0mUpdate for dataSet \33[01;35mpowerUsage\33[0m\n", 96) = 96
write(2, "\33[01;34m[\33[01;36mhdrv\33[0m\33[01;34m]\33[0m\33[01;35mDS: \33[0mSend full \33[01;35mpowerUsage\33[0m update to \33[01;35meneco-002-XXXXXX:qt-gui\33[0m\n", 133) = 133
write(3, "<action class=\"invoke\" uuid=\"eneco-002-XXXXXX:happ_pwrusage\" destuuid=\"eneco-002-XXXXXX:qt-gui\" serviceid=\"urn:hcb-hae-com:serviceId:specific1\">\n    <u:UpdateDataSet xmlns:u=\"urn:hcb-hae-com:service:specific1:1\"><powerUsage><value>890</value>\n    <dayCost>4.70</dayCost>\n    <valueProduced>0</valueProduced>\n    <dayCostProduced>0.00</dayCostProduced>\n    <valueSolar>NaN</valueSolar>\n    <maxSolar>NaN</maxSolar>\n    <dayCostSolar>NaN</dayCostSolar>\n    <avgSolarValue>NaN</avgSolarValue>\n    <avgValue>34.61</avgValue>\n    <avgDayValue>830.57</avgDayValue>\n    <avgProduValue>296.56</avgProduValue>\n    <avgDayProduValue>7117.43</avgDayProduValue>\n    <dayUsage>5605</dayUsage>\n    <dayLowUsage>3391</dayLowUsage>\n    <meterReading>10857074</meterReading>\n    <meterReadingLow>10960915</meterReadingLow>\n    <meterReadingProdu>6669844</meterReadingProdu>\n    <meterReadingLowProdu>2570472</meterReadingLowProdu>\n    <isSmart>1</isSmart>\n    <lowestDayValue>0</lowestDayValue>\n    <solarProducedToday>NaN</solarProducedToda"..., 1073) = 1073
write(2, "\33[01;34m[\33[01;36mhdrv\33[0m\33[01;34m]\33[0m\33[01;35mDS: \33[0mSend full \33[01;35mpowerUsage\33[0m update to \33[01;35ma0b6a2690-f343-11e1-a21f-0800200c9a66\33[0m\n", 147) = 147
write(3, "<action class=\"invoke\" uuid=\"eneco-002-XXXXXX:happ_pwrusage\" destuuid=\"a0b6a2690-f343-11e1-a21f-0800200c9a66\" serviceid=\"urn:hcb-hae-com:serviceId:specific1\">\n    <u:UpdateDataSet xmlns:u=\"urn:hcb-hae-com:service:specific1:1\"><powerUsage><value>890</value>\n    <dayCost>4.70</dayCost>\n    <valueProduced>0</valueProduced>\n    <dayCostProduced>0.00</dayCostProduced>\n    <valueSolar></valueSolar>\n    <maxSolar></maxSolar>\n    <dayCostSolar></dayCostSolar>\n    <avgSolarValue></avgSolarValue>\n    <avgValue>34.61</avgValue>\n    <avgDayValue>830.57</avgDayValue>\n    <avgProduValue>296.56</avgProduValue>\n    <avgDayProduValue>7117.43</avgDayProduValue>\n    <dayUsage>5605</dayUsage>\n    <dayLowUsage>3391</dayLowUsage>\n    <meterReading>10857074</meterReading>\n    <meterReadingLow>10960915</meterReadingLow>\n    <meterReadingProdu>6669844</meterReadingProdu>\n    <meterReadingLowProdu>2570472</meterReadingLowProdu>\n    <isSmart>1</isSmart>\n    <lowestDayValue>0</lowestDayValue>\n    <solarProducedToday></solarProducedToday"..., 1072) = 1072
write(3, "<action class=\"invoke\" uuid=\"eneco-002-XXXXXX:happ_pwrusage\" destuuid=\"a0b6a2690-f343-11e1-a21f-0800200c9a66\" serviceid=\"urn:hcb-hae-com:serviceId:specific1\">\n <u:UpdateDataSet xmlns:u=\"urn:hcb-hae-com:service:specific1:1\"><powerUsage><value>890</value>\n

Based on that last write line, i constructed a XML that would update a "known" dataset and destination uuid. Something in the lines of:

Code: Select all

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<action class="invoke" uuid="eneco-002-XXXXXX:happ_pwrusage" destuuid="a0b6a2690-f343-11e1-a21f-0800200c9a66" serviceid="urn:hcb-hae-com:serviceId:specific1" commonname="eneco-002-XXXXXX" xmlns:u="urn:hcb-hae-com:service:specific1:1" 
    <u:UpdateDataSet>
    	<powerUsage>
        <Value>8000</Value>
        </powerUsage>
    </u:UpdateDataSet>
User avatar
RDNZL
Forum Moderator
Forum Moderator
Posts: 1008
Joined: Sun Sep 24, 2006 1:45 pm
Location: Dordrecht, The Netherlands
Contact:

Re: Toon app: Zon op Toon using PVoutput

Post by RDNZL »

While having a look too, I found a (bit offtopic) url which returns alot of data:

Code: Select all

http://x.x.x.x:10080/hcb_config?action=getPackageConfig.json&package=happ_pwrusage
http://x.x.x.x:10080/hcb_config?action=getPackageConfig.json&package=happ_thermstat
etc
Your port can be different.
I didn't post the results, because it seems to contains a lot of personal data.
The resulting json contains items like:

Code: Select all

{daysCountBoilerWasActive":"28"}
{"realCost":"106.46"},{"realLowCost":"81.29"},{"benchmarkResult":"0"},{"visitedMonth":"1"},{"package":"happ_pwrusage"},{"uuid":"eco-001-xxxx:happ_pwrusage_xxxx"},{"type":"elec"},{"internalAddress":"elec_11-116"},{"visibility":"0"},{"mostExpensiveDay":"11"},{"mostExpensiveDayUsage":"24809"},{"worstCaseMonthCost":"172.55"}
Whats the deal with all that dojo stuff in the html code, it seems not complete?

Code: Select all

<script type="text/javascript" src="/inc/dojo-new/dojo/dojo.js" djConfig="parseOnLoad: true, usePlainJson: true"></script>
<script type="text/javascript" src="/inc/dojo-new/dojo/dojo-hcb.js"></script>            
                                                                                                         
<script type="text/javascript">                                                                                             
        dojo.require("hcb.utils");                                           
        dojo.require("hcb.translate");  
Regards, Ron.
Ierlandfan
Member
Member
Posts: 151
Joined: Thu Oct 03, 2013 7:53 pm

Re: Toon app: Zon op Toon using PVoutput

Post by Ierlandfan »

Thanx! The xml revealed the name of the value to be updated! Try it with /hcbv2/bin/bxt - c - d *UUD* -a value produced -v some-value. I hate not having the exact command but tablet is empty so I only have my phone now :-) bxt will give you some output being ok or error on that. I am not sure on the -v but invoke bxt with no options and correct them if necessary.

Edit: we have to construct the xml...out of my head that's another option. At least we have the xml and value name so not a real problem. Well done!

Question is: is Toon subtracting the values from the meter readings...that would be interesting since mine is reversing while producing Energy..I have a non-smart electricity meter...and intend to keep it that way...no alu-head just economically.
TheHogNL
Forum Moderator
Forum Moderator
Posts: 2125
Joined: Sun Aug 20, 2017 8:53 pm

Re: Toon app: Zon op Toon using PVoutput

Post by TheHogNL »

On a side note. I recently did something else to get my solar output visible on the Toon.

I have two seperate solar installations which are impossible view togheter using the normal Toon solar meter adapter. They are not on the same kWh meter.

I created a script on a raspberry pi which calculates the total generated solar Watts (which it gets from the seperate kWh meters per each installation, but the source can ofcourse be anything which reports the current Watts of each installation) and then flashes a red LED in just the right amount of interval. The Toon solar sensor is directed towards this red LED and now the Toon is showing the right amount of generated Watts.

A bit crude but it works :D
Member of the Toon Software Collective
Post Reply

Return to “Toon Apps”