Page 8 of 9

Re: Toon app: PostNL

PostPosted: Mon Apr 06, 2020 9:28 am
by Toonz
Hi all,

for folks who have performance problems on Toon after installing the PostNL app:
Most issues are caused by huge inboxes, they way Toon is handling the JSON data is using quite some memory.
It certainly helps if you remove old packages from your inbox via the postnl website or their app on your phone/tablet.
Some users have never cleaned it, leading to very large datasets.

Secondly, if you do not have a lot of packages and still do not see any data, their could be an issue with the data itself.
I have to say that the contents of the JSON dataset can have an enormous variety of cases and data elements to be populated or not.
I'm pretty sure my Toon app has not covered all of the posible variations.
If this is the case, I am more than happy to have a look and update the app but you need to email me your JSON dataset (/tmp/postnl/POSTNL-Inbox.json).

Thirdly, if I have time, I can look at the option to completely abandon the use of JSON and treat the inbox as a simple text file.
This should greatly reduce the memory consumption when having huge inboxes but it is a lot of (inefficient) programming to do (no ETA).

Kind regardz,

Toonz

Re: Toon app: PostNL

PostPosted: Tue Apr 07, 2020 2:32 pm
by RdP
Can you share your code on the PostNL app, I like to have a look if it can be used for Home Assistant too?

Thanks in advance,

Rien

Re: Toon app: PostNL

PostPosted: Tue Apr 07, 2020 2:58 pm
by Toonz
RdP wrote:Can you share your code on the PostNL app, I like to have a look if it can be used for Home Assistant too?

Thanks in advance,

Rien

it's open source :-) : https://github.com/ToonSoftwareCollective/postnl

Re: Toon app: PostNL

PostPosted: Wed Apr 08, 2020 2:58 pm
by RdP
THaNK YOU !!

Re: Toon app: PostNL

PostPosted: Wed Apr 08, 2020 8:51 pm
by msx4ever
I have done some checking on my part. I only had one entry delivered in the json file and displaying on Toon. On the app there where two on delivered and one on going to be delivered today. Then I did a restart of the Toon (shutdown -r now) and after some time (I checked 2 hours later), I got two delivered and one going to be delivered. So it looks like that fetching the data is not going ok.

Re: Toon app: PostNL

PostPosted: Wed Apr 08, 2020 9:30 pm
by Toonz
msx4ever wrote:I have done some checking on my part. I only had one entry delivered in the json file and displaying on Toon. On the app there where two on delivered and one on going to be delivered today. Then I did a restart of the Toon (shutdown -r now) and after some time (I checked 2 hours later), I got two delivered and one going to be delivered. So it looks like that fetching the data is not going ok.

I am happy to investigate, can you email me your /tmp/postnl/POSTNL-Inbox.json file or put the contents in a PM if that fits?

Re: Toon app: PostNL

PostPosted: Thu Apr 09, 2020 7:26 pm
by msx4ever
It looks like the reboot did the trick. Now it is working as it should. Package was due to arrive today, also on Toon. Has been delivered today and Toon has that status as well now. I will keep an eye at it and let you know through a PM (with the file) when it is not working.

Re: Toon app: PostNL

PostPosted: Fri Apr 10, 2020 7:36 pm
by glsf91
Looks like the login at postnl is not working most of the time. But sometimes it succeeds. Looks like postnl has changed the security at the login a little bit (bot detection).
I see the problem and also have a solution I think.

Need to integrate it in the script and to test it for a little longer. Also when I try to much with failures, I will be marked as a bot. I then have to wait when it is reset again.
So this will take a few days (also other things to do).

Maybe it is an idea to show the date/time of the retrieved inbox file on the screen on the bottom (like I do with NL-alert). Then we can easily see if we still retrieving and see recent data.

So will be continued...

Re: Toon app: PostNL

PostPosted: Fri Apr 10, 2020 8:01 pm
by Toonz
showing the refresh date time was also on my mind. Will show that in the next release of the app with the renewed script when ready....

Re: Toon app: PostNL

PostPosted: Mon Apr 13, 2020 2:28 pm
by Toonz
Hi all,

version 1.1.5 is available in the ToonStore. This version is much more memory friendly, usable on many more Toons now.
Note that extremely large inboxes can still cause problems but the performance should be much better than before.

Changelog:
- showing the last successful inbox retrieval date/time on screen
- only show packages from the last month to improve performance
- significantly reduced memory footprint of the app, just only need some memory now when processing the inbox
- inbox retrieval process reliability significantly improved by implementing anti-bot detection by glsf91 (great job again)

Kind regardz,

Toonz

Re: Toon app: PostNL

PostPosted: Fri May 01, 2020 11:31 am
by msx4ever
Hello Toonz,

Works like a charm now!

Thanks.

Re: Toon app: PostNL

PostPosted: Fri May 01, 2020 5:01 pm
by Toonz
Hi all,

Version 1.1.6 is available in the ToonStore. The old version frequently failed inbox refresh cycles due to the anti-bot detection algorithms on their servers.
Glsf91 greatly improved the anti-bot detection, the new version has worked flawlessly on multiple toons for some days now. Great job!
It should update the inbox every 2 hours now (last successful update time is shown on the screen), at least until they further tighten the security.
The battle continues. :-)

Changelog:
- bypassing anti-bot detection much better (100% in the past days)

N.B. make sure your inbox isn't too big to handle for Toon. You can remove old shipments via your mobile PostNL app or via the postnl.nl website.

Kind regardz,

Toonz

Re: Toon app: PostNL

PostPosted: Sat May 02, 2020 3:39 pm
by Toonz
Hi all,

Version 1.1.7 is available in the ToonStore.

Changelog:
- adding two items to the configuration screen:
inbox refresh interval in minutes (20-1440)
number of months of history to be shown on the screen

N.B. make sure your inbox isn't too big to handle for Toon. You can remove old shipments via your mobile PostNL app or via the postnl.nl website.

Kind regardz,

Toonz

Re: Toon app: PostNL

PostPosted: Tue May 05, 2020 6:47 pm
by Toonz
Hi all,

another update, version 1.1.8.
Changelog:
- screen with packages is now scrollable.
- two different screens with received and sent packages
- further performance improvements because sorting routine isn't needed anymore.

Users which Toon 1 responded slowly with this app are invited to try again.

Kind regardz,

Toonz

Re: Toon app: PostNL

PostPosted: Thu May 07, 2020 10:08 pm
by Toonz
Hi all,

version 1.1.9 is available in the ToonStore.
Changelog:
- added setting to show packages with the custom name instead of the barcode (thanks Bonno)

KInd regardz,

Toonz