Solved

Can we get access to the live usage API?


Userlevel 2
Badge

Hi Ovo,

I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

For instance (more sport related), Strava and Fitbit.

icon

Best answer by Tim_OVO 27 January 2020, 11:06

I’ve got our official stance on this from our dev team:

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking and our product teams are focused on building a great online experience and working with our members towards our Plan Zero initiative. When this changes in the future we’ll announce it on this topic first.

I’ll stick this as the ‘best answer’ so others know. 

Thoughts below please! 

View original

141 replies

Userlevel 3

… Any script which tries to lift data from the online Usage pages will need to be continually updated to match the current Billing Software version.

To make it even more tricky, it’s becoming apparent that not all customers are running the same software version. See this discussion about the Zero-in option.

 

Yes any script will need to be updated as the online interface changes, but I have only updated mine once in the last 3 years., and this latest one still works for me.  I don’t know about zero-in and am not interested as I have a near zero-energy new build and track all of my energy use through my own home automation system.

As to @uvarvu ‘s Q I use a password keeper / generator for my online passwords which generates strong passwords and these will often need quoting using the r’’’password’’’ syntax as below:

 PASSWORD = r'''D#adR'xGFj{Ks";=$'''

Also note that I have an electricity-only supply and the script may need tweaking for multiple supplies.

Userlevel 7
Badge +3

Can I point out that the “new” Billing Software, which is the connection point to our Smart Meters, is still undergoing development and bug-eradication. Any script which tries to lift data from the online Usage pages will need to be continually updated to match the current Billing Software version.

To make it even more tricky, it’s becoming apparent that not all customers are running the same software version. See this discussion about the Zero-in option.

My current Online Usage interface has changed further since that discussion. Not only do I not have a Zero-in Option, but there is no longer the ability to choose Yearly or Monthly from the Usage entry on the let menu.

 

I’m afraid we’re chasing a moving target.

@TerryE I tried to run your script but it’s erroring for me - something is wrong with the authentication.   Does the password field need to be in a certain format?   I left all of the quotes in place.

 

pi@raspberrypi:~/Documents $ python2 energy.py 
Date,00:00,00:30,01:00,01:30,02:00,02:30,03:00,03:30,04:00,04:30,05:00,05:30,06:00,06:30,07:00,07:30,08:00,08:30,09:00,09:30,10:00,10:30,11:00,11:30,12:00,12:30,13:00,13:30,14:00,14:30,15:00,15:30,16:00,16:30,17:00,17:30,18:00,18:30,19:00,19:30,20:00,20:30,21:00,21:30,22:00,22:30,23:00,23:30
{"message":"No refresh token cookie or authorization header sent","redirect":"https://my.ovoenergy.com"}
Traceback (most recent call last):
File "energy.py", line 86, in <module>
main()
File "energy.py", line 78, in main
ovo = OVOreadings(USERNAME, PASSWORD)
File "energy.py", line 50, in __init__
self.account = json.loads(resp.content)['accountIds'][0]
KeyError: 'accountIds'
pi@raspberrypi:~/Documents $

 

Badge

For those interested in something like this being easily accessible I’ve made a terrible walkthrough of how to do so. hope you don’t mind me posting this Tim.

https://github.com/ThePaulAdams/OvOEnergy

https://www.thepauladams.com/blog/2020/02/24/ovo-energy-integration-to-smart-home-dashboard/

I’m just looking at switching energy providers and access to this data is a prerequisite.

If this data exists then surely there is a right to be able to access it (and not a simplified form) under GDPR?
I’ve got a Chameleon Technology IHD6-CAD-PPMID smart meter which can send the real time data via wifi.
Bulb offer an integration with smarthings suggesting that the realtime data is accessible.
But I’m hoping to find another provider that are slightly cheaper. Hence why I looked at OVO.

 

Userlevel 3

 

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking

 

Thoughts below please! 

I’ve just spent another 4 fours re-engineering the Smart Meter API, and updated my python automatic script.

Anyone that is interested can use this python script; just update the user name and password.  This queries against the new API and unlike the previous version which output to a MySQL schema, this simply produces a CSV so users can also paste the O/P into Excel, Calc or whatever. :relaxed:

 

Anyone unhappy with the OVO developers position is free to use my Python script which wraps the normal Web interface to achieve the same.

That is really disappointing to hear, that you’re not going to make it easy for us to get our data from your system. 

 

I don’t think anyone is wanting anything particularly complex. a url which we can hit, which gives us the last 30 days of data (ideally with something like a 5 minute granularity) would be perfect. No requirement to go for an oauth tie-in or anything like that. A simple long token which can be reset. 

 

 

Either that, or a way to monitor our smart meters directly. I’ve heard something about them using zigbee, though I have no idea how valid that is. Any thoughts on that?

Userlevel 1

I am also working towards a plan zero initiative:

I’d like to be able to track energy usage through the day, not just a daily, weekly, monthly graph so I know where to optimize.

I’d like to be able to create charts using something like Graphana and map the energy data against light levels, temperature, humidity and other atmospheric conditions so I know when there are anomolies.

I’d like to combine the energy usage data with my smart thermostat to better understand the relationship between temperature and cost.

I’d like to set up an alert the moment my smart meter stops posting data so that I’m not sent an email months later requesting a manual submission.

I am not alone having these ideas and being capable of implementing them. But I will not scrape the data. I will simply move to a provider who does support an API.

Userlevel 1

That’s a great shame.  Ovo used to be leading in the area of engaging digitally with its customers and this seems to be a backwards step.

Userlevel 7
Badge +2

I’ve got our official stance on this from our dev team:

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking and our product teams are focused on building a great online experience and working with our members towards our Plan Zero initiative. When this changes in the future we’ll announce it on this topic first.

I’ll stick this as the ‘best answer’ so others know. 

Thoughts below please! 

Userlevel 7
Badge +2

I’ve already been speaking to the dev team this week (most recently yesterday) about what is required, how feasible, and when etc  @mattc 

 

I’ll be able to get an update shared next week. Thanks for bearing with - this query has not been forgotten. :)

Userlevel 1

Just make an official API already. Sorry to get short but people hacking their way to relevant data shows there is a need for it and you’re an energy company, I doubt you’re getting rich cos of your apps... 

https://segment.com/blog/why-we-re-transitioning-to-api-first/

Userlevel 7
Badge +2

Update: I’m still looking into this, @TerryE  - what we can offer, what we can’t and why. It would be good for everyone interested to get some clarity. Leave this with me!

Userlevel 3

I’ve just spent another 4 fours re-engineering the Smart Meter API, and updated my python automatic script.

Anyone that is interested can use this python script; just update the user name and password.  This queries against the new API and unlike the previous version which output to a MySQL schema, this simply produces a CSV so users can also paste the O/P into Excel, Calc or whatever.  I run this as a cmd from NodeRED and parse the response in a NodeRED function.  Enjoy.  :relaxed:

@Eva_OVO this is unlicensed Open Source and my original work.

Userlevel 5

We’ll take that away, @TerryE, and get a response from our dev team. We’ll be in touch when we have more info! :)

Userlevel 3

Further to @ErTnEc's git contribution, I have updated this to do the V2 API for historic consumption. I run the following script daily to download the previous day's half-hourly consumption into the MySQL DB that underpins my NodeRED installation: load_readings.py.
 

 

The new online interface has broken this unofficialt API :(  @Tim_OVO , any traction with the dev team.   Under the old scheme, the cookies were generated server-side, so you could simply script the logon, plus the API request. Now some of the session cookies are generated in obfuscated JS code client-side.  I can log on with the browser and capture these session cookies to part automate this, but this is a total PITA. 

Any chance of opening a dialogue with one of the dev team?  There is no reason why OVO could publish a Python script to make this API easily accessible to advanced users.

+1 for the API trial please.

Userlevel 1
Badge
Version one of the python library for the Hildebrand API. May well be a bit sketchy.

https://github.com/ghostseven/Hildebrand-Glow-Python-Library
Great, keep us updated on the python code please.
Userlevel 1
Badge
Think its 0330 303 5063
OK, will give them a call, don't really want to purchase another Zigbee hub and have two running if I can get away with one. Thanks for the info.

Do you still have their number please ?
Userlevel 1
Badge
Hi jdenver, I am not sure, I can only comment on the Hildebrand one, I purchased it from them and they have an agreement with OVO to be able to white list the devices on the HAN network. They did this from their end and the device connected automatically.

However prior to this I did talk to a rep over the phone at OVO and they were prepared to unlock the HAN to allow me to pair. I did not need to in the end but they were able to, they did ask what device I had so its worth a call. Dont use twitter / social media as via that method I was told it was not possible due to 'security' reasons.
Interesting 'blakedrayson' I have a ConBee II usb Zigbee stick, can't that be used?
Userlevel 1
Badge
You can work around this now, I have just done it. You can buy and pair a third party HAN very easily. In fact the Hildebrand Glow HAN can be paired by them without you having to ask OVO to unlock your smart meters.

There is access to an app and an API for getting real time data and historical data. The API is ok (it falls down in places but can be worked around). I am in the process of writing a Python class interface with the API that sorts out some of the oddness.

Hildebrand as a company appear to be quite responsive and helpful and the setup was a doddle.

Will post back here when I have the Python code up on GitHub.

**UPDATE**
Version one of the python library for the Hildebrand API. May well be a bit sketchy.

https://github.com/ghostseven/Hildebrand-Glow-Python-Library
Count me in! I’d love to get access to the API to build a component for HomeAssistant.
Me too, I'm looking for Home Assistant integration. Come on OVO....

Reply