Solved

Can we get access to the live usage API?



Show first post

204 replies

Userlevel 7

Wowza this is great. @g-de I’m going to make your latest comment the ‘best answer’, which will mean it appears at the top of this topic below the question. Good to see @Jequinlan willing and able to support! 

 

I will be checking in with our devs though, to see if there’s any implications for customers using this. It might not be something we can stop, but it might not be something we want to promote. The fact that I’m unaware just shows how limited my knowledge is. 

 

Either way, a brilliant post sharing some great work that’s aimed at helping others to monitor their energy usage (and other use cases?). That’s what this community is allll about :)

Userlevel 1

So I have installed the HomeAssistant on a virtual Pi and am indeed able to get some OVO data! As I am not a programmer, actually getting the data from the API isn’t my expertise but if someone can help then I AM the right (type of) person to be taking that data and doing something useful with it. @g-de can you help?

 

Thanks.

@Jequinlan Unfortunately I don’t think I’ll be much help on this. I’m not too familiar with Python, only enough to be able to read it and do basic stuff and my home assistant experience is all around setup and automation rather than data science. I suspect the data collection in home assistant may be too brief for this purpose (by default it only keeps 3 days I think).

That being said, there’s a data science section on the Home Assistant website which may be of use: https://data.home-assistant.io/

Alternatively, someone here can probably create a small python script using the library to output the data to CSV. Should make it fairly easy to import into something else then.

 

Wowza this is great. @g-de I’m going to make your latest comment the ‘best answer’, which will mean it appears at the top of this topic below the question.

Glad I could help!

 

I will be checking in with our devs though, to see if there’s any implications for customers using this. It might not be something we can stop, but it might not be something we want to promote.

I think the main issues would be the API changing unexpectedly and storage of the username and password. If OVO were to have provided some kind of developer API it would have allowed use of tokens and permissions to be configured for read-only access instead. Much more secure but quite a bit of dev and management effort for OVO.

It is unsupported and rather hacky so the choice will probably be to not promote it. Preventing it would be rather difficult. The open source community always seems to find a way around any restrictions :slight_smile:

Count me in also for any trials of a csv/xls download utility for usage and payment data.
+1 again.


My use-case?

Alerting me when the underfloor heating has come on unexpectedly while we're out, due to the frost stat being triggered. I could then decide to turn on the GCH heating remotely, saving on total energy use.


How?

Use an Ovo API to monitor for when the kW rate jumps by more than a threshold, for a period of time. This would indicate that a high-load device has turned on.

I'd then check whether the house is occupied, e.g. by connecting to Tado's location service or similar.

Finally I'd check if the weather service suggests it's a cold day outside.

If nobody is at home, and the load has jumped, it suggests the underfloor heating has turned on due to the frost stat. Send an alert to mobile phones, so we can decide to switch on Tado remotely to mitigate the underfloor heating.


Tech stack?

Raspberry Pi, Tado, Home Assistant, Node-RED, Python.
Userlevel 3

… Any script which tries to lift data from the online Usage pages will need to be continually updated to match the current Billing Software version.

To make it even more tricky, it’s becoming apparent that not all customers are running the same software version. See this discussion about the Zero-in option.

 

Yes any script will need to be updated as the online interface changes, but I have only updated mine once in the last 3 years., and this latest one still works for me.  I don’t know about zero-in and am not interested as I have a near zero-energy new build and track all of my energy use through my own home automation system.

As to @uvarvu ‘s Q I use a password keeper / generator for my online passwords which generates strong passwords and these will often need quoting using the r’’’password’’’ syntax as below:

 PASSWORD = r'''D#adR'xGFj{Ks";=$'''

Also note that I have an electricity-only supply and the script may need tweaking for multiple supplies.

Userlevel 7
I'm interested in making the current live usage more accessible - the page on the website is great but means physically going to look at it, so I'd like to have the current usage readily available on something such as my phone.

I think it would also be interesting to look at the historic usage on other ways such as day of the week and cross reference with freely available weather history to see if there are correlations temperature and gas usage.🆒


Our developers will love this feedback, Gary - keep it coming! Anyone else have any suggestions for how we could use your smart meter data?

Tim
Userlevel 7
Badge +2

As a small update, A rival who offers TOU have published a download file for rates, so I can start the work on the comparison model!

Userlevel 7
Right then, @atilagiant, @SianiAnni and @Deanohol - it's great that you're interested in being part of any live-data trials we have planned. So that I can get you added to our list of interested customers, send me a PM with your account number, name and DoB. I'll make it happen! 🆒

Tim
Userlevel 7
Hi Tim,
I too would like the ability to download my energy usage data as a csv / xls(x) file to complement the data I can get from my weather station and my PV Inverter data.

Please sign me up for any trials that are going.
Many thanks

Peter


In that case, @Peter_Mat send over a PM with your account number, name and DoB. I'll get you added to our list of testers! 🆒

Tim
Userlevel 3

For replicating this in other languages, the process seems to be as follows:

  1. Login by sending a POST request to https://my.ovoenergy.com/api/v2/auth/login with the following JSON in the body:
    {
    "username": "...",
    "password": "...",
    "rememberMe": true
    }

    From the response, save the cookies to use in all future requests.

  2. Next, send a GET request to https://smartpaym.ovoenergy.com/api/customer-and-account-ids . Be sure to save the items in the “accountIds” property (usually only one item).

 

 

The issue with caching the cookies is that authorisation cookies have a lifetime.  IIRC, it’s 31 days, so your code still needs to handle cookie expiry and do reauthentication.  You might find it simpler just to do the authentication every time or to have a separate (say weekly) process to refresh the authentication cookie.

Hi,

I'm more of a configurer than a coder but I'm very keen to extract historical data either directly from the meter or via your back-end web services. I'm using Honeywell Evohome to control heating and water in my house and I'd like to be able to measure the impact of its use on my energy consumption.

Ideal for me would be a domoticz (https://www.domoticz.com/) integration - domoticz is already proving to be a great way to bring together a number of different smart home activities to visualise and control them together.

Do the dev team have an idea of when this functionality is likely to be implemented?
Userlevel 1
You can work around this now, I have just done it. You can buy and pair a third party HAN very easily. In fact the Hildebrand Glow HAN can be paired by them without you having to ask OVO to unlock your smart meters.

There is access to an app and an API for getting real time data and historical data. The API is ok (it falls down in places but can be worked around). I am in the process of writing a Python class interface with the API that sorts out some of the oddness.

Hildebrand as a company appear to be quite responsive and helpful and the setup was a doddle.

Will post back here when I have the Python code up on GitHub.

**UPDATE**
Version one of the python library for the Hildebrand API. May well be a bit sketchy.

https://github.com/ghostseven/Hildebrand-Glow-Python-Library
Interesting 'blakedrayson' I have a ConBee II usb Zigbee stick, can't that be used?
Userlevel 1
Hi jdenver, I am not sure, I can only comment on the Hildebrand one, I purchased it from them and they have an agreement with OVO to be able to white list the devices on the HAN network. They did this from their end and the device connected automatically.

However prior to this I did talk to a rep over the phone at OVO and they were prepared to unlock the HAN to allow me to pair. I did not need to in the end but they were able to, they did ask what device I had so its worth a call. Dont use twitter / social media as via that method I was told it was not possible due to 'security' reasons.
Userlevel 4
Badge

Advanced programmers could no doubt script the whole session connection to recover the session cookies. I am one, but I haven't bothered.


Further to @ErTnEc's git contribution, I have updated this to do the V2 API for historic consumption. I run the following script daily to download the previous day's half-hourly consumption into the MySQL DB that underpins my NodeRED installation: load_readings.py.
There's an issue with NodeRED HTTPrequest which causes logon problems so I haven't got the native NodeRED version working. Sorry. It's not worth my spending time on this because I will moving supplier when my contract ends. I really like OVO, but a 30+% price hike is just too much, when other suppliers offer better deals.


Thanks for this. I haven't managed to get your load_readings.py script to run on Windows, but as long as I'm already logged into MyOvo, pasting a handcrafted url from your script certainly returns the data, which is great because the url which I was using last winter no longer works! I copy the displayed data into a python script which then parses and outputs something human readable and graphable into a csv file.
OK, will give them a call, don't really want to purchase another Zigbee hub and have two running if I can get away with one. Thanks for the info.

Do you still have their number please ?
Userlevel 1
Think its 0330 303 5063
Great, keep us updated on the python code please.
Userlevel 1
Version one of the python library for the Hildebrand API. May well be a bit sketchy.

https://github.com/ghostseven/Hildebrand-Glow-Python-Library
Userlevel 1

Just make an official API already. Sorry to get short but people hacking their way to relevant data shows there is a need for it and you’re an energy company, I doubt you’re getting rich cos of your apps... 

https://segment.com/blog/why-we-re-transitioning-to-api-first/

Userlevel 2
Right then, @atilagiant, @SianiAnni and @Deanohol - it's great that you're interested in being part of any live-data trials we have planned. So that I can get you added to our list of interested customers, send me a PM with your account number, name and DoB. I'll make it happen! 🆒

Tim


Tim - I have sent you a message and am keen to get involved to.. This sort of process/analysis is right up my street and is what I do for a day job (sort of.. Different industry)
Userlevel 1

That’s a great shame.  Ovo used to be leading in the area of engaging digitally with its customers and this seems to be a backwards step.

That is really disappointing to hear, that you’re not going to make it easy for us to get our data from your system. 

 

I don’t think anyone is wanting anything particularly complex. a url which we can hit, which gives us the last 30 days of data (ideally with something like a 5 minute granularity) would be perfect. No requirement to go for an oauth tie-in or anything like that. A simple long token which can be reset. 

 

 

Either that, or a way to monitor our smart meters directly. I’ve heard something about them using zigbee, though I have no idea how valid that is. Any thoughts on that?

Hi Lucy / Tom,

Am I too late to be added into the API trial ?

Has anyone got any datasets yet ?

Thanks
Steve
Hey there, I've just signed up as part of the community to see if I can join in on the trials - would you mind adding me if I send the required PM?

Thanks!

Reply