Unfortunately @SianiAnni , that integration looks like it’s a community contribution, for which the source code and GitHub Repository can be found at https://github.com/timmo001/ovoenergy .

It doesn’t look like it’s anything OVO’s done in an official capacity either. If you need help with that integration specifically, it might be worth creating an Issue on that repository.

I’m not really a coding expert or a developer myself, but I’ll try to read the code overnight and see if I can figure it out. It’s only Python, so it’s not too hard for me to read it!

Thanks for pointing me in the right direction @Blastoise186!  I can see that there was a minor change to the code just 10 days ago, although I haven’t figured out whether it’s relevant, and there is no indication in Home Assistant that there’s an update available, so will raise a query via the Home Assistant forum. They’re a helpful bunch, much like the people on here:)

I have home assistant and the OVO integration. In my case it shows yesterdays gas and electricity consumption in kWh and £. If you want more data I’d have a look at Hildebrand for their glow stick https://shop.glowmarkt.com/products/glow-stick which you can integrate with Home Assistant.

Ok, I’ve spent some time reviewing the code behind the integration, both on the Home Assistant repository itself on GitHub and the upstream repository for which it gets most of the functionality from.

Please don’t forget that I’m no coding expert however and I can’t provide any kind of support for something like this. Fortunately, it’s only Python so it’s not exactly impossible to figure out what it does.

By the looks of things, the integration appears to basically be pulling data straight from the same OVO API that MyOVO uses to show you all the data in your online account. The only significant differences are that it strips out the UI and pulls raw data directly.

On the plus side, at least it’s not technically scraping the website itself, which would be pretty load generating and fairly wasteful. The downside however, is that the way the integration works means that it does technically have full access to your online account in MyOVO. Make sure to be cautious and don’t use the integration if you’re not fully confident that you can trust the code and the developers behind it.

That’s probably about as much as I can contribute though I’m afraid. But I hope it helps either way

Thanks for delving into the code:) No doubt the same API that many of us have been asking for for years!

Before seeing this  I had managed to confirm that some of the kWh values match those for the daily values as shown on MyOVO.

But I don’t understand what triggers the integration to poll MyOVO. There seem to be three data points for gas yesterday (12th), the first and second were the same gas usage for the 11th as shown on MyOVO, the third at 23:23  was a little higher, and the only one for today was just at 8:43 this morning and corresponding to the MyOVO value for 12th. The latest electricity value in both HA and MyOVO is for 11th - no data on either for yesterday, the 12th.  All seems a bit random...  But it will be useful to check gas usage against number of hours Hive switches boiler on. I’m not too interested in electricity usage at the moment. 

Digs through even more GitHub repos… 

Well… Based on this file https://github.com/home-assistant/core/blob/dev/homeassistant/components/ovo_energy/sensor.py I found a clue or two. Line 14 in particular seems to be relevant.

If I’m understanding it all correctly, it appears to trigger using a timer based schedule that fires up the polling roughly every five minutes. Personally, I’d say that’s a little bit too aggressive for my liking, especially if the data isn’t changing that often and polling the API 288 times a day per account is an awful lot of requests smashing the server to pieces! But it is what it is I guess...

If I had access to the server logs, I could probably look through those and identify activity from this stuff fairly easily which would make it a piece of cake to figure out the polling frequency. Alas, that’s also something I can’t do as a volunteer! 

Bit of bad news I’m afraid.

Thanks for delving into the code:) No doubt the same API that many of us have been asking for for years!

Unfortunately, I’m not sure that’s what this particular API is intended for and I get the feeling it’s intended for internal use only within MyOVO right now. If it was designed to be opened up, I would expect to see things like API Keys, permission scopes and possibly OAuth2 to ensure security while still allowing third-party apps to have access to just the stuff they need.

However, this doesn’t seem to be the case right now. The API just seems to be used for connecting the back end systems to MyOVO via a secure setup.

I actually happen to know of someone who could potentially stop by and leave his thoughts on this topic - who happens to be following me on the forums as well! Unfortunately however, he’s a bit shy and doesn’t like to reveal himself. You won’t be able to guess who he is however, my followers list is private and only myself and the Mods can see it. 

I can’t see where the 5 minute constant is used. But I completely agree that for daily readings it doesn’t need to be that often. 

Hang on…

Of course… I was looking at the wrong file! This probably shows you why I’m not an expert here! Lines 70 to 77 in this other file are relevant: https://github.com/home-assistant/core/blob/dev/homeassistant/components/ovo_energy/__init__.py

But once again, seems to confirm a polling interval of every five minutes… It might as well be a WebSocket if it’s going to be that frequent! 

The code for the integration itself does appear to be capable of doing half-hourly mode, but it appears to be disabled by default. I suspect this is intended to allow it to pull down the half-hourly readings data if such data exists in MyOVO.

The problem there however, is that even the Half-Hourly readings from a smart meter don’t get uploaded every 30 minutes! The meter(s) do of course capture readings every 30 minutes on that setting, but it’s still only submitted on a daily basis, shortly after midnight. So in effect, polling the API every five minutes is only either going to do one of the following:

• Get new data, but only if it’s the first poll after MyOVO has been refreshed with the latest meter data
• Get the same data that’s already been polled for 50+ times that day
• Return a server error of some kind, such as HTTP 404 Not Found or HTTP 500 Internal Server Error because you crashed it…
• Crash something and completely break
• Get itself rate-limited and keep slamming into HTTP 429 Too Many Requests
• Potentially get your online account in MyOVO flagged by any anti-abuse mechanisms, which might result in you getting locked out

It’s a pretty interesting concept they’ve got there, but I definitely think it would be a lot more appropriate - and a lot more respectful to OVO - if that integration were to perhaps slow down a bit? I can’t speak officially on behalf of anyone, but I’d say reducing the polling interval to  maybe once per hour or perhaps even once every 6 or 12 hours would be worth considering. Possibly also with some other protective measures in there too, in order to help prevent all these bots from smashing up the API all at the same time?

I did have a look at expanding on this integration but gave up because of the limitations and reliability. I switched to Hildebrand and using their MQTT stream because it allowed so much data and manipulation of that data to create Home Assistant sensors and meters. I get ½ hour gas readings and near real time electric readings. The only thing it doesn’t support natively (but there is an easy work around) is export readings from the smart meter.

Ok, thanks @Blastoise186 and @ArundaleP for info. Looks like I should disable the Ovo integration .   

Hi all, only just joined the forum and I don’t yet have a smart meter fitted (last attempt by Ovo engineers was rather a disaster! A few years ago now though. The problem I have now is that with working from home full time, finding a time to allow the power to be taken out is a major problem).

However, I am a contributor to Node-RED and I’ve previously converted Drayton Wiser python API code to JavaScript so I’d be interested in picking this up if I can ever get a new meter set installed.

In terms of the existing Python code, I recommend raising an issue on GitHub to get the author to tone down the request schedule. In fact, I’ll go and do that now while I think about it.

Of course, it would be a LOT better for Ovo to actually publish a proper API. It isn’t that hard though of course it would need a security test. 

Issue raised: 

Polling rate is far too aggressive - risks getting blocked & is pointless anyway · Issue #21 · timmo001/ovoenergy (github.com)

Thanks! I don’t know why I didn’t raise an Issue a while ago. XD

But yeah, unfortunately it’s not worth smashing up OVO’s API 48 times a day if you’ll only get the same data back. In the words of another website I know of “If you need data on all nations in a particular region at once, please use the Daily Dumps rather than spamming the Shards API with a million pointless requests and then just use the Shards API to get up-to-the-minute information for a specific nation”.

The way that smart meters send data back to suppliers isn’t controlled by suppliers as much as it’s controlled by the SMETS1 and SMETS2 specifications themselves. If the data really was being sent to the supplier every 30 minutes, it would be a huge amount of data and there’s no guarantee that enough people would actually be looking at it often enough to justify the massively increased resource usage.

If you are going to develop tools like this, I would probably also suggest that you try to spread the load out as best you can so that not everyone tries to hit the API all at once. Randomising by at least an hour should help, but I’d also argue that randomising over at least three hours is a bit more polite since that will really help to keep the spikes down.

Strictly speaking, it’d also be better to register yourself as a DCC Other User (somehow!) and pull real-time half-hourly data directly from a specific smart meter via DCC than it would be to smash up a supplier that can only get you snapshots that were uploaded the previous day. I’ve got no idea how you can register with DCC however, so I’m afraid you’d be on your own there.

I already made the point in my issue about not having a fixed time so that everyone hits the API at the same moment.

Not sure I agree actually about connecting direct to the source. Mainly because it would mean that you couldn’t get pricing data I don’t think.

In any case, my contract would not be with them, it is with Ovo. So you need to connect to Ovo has they have the accountability for the data. Specifically, they have the accountability for MY data. 

It would be interesting to see a privacy case if the DCC system exposed lots of data. It would be an odd situation since they presumably only have the physical address and the meter details. Do they have personal details? I would hope not.

At the end of the day, Ovo aren’t a small company any more and the cost of running web-based API’s is stupidly cheap these days. Once you’ve set one up, the cost of scaling, especially if using global cloud infrastructure, is so low as to make it a no-brainer. Especially given that you can use the same API for your own apps (both web and mobile). As one already exists, the additional cost of making it scalable should be minimal (assuming it was designed correctly in the first place). The further cost of adding a suitable open IDaM solution like OAuth is also not really very high. I would warrant that Ovo are making a reasonable margin and the benefits of being customer responsive are considerable. So almost certainly, it would be a good investment on every front.

Great points raised here about the benefits of a public API, @knight.

This might be a good place to introduce Kaluza, the tech branch of the OVO group. They’re the ones responsible for building our new Orion billing platform and they’ve assured us a public API is in development - we can’t say when yet, but you can count on us to shout about it once it’s launched!

In the meantime check out this thread for a member-created workaround - 

Thanks to @Jequinlan, @TerryE, and @g-de for your amazing work on that!

Cool, thanks for sharing that. I will certainly keep an eye on things and assuming I can find some time, I expect that I’ll at least highlight the API with an example or two in the Node-RED forum when it is live  Maybe even write a wrapper if it is worth it.

I’ll keep that article on bookmark and if I get some time, I’ll give it a go as well. The nice thing about Node-RED is how ridiculously easy it is to write a flow around a web request and present the results in a UI or use them in another process.

Just a quick update for ya!

timmo01 did respond to us but suggested that the issue in question be redirected to Home Assistant rather than himself. I’ve gone ahead and opened a new Issue in the main Home-Assistant/Core repository where it’ll hopefully get more attention. You can find that right here:

OVO Energy polling is excessively aggressive #51496

While it’s simple enough to swap out a single variable to match 24 hours, I’m not skilled enough to actually submit a Pull Request that changes the entire polling behaviour outright to play nice. That’s way beyond my skills!

Mind you, the way they’ve got probot set up is pretty neat! Credit where it’s due there! XD

Hmm…

No response to the Issue so far…

I might go ahead and PR this for now to try and get it down to something more reasonable, even if not perfect.

Dropping it down to once per hour is about as far as I’m willing to go in the meantime.

Created a PR, I went with 12 hours as I think it’s more realistic.

https://github.com/home-assistant/core/pull/51613

Turns out that hourly seems to be more acceptable with HA, so I’m happy to call it a deal. If anyone knows how to make the schedules even smarter later on so that it can be even more friendly, that would be great.

But in the meantime at least, I get the feeling that HA polling OVO’s API once an hour per user is probably nowhere near as bad as polling it every five minutes per user. That seems to be the sweet spot for now at least.

By the way, here’s a fun little fact actually.

This clever trick does some really cool stuff.

The word “fixes” in this case is a magic word that does something useful (similar words like fix, close, closes etc also work too!). If PR #51613 gets Merged, it will automatically close out Issue #51496 as resolved at the same time - but ONLY if the PR gets Merged. If the PR gets Closed instead, the Issue remains Open.

It’s a really neat way of helping to magically clean up the lists in one go, on the basis that once a PR fixes an Issue, the issue is no longer an Issue and is therefore no longer needed since it doesn’t need to remain open anymore.

My skills are only really enough to do the most basic stuff like changing very simple code which controls the polling rate in this case. But even small stuff helps! And in actual fact, this PR is so tiny that it seems like I’m able to skip some of the usual checks that would need to take place i.e. actually doing all the full testing and automation stuff.

I still have to wait for a Maintainer to review the PR and Merge/Close/Review it anyway as I don’t have the ability to Merge code directly to their repository.

Oh, and there’s also the small matter of Reviews, Approvals and Checks too. Everything down here MUST go green before you can Merge a PR. Technically speaking, there are cases where the Owner of a Repository or Members of an Organisation might be able to override and break through a red/pending status, but you generally want to avoid doing that.

And finally, this Verified status helps to prove that my code change was committed by me via GitHub’s web editor (GitHub signs my commits on my behalf when I do it this way), the code hasn’t been tampered with by anyone else and acts as a way to prove my identity as the author of the commit in question. It also helps to act as an audit trail too.

But here’s the cool thing. If they ask me to make changes to the PR, I can simply do them on my Fork under the ovopollfix Branch and when I commit the changes, the PR gets updated automatically with them so that I don’t have to touch it.

And that is something, you might not have known. XD

Tada!

All done. :)

My PR has been merged and is planned to be included in the next update to Home Assistant Core. I don’t know when that’ll go out, but when it does, my fix will be included. From there, once users update their own instances, they’ll slow down the polling to once per hour rather than once every five minutes. This will result in each HA instance massively slowing down from 288 pings a day to just 24 pings a day. Much nicer!

Trust me @Tim_OVO , you’ll thank me later. I bet this will also make a few of your friends happy too. XD

Nice one, @Blastoise186! 

Do we know of any communtiy members here using this integration? I think @ArundaleP mentioned they were in another thread, despite this:

And @SianiAnni was close to. 

Just an FYI to you both and anyone else, if you ‘update your instance’ this change will kick in. @Blastoise186 is it obvious how to do this? 

Yup, I do indeed.

Please bear in mind that right now, my code changes have been Merged to the Dev branch of the Home Assistant Core repository and have not yet been released. It’s been added to the 2021.6.4 Milestone as seen in PR #51613 and will go out when that release is published. They only just released 2021.6.3 yesterday, so it might be a few days yet but it’ll go out soon!

As for updating, that’s pretty easy really, especially if you installed HA from the Snap Store. Assuming you haven’t told snapd to pin the home-assistant-core-snap Snap to any specific releases or disabled auto-updates, you can simply let snapd run an auto-refresh once the update is published on the Snap Store - HA will then be updated automatically and you don’t have to touch anything. Alternatively, you can force snapd to refresh all Snaps immediately if you want to - the command for that is sudo snap refresh in Terminal. You need to have sudo permissions to do this (you will if you’ve got admin rights). And then just give it a few moments. To check on the status of recent updates, simply do snap changes - you don’t need to use sudo for this command. And to monitor a specific task, it's sudo snap watch X replacing X with the task number you’d like to monitor.

I used the HA Ovo integration briefly, but didn’t find the HA lovelace display particularly helpful, as it only changed once or twice a day (the twice a day update looked like some kind of glitch).

I also use a HA Hive integration which shows length of time gas boiler is ON. 

I also already have access to gas and electricity half hourly data via a Carbon Co-op members service, which I can download as a csv file, and then do with as I like.

Would be interested in giving the HA Ovo integration another go though. Took delivery of yet another pi yesterday, with HA installed and configured for another Carbon Co-op prototype service.  I now have seven Aqara temp sensors connected and am building up a really good picture of how the house responds to weather and heating! Now I just need to get that retrofit done...  

No worries :)

I think it’d definitely work better when OVO releases a Public API in the future - which is something that’s on the to-do list from my understanding.

The maintainer behind the ovo_energy integration on GitHub appears to be pretty active as well, so I’m confident he would be happy to update the integration to use any future Public APIs rather than the current one.

Just as a heads up though, as this stuff isn’t official, OVO’s teams can’t help with setting it all up, fixing bugs and offering support. The best places to go for that would be via GitHub and you can find all the docs you need via the links over there.

These are the two repositories you’ll be most likely to want to check out.

Home Assistant Core - GitHub - for anything to do with Home Assistant itself, this would be the place to check out first. This is also the best place to ask for the polling rate on the ovo_energy integration to be tweaked if needed in the future.

ovo_energy HA integration - GitHub - for anything related to the ovo_energy HA integration, this is the place to go.