Ovo integration for Home Assistant - Unofficial API

Ok, thanks @Blastoise186 and @ArundaleP for info. Looks like I should disable the Ovo integration .   

Hi all, only just joined the forum and I don’t yet have a smart meter fitted (last attempt by Ovo engineers was rather a disaster! A few years ago now though. The problem I have now is that with working from home full time, finding a time to allow the power to be taken out is a major problem).

However, I am a contributor to Node-RED and I’ve previously converted Drayton Wiser python API code to JavaScript so I’d be interested in picking this up if I can ever get a new meter set installed.

In terms of the existing Python code, I recommend raising an issue on GitHub to get the author to tone down the request schedule. In fact, I’ll go and do that now while I think about it.

Of course, it would be a LOT better for Ovo to actually publish a proper API. It isn’t that hard though of course it would need a security test. 

Issue raised: 

Polling rate is far too aggressive - risks getting blocked & is pointless anyway · Issue #21 · timmo001/ovoenergy (github.com)

Thanks! I don’t know why I didn’t raise an Issue a while ago. XD

But yeah, unfortunately it’s not worth smashing up OVO’s API 48 times a day if you’ll only get the same data back. In the words of another website I know of “If you need data on all nations in a particular region at once, please use the Daily Dumps rather than spamming the Shards API with a million pointless requests and then just use the Shards API to get up-to-the-minute information for a specific nation”.

The way that smart meters send data back to suppliers isn’t controlled by suppliers as much as it’s controlled by the SMETS1 and SMETS2 specifications themselves. If the data really was being sent to the supplier every 30 minutes, it would be a huge amount of data and there’s no guarantee that enough people would actually be looking at it often enough to justify the massively increased resource usage.

If you are going to develop tools like this, I would probably also suggest that you try to spread the load out as best you can so that not everyone tries to hit the API all at once. Randomising by at least an hour should help, but I’d also argue that randomising over at least three hours is a bit more polite since that will really help to keep the spikes down.

Strictly speaking, it’d also be better to register yourself as a DCC Other User (somehow!) and pull real-time half-hourly data directly from a specific smart meter via DCC than it would be to smash up a supplier that can only get you snapshots that were uploaded the previous day. I’ve got no idea how you can register with DCC however, so I’m afraid you’d be on your own there.

I already made the point in my issue about not having a fixed time so that everyone hits the API at the same moment.

Not sure I agree actually about connecting direct to the source. Mainly because it would mean that you couldn’t get pricing data I don’t think.

In any case, my contract would not be with them, it is with Ovo. So you need to connect to Ovo has they have the accountability for the data. Specifically, they have the accountability for MY data. 

It would be interesting to see a privacy case if the DCC system exposed lots of data. It would be an odd situation since they presumably only have the physical address and the meter details. Do they have personal details? I would hope not.

At the end of the day, Ovo aren’t a small company any more and the cost of running web-based API’s is stupidly cheap these days. Once you’ve set one up, the cost of scaling, especially if using global cloud infrastructure, is so low as to make it a no-brainer. Especially given that you can use the same API for your own apps (both web and mobile). As one already exists, the additional cost of making it scalable should be minimal (assuming it was designed correctly in the first place). The further cost of adding a suitable open IDaM solution like OAuth is also not really very high. I would warrant that Ovo are making a reasonable margin and the benefits of being customer responsive are considerable. So almost certainly, it would be a good investment on every front.

Great points raised here about the benefits of a public API, @knight.

This might be a good place to introduce Kaluza, the tech branch of the OVO group. They’re the ones responsible for building our new Orion billing platform and they’ve assured us a public API is in development - we can’t say when yet, but you can count on us to shout about it once it’s launched!

In the meantime check out this thread for a member-created workaround - 

Thanks to @Jequinlan, @TerryE, and @g-de for your amazing work on that!

Cool, thanks for sharing that. I will certainly keep an eye on things and assuming I can find some time, I expect that I’ll at least highlight the API with an example or two in the Node-RED forum when it is live  Maybe even write a wrapper if it is worth it.

I’ll keep that article on bookmark and if I get some time, I’ll give it a go as well. The nice thing about Node-RED is how ridiculously easy it is to write a flow around a web request and present the results in a UI or use them in another process.

Just a quick update for ya!

timmo01 did respond to us but suggested that the issue in question be redirected to Home Assistant rather than himself. I’ve gone ahead and opened a new Issue in the main Home-Assistant/Core repository where it’ll hopefully get more attention. You can find that right here:

OVO Energy polling is excessively aggressive #51496

While it’s simple enough to swap out a single variable to match 24 hours, I’m not skilled enough to actually submit a Pull Request that changes the entire polling behaviour outright to play nice. That’s way beyond my skills!

Mind you, the way they’ve got probot set up is pretty neat! Credit where it’s due there! XD

Hmm…

No response to the Issue so far…

I might go ahead and PR this for now to try and get it down to something more reasonable, even if not perfect.

Dropping it down to once per hour is about as far as I’m willing to go in the meantime.

Created a PR, I went with 12 hours as I think it’s more realistic.

https://github.com/home-assistant/core/pull/51613

Turns out that hourly seems to be more acceptable with HA, so I’m happy to call it a deal. If anyone knows how to make the schedules even smarter later on so that it can be even more friendly, that would be great.

But in the meantime at least, I get the feeling that HA polling OVO’s API once an hour per user is probably nowhere near as bad as polling it every five minutes per user. That seems to be the sweet spot for now at least.

By the way, here’s a fun little fact actually.

This clever trick does some really cool stuff.

The word “fixes” in this case is a magic word that does something useful (similar words like fix, close, closes etc also work too!). If PR #51613 gets Merged, it will automatically close out Issue #51496 as resolved at the same time - but ONLY if the PR gets Merged. If the PR gets Closed instead, the Issue remains Open.

It’s a really neat way of helping to magically clean up the lists in one go, on the basis that once a PR fixes an Issue, the issue is no longer an Issue and is therefore no longer needed since it doesn’t need to remain open anymore.

My skills are only really enough to do the most basic stuff like changing very simple code which controls the polling rate in this case. But even small stuff helps! And in actual fact, this PR is so tiny that it seems like I’m able to skip some of the usual checks that would need to take place i.e. actually doing all the full testing and automation stuff.

I still have to wait for a Maintainer to review the PR and Merge/Close/Review it anyway as I don’t have the ability to Merge code directly to their repository.

Oh, and there’s also the small matter of Reviews, Approvals and Checks too. Everything down here MUST go green before you can Merge a PR. Technically speaking, there are cases where the Owner of a Repository or Members of an Organisation might be able to override and break through a red/pending status, but you generally want to avoid doing that.

And finally, this Verified status helps to prove that my code change was committed by me via GitHub’s web editor (GitHub signs my commits on my behalf when I do it this way), the code hasn’t been tampered with by anyone else and acts as a way to prove my identity as the author of the commit in question. It also helps to act as an audit trail too.

But here’s the cool thing. If they ask me to make changes to the PR, I can simply do them on my Fork under the ovopollfix Branch and when I commit the changes, the PR gets updated automatically with them so that I don’t have to touch it.

And that is something, you might not have known. XD