Yes!

Done that, but the DB is still empty. Could there be a Windows problem with storing data from an unsigned application to ProgramData? Perhaps change that to LocalAppData? I do have full control of ProgramData as an administrator, and I own the OvoData folder, but still ...

>The image went away, presumably to be edited by a mod to obfuscate the username. It’s back now ]

When I click read, there’s a brief delay with a status message Checking 2024, then the dialogue disappears (and the app closes, apparently). I’ve not seen anything in the box you highlighted.

There’s plenty of data available to download: I can see my own half-hourly data back to March 2017.

Would you prefer to continue this troubleshooting backstage? You could PM me. 

With the aid of @Firedog I have managed to work out why the program kept crashing.

I will be deploying a new version to GitHub later, watch this space.

Mea culpa: it looks as if the app was choking when it found no gas data for me. Mike was quick to spot the hurdle and remove it, so now I’ve run out of ways to break it.

I can really recommend Mike’s solution to anyone happy to work from an SQLite database.

I was tempted to say why, but I considered that your choice @Firedog to do so, for privicy reasons.

As I said earlier I intend to release the fixed version to GitHub shortly (hopefully tomorrow).

For those of you on a single fuel deal, please wait until I announce it’s release.

My solution also allows you to export the data to either CSV or Excel files.

/Mike

I have just release V1.0.1 to GitHub, this fixes the issue found by @Firedog when you only have electricity supplied by OVO.

Just download the zip file from https://github.com/MikeWilliams-UK/My-Ovo-Data/releases/tag/V1.0.1 and unzip it’s contents to any folder, then run OvoData.exe

/Mike

I know this is greedy of me but I don’t suppose you would consider making that into something embeddable in Node.js? If you did, it could be made available to Node-RED and Home Assistant users quite easily.

It's a WPF application written in C# so I am pretty sure it won't be an easy port to node.js, also I don't know node.js

The full source code is on GitHub, so you can go "fill your boots" there …

It also directly accesses the file system, which is prohibited in all browsers.

Just tried to update my useage data today and the fetching of my account id(s) is being blocked

Using postman after logging in and trying to get the data from https://smartpaym.ovoenergy.com/api/customer-and-account-ids

I get this in the html comming back

I have been using the HA Ovo add-on and it has now got me band from the  “smartpaym.ovoenergy.com/api/customer-and-account-ids” url it uses or it has been turned off, so the add-on no longer works in HA. Has any one else come a cross this?

Just fixed a bug in my Ovo data fetching program, one of the OVO api’s I was using has been deprecated.

You can find release 1.0.2 at https://github.com/MikeWilliams-UK/My-Ovo-Data/releases/tag/V1.0.2
 

The url has been blocked by Cloudflare, my app has suffered from it since Friday (19th April) presumably the old API has now been turned off.

There is a new API with a new ul which returns the same data, can the app that you are using have the url changed?

/Mike

… working fine here. Thanks!

I’ll have a look, thanks.

File system access is not an issue for Node.js and can be worked around even for a browser.

Update: Took me about 25 minutes with Node-RED to get a successful login, most of which was decoding your C# code (which I don’t write!). This is the login flow: 

Should easily now be able to get the rest of the data, I can already see that I can get my monthly data even though I don’t have a Smart Meter.

Nice work Ovo, easy enough to use the API. Excellent work Mike.

Update 2: I split the flow in 2 - the main reason is that I don’t have to keep logging in and querying the account data when testing. I have made the last part of the flow a JavaScript function node that will let me better reformat the data into something easier to show.

So next step is to finish that formatting and then produce a web page to show the data.

This is, of course, fully cross-platform and you can run it on anything from a Raspberry Pi to Azure or AWS and everything in between whether desktop or server.

Once I’ve got it to a better place, I will be posting to the Node-RED forum. I will add a link to that here.

Here is the link to the post in the Node-RED forum which also contains the flow for anyone to use.

https://discourse.nodered.org/t/ovo-energy-api-get-your-energy-data-from-ovo/87440

I’ve been blocked by Cloudflare using the official home assistant plugin. Let me be clear, this integration is the sole reason I’m still with OVO. Either this gets fixed or I go elsewhere

It’s a shame to hear you feel that way @pauldsmyth - It’d be a shame to lose you as an OVO customer due to something like this. 

I wonder if @Blastoise186 @Firedog and co may be able to offer any support or suggestions, using their vast and technical knowledge. Like Tim, my personal knowledge on this subject is basic.

I’ve just been catching up via this thread and can only redirect you again to the Best Answer by @Tim_OVO that explains the limitations and issues around how API is currently used by OVO and its sister companies. Support will be minimal as we don’t promote this particular process/service.

Tracing indicates the Issue exists in the Home Assistant integration ovoenergy. As this is community developed, https://github.com/timmo001/ovoenergy is the place to ask for help.

Please go there to see if this can be fixed - OVO can’t touch that particular code.

Full response can be found at the thread below:

I've already commented on the GitHub issue related to this and await a response but the GitHub repo has no control over the OVOs Cloudflare account which provides no methodology to get unblocked. 

The reason that the HA extension isn't working is because of Cloudflare blocking IPs. Work is required at both ends. The extension needs to reduce it's polling frequency but your Cloudflare distribution needs to be configured to recognise these API requests and provide a methodology to get unblocked 

As I mentioned previously, there is no possible way for Cloudflare to tell the difference between attack traffic that’s flooding a customers systems, and a legitimate tool that’s merely encountered a bug and gotten stuck in a loop while trying to authenticate.

I’m a trained computer hacker so I should know this stuff. Specifically, I’m a qualified Penetration Tester, or PenTester for short - otherwise known as a White Hat. You are quite safe though - unlike Black Hats, I never touch stuff without permission of the owner. As such, I’m expected to know how things like Cloudflare work.

You will get automatically unbanned eventually - but only once you stop your HA instance sending traffic to OVO until the broken HA Integration gets fixed. I already reduced the polling rate to 3600 seconds = 1 hour across ALL Home Assistant instances using the ovoenergy integration and Timmo knows that I did so - and why. This is far more compliant with OVO’s rules than the 300 seconds = 5 minutes it was on previously. The broken code however, lies somewhere my own code contribution does not control.

Turning the shield into a cheese grater only benefits Black Hats - and I 100% promise you that you really don’t want those guys getting anywhere near OVO’s systems. But that’s what you’re asking OVO to do. Having it be all or nothing is a necessary evil - you’ll just have to accept that. It has to be set to either block EVERYTHING that breaches the rate limits, or block NOTHING that breaches the rate limits - after all that’s what Anti-DDoS is for and it’s kinda the entire point of Cloudflare’s existence in the first place.

By providing a way to programmatically get unbanned and making that endpoint publicly accessible with zero limits, you’re just asking for trouble as hackers could just constantly unban themselves so you may as well just give them the keys to the kingdom upfront and let them do whatever they want.

Timmo needs to fix his code to slow down the authentication and make it abandon further attempts if it’s the wrong login info - OVO cannot help with that.

Yes, I know all of this because I’m  Senior DevOps Engineer who manages microservice architecture including CloudFlare. I’ve already acknowledged that the extension owner needs to make changes to the plugin to reduce polling frequency to prevent it looking like an attack. 

“By providing a way to programmatically get unbanned and making that endpoint publicly accessible with zero limits, you’re just asking for trouble as hackers could just constantly unban themselves so you may as well just give them the keys to the kingdom upfront and let them do whatever they want.”

Where did I ask for this? I’ve already agreed with the principle of protecting the API. The Cloudflare page itself recommends emailing the website owner. If I go and hack the plugin myself I need to be able to test it. I’m still banned 24 hours after disabling the plugin, which is what I did IMMEDIATELY I saw that that was the cause of the failure. 

As for the timing, 3600 seconds is more than adequate for this purpose. 

I appreciate your response though. 

Just looking at the code and this needs a failure breakout rather than keep retrying. Has a check been added with Cloudflare being put in front of the API? I’m just wondering what caused the credentials to fail in the first place as they haven’t changed in over a year. I have them in Bitwarden.

I’ll have to brush up on my Python and submit a PR

    """Authenticate with OVO Energy."""
    /client, client_session] = loop.run_until_complete(
        _setup_client(username, password)
    )

    bootstrap_accounts = loop.run_until_complete(client.bootstrap_accounts())

    typer.secho(
        asdict(bootstrap_accounts),
        fg=typer.colors.GREEN,
    )

    loop.run_until_complete(client_session.close())

There are ways of testing it - GitHub Codespaces is pretty useful and easy to fire up.

If you think you can fix Timmo’s code, by all means please do. But you can’t bash OVO for trying to prevent hackers from getting in.