Solved

Can we get access to the live usage API?


Userlevel 2
  • Carbon Cutter****
  • 7 replies

Hi Ovo,

I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

For instance (more sport related), Strava and Fitbit.

icon

Best answer by g-de 27 August 2020, 19:02

can you help me understand if the links above allow users to access raw data as if from an API?

I can certainly give it a go.

 

Firstly, big thanks to Timmo and other contributors for developing the library. All thanks go to them and links to buy them a coffee for their effort can be found on PyPi or GitHub:

https://pypi.org/project/ovoenergy/
https://github.com/timmo001/ovoenergy

 

I assume the way the library was developed this was to reverse engineer the My OVO website. That is, browse the website as normal and and monitor the traffic to see what OVO URLs and APIs it uses. Assuming this is the case, one of these URLs (get account IDs) already seems to be different but the library is still working for the moment. The APIs being used are not officially supported outside of the website, so first, a warning that any changes to the My OVO website may break the library.

 

For anyone using Python, you should be able to include the ovoenergy library and look at https://github.com/timmo001/ovoenergy/blob/master/ovoenergy/cli.py for how to use it.

 

For replicating this in other languages, the process seems to be as follows:

  1. Login by sending a POST request to https://my.ovoenergy.com/api/v2/auth/login with the following JSON in the body:
    {
    "username": "...",
    "password": "...",
    "rememberMe": true
    }

    From the response, save the cookies to use in all future requests.

  2. Next, send a GET request to https://smartpaym.ovoenergy.com/api/customer-and-account-ids . Be sure to save the items in the “accountIds” property (usually only one item).
  3. Finally, make a GET request to https://smartpaym.ovoenergy.com/api/energy-usage/daily/{accountId}?date={yyyy-MM} where {accountId} is the value from the previous request and {yyyy-MM} is the month in the format 2020-08.

You can also get monthly or half-hourly usage using the following URLs similar to step 3:

As mentioned above, if this is wanted for Home Assistant, the integration was added in version 0.114 (https://www.home-assistant.io/integrations/ovo_energy/).


Unfortunately the data only goes up to the end of the previous day as OVO no longer provide real-time data:

 

View original

203 replies

@Tim_OVOI suspect you should probably hold off on that for now until the source code is published. As others have mentioned, it’s generally not a good idea to run DLLs found on the internet, especially when raw credentials are being used.


Yes, you should be suspicious, at least when it’s published as Open Source it can be verified that it is not stealing your credentials whether intentionally or accidentally.

As a developer, I have access to tools which can make this as safe as possible before I try something out.

 

/Mike

Userlevel 1

@Tim_OVO I suspect you should probably hold off on that for now until the source code is published. As others have mentioned, it’s generally not a good idea to run DLLs found on the internet, especially when raw credentials are being used.

Userlevel 7

@Tim_OVO @Transparent @TerryE @andsee @SianiAnni 

 

Here is it: The API exe I promised that takes all that HH data and nicely formats it into a consumable Sheet

 

https://drive.google.com/drive/folders/1A7XRov0579O53T8OsIGe3wXCk09jul0F?usp=sharing

 

Download all the files, and read the Readme! (oh, and Enjoy) - now I can start work on ther TOU comparison tool!! 

 

If anyone is willing to send me their export please do!  - It will help me build the tooling.

John (at) QuinlanServices (dot) Com 

 

@Jequinlan and others, should I add this comment (above) to the best answer by @g-de ? Is a combination of these two the best customer created workaround for a lack of API we currently have? Let me know please :blush:

Userlevel 7

@Jequinlan as the creator of this latest work, can you help me judge if we need a new ‘best answer’? The current one relates to @g-de’s post...

1. Oops, you are 100% right that download exe etc does not fit with the post title. It's easy to loose sight of this after 8 pages of replies.

2. Yes there is a lot of faffing around to get at your data. Editing the JSON file to add your credentials does not work (yet??) so not worth doing. It is still considered less of a security risk by some.

I would have written this as an azure web app which could be called directly from the browser and generate a choice of xlsx or CSV file directly as a download. This would satisfy the non Windows users.

That's fair, though I'd have thought that 

"Download this exe, add credentials to config json, look at data in spreadsheet format"

Doesn't quite hit the "live usage API" goal of this topic, and would appear more complicated and daunting to a general user.

Being able to buy a third party device that just needs pairing with your account, and then browsing your usage from anywhere using the companion Bright app, while also providing a local API, would. 

Unless Ovo change the devices they use in their installations that's the only solution I can think of that would hit the target of this topic. 

@Tim_OVO

You'd be much better allowing users to pair their own CAD to the system so that they could use the local API off that, rather than it having a dependency on an internet connection (portal scraping etc).

If ovo are still fitting SMETS1 meters, something like the Hildebrand glow would work if it could be paired. https://shop.glowmarkt.com/products/glow-stick

 

Other providers such as octopus (who I moved to) allow this.

That's fine for the more technical users, who don't mind paying for extra gadgets.

However, I am sure that there are loads more who just want to use what they already have. I.e. web connection and either Excel or any other free spread sheet software from the likes of open office or libre office etc.

/Mike

@Tim_OVO 

You'd be much better allowing users to pair their own CAD to the system so that they could use the local API off that, rather than it having a dependency on an internet connection (portal scraping etc).

If ovo are still fitting SMETS1 meters, something like the Hildebrand glow would work if it could be paired. https://shop.glowmarkt.com/products/glow-stick

 

Other providers such as octopus (who I moved to) allow this.

Yes its c# in .net core 3. I spoke to my friend who said i can compile a linux version. So will try that soon too!

At work we write our code as .net core 3 assemblies, then run them with the command line

dotnet assembly.dll

As far as I know we use the same command line and the same assembly running on our windows Dev machine or in our Linux docker containers

/Mike

Sorry for another spamming comment but I’m keen to hear if others have checked out this work done by @Jequinlan this week? 

 

@Gary @GarethMarlow @rgphilpott @atilagiant @Deanohol @Peter_Mat @didge05 @sam_cat @jmontalvo @adec @Pramsbottom @ryck @CeeJayCee @alexbaulch @Random @maccordingley @sfgreenwood @johnk @jadam @Fordy @Steveo67 @Siggy0 @CerebralDigits @deanjames @ChrisB @jadam @DaveJenYearsley @ErTnEc @Marshcroft @jordanharper @Simonfea @obiwanzamora @AdamJ @dorinho @Ross2411 @Chrherms  @blakedrayson @TerryE @VibroAxe @jdenver @maurorappa @adec @100903609 @Riscy @r3m0dul8 @MyZoe.Uk @Ajk @AlanN51 @CloudCircus @r3m0dul8 @technogeoff @resoai @Fuzzysteve @mattc @andsee @uvarvu  @g-de 

 

 

No longer with Ovo, but as a sysadmin I wouldn't trust a compiled exe distributed via gdrive.

That in itself is very bad practice.

Source needs publishing through something like github, and you should host the exe under the releases section of some of that. 

Without it, no one can trust the security of it, especially when you have to put account information in it. 

Userlevel 6
Badge +1

Yes its c# in .net core 3. I spoke to my friend who said i can compile a linux version. So will try that soon too!

Userlevel 7
Badge +2

There’s another very active discussion about missing data on Online Graphs which you should have a look at, @MikeWilliams .

After looking at the data I can see that my half hourly readings for gas have stopped from August 20th 2020.

I am sure that it’s NOT a fault of the utility as I see the same in the web portal.

Userlevel 3

Tim, I haven’t had any WinX devices for over 10 years so Poweshell is useless to me.  My Python script continues to work fine for me. Thanks.

Userlevel 1

Gave up when I saw the file extension, was hoping for something written that I could run on my Pi.   I tried to decompile it using Visual Studio but didn’t get far.

Looks like C++ I’m guessing it’s using a large library for the http requests and that’s taking up the size, but haven’t had too deep a look myself

Gave up when I saw the file extension, was hoping for something written that I could run on my Pi.   I tried to decompile it using Visual Studio but didn’t get far.

Userlevel 1

Sorry for another spamming comment but I’m keen to hear if others have checked out this work done by @Jequinlan this week? 

 

In the same boat without an account now,

So not unless it’s open-sourced, especially when they managed to make it almost 100mb in size. I’d say no to running that
 

Userlevel 6
Badge +1

On a side note, a v2 with fixed year field coming next week.

Userlevel 7

Sorry for another spamming comment but I’m keen to hear if others have checked out this work done by @Jequinlan this week? 

 

@Gary @GarethMarlow @rgphilpott @atilagiant @Deanohol @Peter_Mat @didge05 @sam_cat @jmontalvo @adec @Pramsbottom @ryck @CeeJayCee @alexbaulch @Random @maccordingley @sfgreenwood @johnk @jadam @Fordy @Steveo67 @Siggy0 @CerebralDigits @deanjames @ChrisB @jadam @DaveJenYearsley @ErTnEc @Marshcroft @jordanharper @Simonfea @obiwanzamora @AdamJ @dorinho @Ross2411 @Chrherms  @blakedrayson @TerryE @VibroAxe @jdenver @maurorappa @adec @100903609 @Riscy @r3m0dul8 @MyZoe.Uk @Ajk @AlanN51 @CloudCircus @r3m0dul8 @technogeoff @resoai @Fuzzysteve @mattc @andsee @uvarvu  @g-de 

Userlevel 7
Badge +2

There’s another issue to consider if OVO were to take this on. I believe that their new Billing System is being developed such that it can be licensed to 3rd parties. Such an arrangement means that they would need to secure some rights over any externally-sourced code within the suite and be able to provide rapid support if an element of that code were to fail or require upgrading.

That’s a legal minefield.

And let’s remember that it was the failure of a 3rd party software provider to resolve issues with parts of their billing system which caused OVO to embark on creating their own in the first place. It didn’t help that Ofgem clobbered them with a hefty fine for the failed usage-estimation provisions in that software.

You will understand that there would be a significant level of reluctance for OVO to become embroiled in such a scenario again.

Btw, I can confirm there is no risk of insecure as you will see from the command line the details are on your machine and provided to directly access ovo, the exe has been made by best practice .net core code and is a standalone on pc only exe.


Is the program open or closed source?

At the moment I am discussing if I should open source it or if OVO themselves may be interested to make it or a basis of it integrated as a supported offering.


Would be good if it were made open source, then the community could help to fix any issues.

Userlevel 6
Badge +1

Btw, I can confirm there is no risk of insecure as you will see from the command line the details are on your machine and provided to directly access ovo, the exe has been made by best practice .net core code and is a standalone on pc only exe.


Is the program open or closed source?

At the moment I am discussing if I should open source it or if OVO themselves may be interested to make it or a basis of it integrated as a supported offering.

@Tim_OVO @Transparent @TerryE @andsee @SianiAnni

Here is it: The API exe I promised that takes all that HH data and nicely formats it into a consumable Sheet

https://drive.google.com/drive/folders/1A7XRov0579O53T8OsIGe3wXCk09jul0F?usp=sharing

Download all the files, and read the Readme! (oh, and Enjoy) - now I can start work on ther TOU comparison tool!! 

If anyone is willing to send me their export please do!  - It will help me build the tooling.

John (at) QuinlanServices (dot) Com 

 

Examining the downloaded csv file

  1. The following columns are always empty
    MeterPointIdentifier, CostCurrencyUnit
  2. YearColumn is always same as MonthOfYear column
  3. CostAmmount is always zero
  4. IntervalStart and IntervalEnd columns do not require fractions of a second
    2018-12-01T04:30:00.0000000 → 2018-12-01T04:30:00 please.

Not sure if these are your columns, but the output could do with tidying up.

As a programmer I can help with this.

Hi All,

I have just written a PowerShell script to fetch all of my readings from January 2017 to present-day a month at a time.

You can find it on GitHub at

https://github.com/MikeWilliams-UK/Bulk-Fetch-Ovo-Readings

Have fun

@Tim_OVO Given that some of Ovo’s customers have been asking for this and more for many years, I think @Jequinlan deserves a big round of applause for sharing his code. I agree that without the backing of Ovo, it should be used with a degree of caution. It could just stop working one day, when Ovo next change their api? interface, as happened in the Hive community recently. 

Yes, if the api changes it could stop, but also the apis are soft coded in the config file so if a trivial change the config will just need updating. The risk of any 3rd party integration exists if any 1st party changes an apinin any situation.

 

If ovo want to see the code , then i am happy to discuss.

Warning: This only protects the user if the API end-point changes, not when the format of data it fetches changes.

Reply