Solved

Can we get access to the OVO energy online account API to download our smart meter usage data?



Show first post

142 replies

Userlevel 7
Badge

@Firedog is this an OVO public API? 

Sure. That’s where the data come from to populate the various bits of the account site. Here are a few more sources that work for me:

https://smartpaymapi.ovoenergy.com/usage/api/monthly/{account}?date=2022

https://smartpaymapi.ovoenergy.com/pace/recommended-dds/v1/{account}/projected-costs?limitNextYear=true

 https://smartpaymapi.ovoenergy.com/rlc/rac-public-api/api/v5/supplypoints/electricity/{MPAN}/meters/{MSN}/readings?from=2016-09-09

This isn’t real-time; I think the newest data are from the last daily refresh, usually shortly after 01:00 each day. So today’s data won’t ever be available.

 

[What I’d really like is for an Excel guru to tell me how to include the authentication refresh-token into a query so I could push the JSON data directly into a worksheet.]  

Userlevel 4

You need to go through the login and cache the cookies to get access to the smart meter API.  I’ve got this all coded in Node-RED Javascript.  I dump data into a MySQL D/B.  Here are the URIs that I use:

https://${MYOVO}/login
https://${MYOVO}/api/v2/auth/login
https://${PAYMAPI}/first-login/api/bootstrap/v2/
https://${PAYMAPI}/orex/api/plans/${msg.account}
https://${PAYMAPI}/rlc/rac-public-api/api/v5/supplypoints/electricity/${msg.mpxn}/meters/${msg.msn}/readings?from=${context.get('dailyStart')}
https://${PAYMAPI}/usage/api/half-hourly/${m.account}?date=${m.start}

Except for the 1st login, these all return a JSON payload that is easy to walk in Javascript.   I use the plans request to get the MSN and MPXN fields as well as current peak, off-peak and standing charges. The readings request gives me daily peak and off-peak usage which can be used to calculate charges.  Looping around the half-hourly request give the daily breakdown.

The API is the same for gas and electricity, though I have an electric-only low energy house so only download electrical data.  

Incidentally we are currently running at 80% off-peak use.

Userlevel 6

Thanks to you all for taking the time to explain this to me….sadly it’s not a language I’m familiar so it’ll take a few days to work it all out….I’ll persevere though. Thanks again.

@Tim_OVO @g-de Is there a way to parse the balance from the API? I have not seen that in the documentation and I failed by just playing around with random URLs that could potentially show it in my Python script.  

Userlevel 1

@Tim_OVO @g-de Is there a way to parse the balance from the API?

Are you just wanting the “Today’s balance” section from the home page?

Using the network tab of browser dev tools is the simplest way to reverse engineer the API.

Best I can see is that it uses a POST request to:

https://smartpaymapi.ovoenergy.com/bast/api/graphql

With the following body:

{
"query": "\nfragment FuelFields on BillingFuel {\n consumption {\n rates {\n rate {\n pence\n }\n cost {\n pounds\n }\n kwh\n openingRead\n openingReadType\n intermediateReads {\n date\n read\n readType\n }\n closingRead\n closingReadType\n startDate\n endDate\n friendlyLabel\n }\n }\n standing {\n netCharge {\n pounds\n }\n rates {\n rate {\n pence\n }\n startDate\n endDate\n days\n }\n }\n}\n\nfragment PeriodFields on SelectedPeriod {\n next\n previous\n data {\n electricity {\n ...FuelFields\n }\n gas {\n ...FuelFields\n }\n upgrades {\n description\n grossCharge {\n pounds\n }\n initialGrossCharge {\n pounds\n }\n startDate\n endDate\n taxRate\n }\n transactions {\n description\n netCredit {\n pounds\n }\n taxRate\n }\n payments {\n date\n description\n credit {\n pounds\n }\n }\n start\n openingBalance {\n pounds\n }\n end\n closingBalance {\n pounds\n }\n totalCharge {\n grossCharge {\n pounds\n }\n netCharge {\n pounds\n }\n vatCharge {\n pounds\n }\n }\n energyCharge {\n pounds\n }\n isStatementAvailable\n }\n}\n\nquery Period($id: String!, $index: Int!) {\n billingSummary(id: $id) {\n lastUpdated\n billablePeriod(periodIndex: $index) {\n ...PeriodFields\n }\n }\n}\nquery LatestPeriod($id: String!) {\n billingSummary(id: $id) {\n lastUpdated\n latestPeriod {\n ...PeriodFields\n }\n }\n}",
"operationName": "LatestPeriod",
"variables": {
"id": "<account-number>"
}
}

Remember to insert your account number. The balance should then be at data > billingSummary > latestPeriod > data > closingBalance > pounds. A negative number looks to indicate in debit.

It should be possible to tune the query in the body to only pull back the desired information, but I’m not familiar with graphql.

Best I can see is that it uses a POST request to:

https://smartpaymapi.ovoenergy.com/bast/api/graphql

 


That’s great. I ll try that

 

Just an additional question. I have tried that:

 

headers = {"Content-Type": "application/graphql"}

body = {
"query": "\nfragment FuelFields on BillingFuel {\n consumption {\n rates {\n rate {\n pence\n }\n cost {\n pounds\n }\n kwh\n openingRead\n openingReadType\n intermediateReads {\n date\n read\n readType\n }\n closingRead\n closingReadType\n startDate\n endDate\n friendlyLabel\n }\n }\n standing {\n netCharge {\n pounds\n }\n rates {\n rate {\n pence\n }\n startDate\n endDate\n days\n }\n }\n}\n\nfragment PeriodFields on SelectedPeriod {\n next\n previous\n data {\n electricity {\n ...FuelFields\n }\n gas {\n ...FuelFields\n }\n upgrades {\n description\n grossCharge {\n pounds\n }\n initialGrossCharge {\n pounds\n }\n startDate\n endDate\n taxRate\n }\n transactions {\n description\n netCredit {\n pounds\n }\n taxRate\n }\n payments {\n date\n description\n credit {\n pounds\n }\n }\n start\n openingBalance {\n pounds\n }\n end\n closingBalance {\n pounds\n }\n totalCharge {\n grossCharge {\n pounds\n }\n netCharge {\n pounds\n }\n vatCharge {\n pounds\n }\n }\n energyCharge {\n pounds\n }\n isStatementAvailable\n }\n}\n\nquery Period($id: String!, $index: Int!) {\n billingSummary(id: $id) {\n lastUpdated\n billablePeriod(periodIndex: $index) {\n ...PeriodFields\n }\n }\n}\nquery LatestPeriod($id: String!) {\n billingSummary(id: $id) {\n lastUpdated\n latestPeriod {\n ...PeriodFields\n }\n }\n}",
"operationName": "LatestPeriod",
"variables": {
"id": accountIds
}
}

response = requests.post(url, json={'query': body}, headers=headers,cookies=cookies)

print(response.status_code)
print(response.json())

but I am getting an 401 with a message 

'OAuth plugin - No refresh_token present'

I have tried removing the headers and I am still getting the same error

Userlevel 1

@apolosisk You'll still need to authenticate first. See step 1 in the “beat answer" marked at the top of this thread. That should let you authenticate and store and auth cookie. Then include the cookie with the graphql request you're sending.

Just a note on your code, the content type header might need to be “application/json” and I expect it should just be”json=body” rather than wrapping in another document. I'm not sure on these though. Just something to try if you resolve the auth error and still encounter issues.

@apolosiskYou'll still need to authenticate first. See step 1 in the “beat answer" marked at the top of this thread. That should let you authenticate and store and auth cookie. Then include the cookie with the graphql request you're sending.

Just a note on your code, the content type header might need to be “application/json” and I expect it should just be”json=body” rather than wrapping in another document. I'm not sure on these though. Just something to try if you resolve the auth error and still encounter issues.



I have done that, but I am getting that error. I must be missing something in the latest POST:

#POST REQUEST TO LOGIN/AUTHENTICATE
url = 'https://my.ovoenergy.com/api/v2/auth/login'
myobj = {
"username": "ΧΧΧΧ@ΧΧΧΧ.com",
"password": "ΧΧΧΧ",
"rememberMe": True
}

x = requests.post(url, json = myobj)
#SAVE COOKIES
cookies = x.cookies

#GET ACCOUNT ID REQUEST
url = "https://smartpaym.ovoenergy.com/api/customer-and-account-ids"
y = requests.get(url, cookies=cookies)

#GET ACCOUNT ID BY MANIPULATING LISTS, JSON AND STRINGS
accountIds = list(y.json().values())[0][0]

#POST REQUEST TO GET BALANCE
url = "https://smartpaymapi.ovoenergy.com/bast/api/graphql"
#headers = {"Content-Type": "application/graphql"}

body = {
##The same as above. I removed it avoid making the thread long
}

response = requests.post(url, json={'query': body})

 

Userlevel 4

@apolosisk, see GraphQL.  I am surprised the OVO expose this to the client side APIs -- all sort of potential security vulnerabilities there, IMO.  However, what you are missing is easily obtained from the browser’s debug query log, and this is that the post expects a JSON encoded input:

{
  query: $QUERY,
  operationName: "LatestPeriod",
  variables: {id: $ACCOUNTNO }
}

where $ACCOUNTNO is yours and $QUERY is

query LatestPeriod($id: String!) {
  billingSummary(id: $id) {
    latestPeriod {
      data {
        payments {
          date
          description
          credit {
            pounds
          }
        }
        closingBalance {
          pounds
        }
      }
    }
  }
}

The GraphQL website gives the syntax of the QL, if you are interested.

@apolosisk, see GraphQL.  I am surprised the OVO expose this to the client side APIs -- all sort of potential security vulnerabilities there, IMO.  However, what you are missing is easily obtained from the browser’s debug query log, and this is that the post expects a JSON encoded input:

 

 

third_query = "query LatestPeriod($id: String!) { \n  billingSummary(id: $id) { \n    latestPeriod { \n      data { \n        payments { \n          date \n          description \n          credit { \n            pounds \n          } \n        } \n        closingBalance { \n          pounds \n        } \n      } \n    } \n  } \n} \n"

query = {
"query": third_query,
"operationName": "LatestPeriod",
"variables": {
"id": accountIds
}
}

response = requests.post(url,json=query)

I still do not understand what I am missing exactly. Is it in the headers? I have tried finding examples and playing around. I took the headers from the first POST (the login) and even made an update:

headers['Content-Type'] = "application/graphql"

I am not familiar with the authorisation and the examples online do not help

Userlevel 4

I am not familiar with the authorisation and the examples online do not help

You need to understand how session cookies work and walk through this using your browser debug console to walk through how the client-side JS code on an interactive myovo session builds up the necessary cookies and context.  The scripting syntax and the HTTP API are different for python and JS. It’s bit of tedious but fairly straight forward retro-engineering to work out what calls are needed.  I can only point the way, not do it all for you. 🙁

Userlevel 7

We love seeing the innovative ways our customers are using tech to help monitor and manage their energy usage. It’s inspiring us to think about ways to do energy differently. We know some customers have been using some of the application programming interface (APIs) behind our public facing services. While we’re OK with that, we do need to make you aware of a couple of things.

 

These APIs are designed to be used by OVO teams only, and aren't public facing. There are some downsides to using APIs that aren't for the public, and we wanted to let you know what these are. Behind the scenes, OVO uses APIs to share and update information between systems that power your bills and your online account. This is done in partnership with Kaluza, the tech company that’s part of the OVO family. They’ve built the billing platform designed to put our customers in the driving seat of their energy usage. 

 

These internal APIs are intended for use by Kaluza and its clients, who are energy retailers like OVO, rather than customers. Because of this, there’s no support for them being used anywhere else, which means they may be discontinued with no notice when we update our products and services. 

 

OVO Energy and Kaluza need to be able to monitor these APIs, and may block access if there's any problems in the future.

 

We know some customers may have put time and effort into developing solutions that use these API. So, now that you know the risks, we want to hear from you on how you’re using the APIs and the problems you’re solving with your DIY approach. 

 

Is there anything you’d like to see from OVO to help you monitor and manage your energy better? Leave a comment below to tell us.

Userlevel 1

@Tim_OVO Primarily through Home Assistant to populate the energy dashboard: https://www.home-assistant.io/integrations/ovo_energy/

I found that this integration didn’t provide information reliably enough though (can’t remember what the issue was) so currently use a ShellyEM for energy monitoring. I currently only use the integration for energy/unit costs, however this still has limitations.

The ideal I think for home assistant would be to have separate gas and electricity usage available as quickly as possible (I realise a limitation on the smart meters is every 30 mins), along with the price per unit (for those using separate energy monitoring such as with a Shelly), and overall cost (i.e. how much have I used so far in £). Essentially, the information that is visible on the IHD.

This isn’t currently covered, but solar FIT rates and standing charges would also be very valuable and should be fairly static data. This would allow the energy dashboard to be fully populated in real time for those with something like a ShellyEM doing the monitoring, and accurate to ~30mins for those without.

https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers

 

Works fine for getting me information into home assistant, and it's not likely to have the rug pulled from under you. 

Userlevel 1

@Fuzzysteve Good to know. I think I looked at their products at one point but wasn’t sure about compatibility. That might have changed now so I’ll take a look. The app provided me with closer to real-time usage for a while, but then the APIs to grab the data seemed to start having issues. Possibly DoS protection given I wasn’t paying for anything 🤷‍♂️

Userlevel 1

I use the Home assistant api to get data for my OVO account but it's not real time. I’m aware of the Glowmarkt device but can't justify the cost to be honest.

Userlevel 7

Another bit of feedback from @blakedrayson who is happy for me to post here on their behalf:

I think that from an API perspective Octopus has been great. However I have continued to use the products of Hildebrand (their SMETS 2 IHD is brilliant and has been a good way to work around supplier areas that were lacking).

I have managed to write a lot of code that integrates with their systems and build my own home display and automation.

I think you could do a lot worse than taking a look at what is provided by Hildebrand 
https://glowmarkt.com/support/data and what Octopus provides as well https://developer.octopus.energy/docs/api/.

Userlevel 7
Badge +1

I’ve pinged someone who creates one of the tools in question: https://github.com/timmo001/ovoenergy/issues/80

Hi there,

I just wanted to share with anyone who might be interested that I have developed a package in R (a programming language mostly used for statistical programming) to interact with the Ovo API. I basically wrote the code based on what it has already been explained in this thread.

It is a very early version, so if you are interested mind the rough edges. You can find it on my GitHub account. I decided to call it Uovo Energy (Uovo means egg in Italian).

I also created a web application alongside the package and put it all in a docker file. The README file explains how to use it.

 

Gas consumption plot

 

Userlevel 4
Badge

My experiments in this area have somewhat stalled. Partly due to pressure of family and work life but partly because I still don't have a smart meter. My one and only attempt to get one fitted some years ago resulted in the engineer saying he wouldn't touch the existing gas meter, that it needed someone else to change the mounting. I never heard back. Since then, so many of the people I know who have had one fitted have had nothing but problems, often going years (if ever) before they actually started working, I really don't want one at the moment. Also, since I work from home, I'm reluctant to have to power everything down to get new meters installed.

So currently, I grab my annual figures into a spreadsheet to monitor usage.

One thing that would be super cool would be if the Ovo app was better at showing a this-vs-last year comparison - at the moment, you have to keep switching back and forth between the years and the Y-axis changes making it hard to see visually.

 

Other than that, I would strongly urge Ovo to have a published API that customers can access using standard REST and appropriate open standards security. If for no other reason than it fits nicely with a “green” stance on helping people better manage and reduce power consumption.

However, the API should not only provide near real-time data from the smart meters but should also give access to the underlying customer data for those of us who don’t have smart meters and for those many people who do but they don’t work. That data would presumably just be monthly data but that is more than sufficient to be honest.

Personally, as I’m active in the Node-RED project, that is what I would use to grab my monthly stats, storing the data locally and showing on a dashboard using my own published UIBUILDER contribution to Node-RED. 😁

Node-RED is also accessible from Home Assistant but is easily run stand-alone and I use it for all my home automation as do many people. It is a low-code development environment and server based on Node.js. It has a very large and supportive user base and a very active community.

I have just published my application which fetches your data and stores it in a local SQLite database on your PC.

It can also export the data to CSV and Excel files, to allow easy anaylsis.

The release can be found in my GitHub repostory at https://github.com/MikeWilliams-UK/My-Ovo-Data/releases/latest

From here you download the zip file, extract it to a folder on your PC, then run it.

Data is written to the folder C:\ProgramData\OvoData

Any problems either raise an issue in GitHub or get back to me here.

At the moment it is Windows only, but as it’s .NET Core and C# it could be compiled for other platforms.

/Mike Williams

Userlevel 7
Badge

I have just published my application which fetches your data and stores it in a local SQLite database on your PC.

 

  That’s great, thanks very much 👍🏻

 

… download the zip file, extract it to a folder on your PC, then run it.

 

I must be missing a step. I’ve done all that (ignoring Windows Security warnings), selected a period (This month) and ended up with an apparently empty DB file (84kB) in ProgramData:
  

*edited by mod* 

What am I doing wrong?

I have just published my application which fetches your data and stores it in a local SQLite database on your PC.

 

  That’s great, thanks very much 👍🏻

 

… download the zip file, extract it to a folder on your PC, then run it.

 

I must be missing a step. I’ve done all that (ignoring Windows Security warnings), selected a period (This month) and ended up with an apparently empty DB file (84kB) in ProgramData:

What am I doing wrong?

Possibly a silly question, but did you click on the “Read” button after logging on?

For a first time run it is recomended to select “All Time”, then click on “Read”

/Mike Williams

I have just published my application which fetches your data and stores it in a local SQLite database on your PC.

 

  That’s great, thanks very much 👍🏻

 

… download the zip file, extract it to a folder on your PC, then run it.

 

I must be missing a step. I’ve done all that (ignoring Windows Security warnings), selected a period (This month) and ended up with an apparently empty DB file (84kB) in ProgramData:
  

 

What am I doing wrong?

Possibly a silly question, but did you click on the “Read” button after logging on?

For a first time run it is recomended to select “All Time”, then click on “Read”

/Mike Williams

The highlighted part of the screen shows what data has been read.

 

Reply