Solved

Can we get access to the live usage API?


Userlevel 2
Badge

Hi Ovo,

I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

For instance (more sport related), Strava and Fitbit.

icon

Best answer by Tim_OVO 27 January 2020, 11:06

I’ve got our official stance on this from our dev team:

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking and our product teams are focused on building a great online experience and working with our members towards our Plan Zero initiative. When this changes in the future we’ll announce it on this topic first.

I’ll stick this as the ‘best answer’ so others know. 

Thoughts below please! 

View original

141 replies

Userlevel 2
I'd also be interested in having a play with an API if/when it is available.

Could I also suggest simply making the data for a specified date range available as a downloadable CSV?

I would expect that this would be much easier to implement than an API (mainly because it wouldn't require the OAuth or similar authentication infrastructure that would be needed for the API), and would allow customers to analyse their own data in Excel or whatever without needing the programming skills required to use an API.

Richard.
Userlevel 2
Badge
That's great to hear, I'll keep a look out!

I'm interested in making the current live usage more accessible - the page on the website is great but means physically going to look at it, so I'd like to have the current usage readily available on something such as my phone.

I think it would also be interesting to look at the historic usage on other ways such as day of the week and cross reference with freely available weather history to see if there are correlations temperature and gas usage.🆒
A bit late to answer (just got a smart meter installed last week) but I think this would be a great idea. I would probably use the API to include data in a general home automation dashboard, integrating the electricity usage to see if any link to time spent at home or weather information. Happy to be involved in the early testing when it's being developed.
Userlevel 1

I am also working towards a plan zero initiative:

I’d like to be able to track energy usage through the day, not just a daily, weekly, monthly graph so I know where to optimize.

I’d like to be able to create charts using something like Graphana and map the energy data against light levels, temperature, humidity and other atmospheric conditions so I know when there are anomolies.

I’d like to combine the energy usage data with my smart thermostat to better understand the relationship between temperature and cost.

I’d like to set up an alert the moment my smart meter stops posting data so that I’m not sent an email months later requesting a manual submission.

I am not alone having these ideas and being capable of implementing them. But I will not scrape the data. I will simply move to a provider who does support an API.

Userlevel 7
Badge +2

I’ve already been speaking to the dev team this week (most recently yesterday) about what is required, how feasible, and when etc  @mattc 

 

I’ll be able to get an update shared next week. Thanks for bearing with - this query has not been forgotten. :)

Userlevel 3

 

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking

 

Thoughts below please! 

I’ve just spent another 4 fours re-engineering the Smart Meter API, and updated my python automatic script.

Anyone that is interested can use this python script; just update the user name and password.  This queries against the new API and unlike the previous version which output to a MySQL schema, this simply produces a CSV so users can also paste the O/P into Excel, Calc or whatever. :relaxed:

 

Anyone unhappy with the OVO developers position is free to use my Python script which wraps the normal Web interface to achieve the same.

Userlevel 3
I'm also interested in joining the early access program for the API. At present I'm using Loopenergy to monitor my gas and electric usage, and couple that with my Honeywell Evohome heating controller, I've got a fairly good monitor of overall usage and trends. At present I've a series of python scripts collecting data from both platforms and storing the metrics in an influxdb setup which I query and visualise graphically using grafana. I've also developed custom methods for adapting the overall control of the central heating system beyond what the controller already performs, adapting dynamically to usage patterns, weather, and overall energy consumption, for example, if gas usage increases without heat being called across any zone of the heating system, then the assumption is that the cooker is on and the heating then scales back on the upper floors to allow the increase in ambient temperature from the cooker to be taken into account. Plus I just like seeing what's going on (In work with some fairly big data platforms with work so it's kind of second nature really)

So yes, I'm hoping I can continue to collect data from the new smart meter and continue to use it how I do!
Any chance we've moved on and now have API access or we still waiting?

Thanks
Badge

For those interested in something like this being easily accessible I’ve made a terrible walkthrough of how to do so. hope you don’t mind me posting this Tim.

https://github.com/ThePaulAdams/OvOEnergy

https://www.thepauladams.com/blog/2020/02/24/ovo-energy-integration-to-smart-home-dashboard/

Userlevel 7
Badge +2
Hi Ovo, I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

Great question, Gary, and thanks for sharing it with us!

We definitely agree that an open API (application program interface, for those not in the know), for our customers to access their smart meter data, is a brilliant idea. This is something we have thought about, and plan on making available in the future, so watch this space!

The possibilities for community projects are endless and we'd love to see some of the ideas that you OVO Forum users have - fire away in the comments sections below! ;)

Tim

Updated on 12/07/2019
Badge
Tim - that absolutely works for me!
Userlevel 4
Badge
I've been monitoring internal temperature and external temperature for over a year, but the missing data set in my home fabric performance analysis is gas use. Ever since I got a smart meter I have periodically asked about the availability of a smart router, from which I could get usage data. I understand that data standards are still evolving, but downloading timestamp and kWh reading doesn't seem a lot to ask for:) I have sampled a few days of data by hovering over the usage graph and transcribing times and values into a spreadsheet, but obviously this takes far too long to be able to build up enough data to analyse. I have been successfully monitoring electricity usage (using an Open Energy Monitor emonTx) for a couple of months now, but the clever guys at OEM say that they don't know of any way of reading gas usage from my particular gas meter, although the technology is available to read gas usage from other kinds of gas meter.
So please add me to the list of people interested in a gas usage api.
Hi
Just wondering if OVO are any closer to re enabeling the real time live usage part of the app.
thanks
Userlevel 7
Badge +2

I’ve got our official stance on this from our dev team:

At the moment I’m afraid that we’re not planning to be offering something like this in the near future. It’s quite a complex undertaking and our product teams are focused on building a great online experience and working with our members towards our Plan Zero initiative. When this changes in the future we’ll announce it on this topic first.

I’ll stick this as the ‘best answer’ so others know. 

Thoughts below please! 

Would it be possible to add a feature to analyse the gas smart meter data against the degree day data for my area.

This is a widely used tool to analyse the energy efficiency of building heating, as it allows usage to be normalised against the weather.
Userlevel 3
Sorry, I know I'm jumping the gun... but I'm:

a) Impatient
b) Curious

I've spent about an hour messing about to try and aggregate the realtime metrics for my Smartmeter usage, and although somewhat inconclusive (due to gaps in data going back to ovo it seems). I've done some reading up on how the SmartThings integration points work, and between that and some trial and error, got a small python script up & running which will log in, poll for active smartmeters on the account, and check the instant usage data against them (if present):

vagrant@vagrant:~/Scripts# ./ovo.py
Getting readings for GAS meter with mpan 7********4:
Estimated Annual Usage is 7144 M3
Demand : 11.693 KW
Cost/Hour: 0.3776839
Getting readings for ELECTRICITY meter with mpan 1********3:
Estimated Annual Usage is 7091.1 KWH
Demand : 2.564 KW
Cost/Hour: 0.3610112

I'm looking now at doing historical polling, so aggregating data over a set period of time, which seems easy enough to do as it's just serialised JSON in the results for example:

{"power":[{"startTime":1513036800000,"instantPower":0.8766,"tariff":0.1408,"price":0.12342528},{"startTime":1513036860000,"instantPower":0.8114,"tariff":0.1408,"price":0.11424512},{"startTime":1513036920000,"instantPower":0.8276,"tariff":0.1408,"price":0.11652608},{"startTime":1513036980000,"instantPower":0.8102,"tariff":0.1408,"price":0.11407616},{"startTime":1513037040000,"instantPower":1.6943,"tariff":0.1408,"price":0.23855744},{"startTime":1513037100000,"dataError":"NotFound"},{"startTime":1513037160000,"dataError":"NotFound"},{"startTime":1513037220000,"dataError":"NotFound"},{"startTime":1513037280000,"dataError":"NotFound"},{"startTime":1513037340000,"instantPower":1.6696,"tariff":0.1408,"price":0.23507968},{"startTime":1513037400000,"instantPower":1.7693,"tariff":0.1408,"price":0.24911744},{"startTime":1513037460000,"instantPower":1.8,"tariff":0.1408,"price":0.25344},{"startTime":1513037520000,"instantPower":1.9155,"tariff":0.1408,"price":0.26970240},{"startTime":1513037580000,"dataError":"NotFound"},{"startTime":1513037640000,"dataError":"NotFound"},{"startTime":1513037700000,"instantPower":0.7824,"tariff":0.1408,"price":0.11016192},{"startTime":1513037760000,"instantPower":0.7688,"tariff":0.1408,"price":0.10824704},{"startTime":1513037820000,"dataError":"NotFound"},{"startTime":1513037880000,"dataError":"NotFound"},{"startTime":1513037940000,"instantPower":0.8776,"tariff":0.1408,"price":0.12356608},{"startTime":1513038000000,"instantPower":0.7783,"tariff":0.1408,"price":0.10958464},{"startTime":1513038060000,"dataError":"NotFound"},{"startTime":1513038120000,"dataError":"NotFound"},{"startTime":1513038180000,"dataError":"NotFound"},{"startTime":1513038240000,"dataError":"NotFound"},{"startTime":1513038300000,"instantPower":0.8985,"tariff":0.1408,"price":0.12650880},{"startTime":1513038360000,"instantPower":0.833,"tariff":0.1408,"price":0.1172864},{"startTime":1513038420000,"instantPower":0.8089,"tariff":0.1408,"price":0.11389312},{"startTime":1513038480000,"instantPower":0.7962,"tariff":0.1408,"price":0.11210496}]}

I just wish that the signal from the meter was more reliable so not to have gaps in the metrics... (hence the NotFound events)
Userlevel 3
You already have an internal RESTful API that your node.js framework uses for callbacks withing the Usage page et al. And this JSON return is all that you need for a Node RED flow.

However doing the page request to establish a valid session token is a bit of a PITA. It's a pity that you don;t faciliate direct access to your /api/live/meters/NNNNN/consumptions/aggregate read functions using a secret api key approach that many such system use.

Nonetheless, I'd rare do this the "approved " way, so can you sign me up for trials? Thanks.
Userlevel 3
You already have an internal RESTful API that your node.js framework uses for callbacks withing the Usage page et al. And this JSON return is all that you need for a Node RED flow.

Just in case anyone is interested in doing this, the JS code in the graphics display package running on your browser does a callback to retrieve the JSON data which it uses to draw the pretty usage graphs. For this to work your session must have been properly authenticated with your account and passwork because the API does a security check to ensure that any request for your data has the correct session cookies so that only you can see your data. Notwithstanding this, once your browser has established a session, then you can execute a query of the following format in your browser to pull this data down. You can then open this in your favourite spreadsheet application and manipulate / store the data as your see fit.
code:
https://live.ovoenergy.com/api/live/meters/MPAN/consumptions/aggregated?from=FROM&to=TO&granularity=HH

where the MPAN comes from your profile and the FROM and TO fields are of the format YYYY-MM-DDTHH:00:00.000Z.
The API does not seem to limited the range size, so I do this once a month and use a script to load the data into my Home Automation MySQL database. Please don't abuse this or I am sure that the developers will apply extra validation. I personally think that 1 query a month is better than going on a couple of times a day as I was doing previously.

Advanced programmers could no doubt script the whole session connection to recover the session cookies. I am one, but I haven't bothered.
Count me in also for any trials of a csv/xls download utility for usage and payment data.
+1 again.


My use-case?

Alerting me when the underfloor heating has come on unexpectedly while we're out, due to the frost stat being triggered. I could then decide to turn on the GCH heating remotely, saving on total energy use.


How?

Use an Ovo API to monitor for when the kW rate jumps by more than a threshold, for a period of time. This would indicate that a high-load device has turned on.

I'd then check whether the house is occupied, e.g. by connecting to Tado's location service or similar.

Finally I'd check if the weather service suggests it's a cold day outside.

If nobody is at home, and the load has jumped, it suggests the underfloor heating has turned on due to the frost stat. Send an alert to mobile phones, so we can decide to switch on Tado remotely to mitigate the underfloor heating.


Tech stack?

Raspberry Pi, Tado, Home Assistant, Node-RED, Python.
Userlevel 3

… Any script which tries to lift data from the online Usage pages will need to be continually updated to match the current Billing Software version.

To make it even more tricky, it’s becoming apparent that not all customers are running the same software version. See this discussion about the Zero-in option.

 

Yes any script will need to be updated as the online interface changes, but I have only updated mine once in the last 3 years., and this latest one still works for me.  I don’t know about zero-in and am not interested as I have a near zero-energy new build and track all of my energy use through my own home automation system.

As to @uvarvu ‘s Q I use a password keeper / generator for my online passwords which generates strong passwords and these will often need quoting using the r’’’password’’’ syntax as below:

 PASSWORD = r'''D#adR'xGFj{Ks";=$'''

Also note that I have an electricity-only supply and the script may need tweaking for multiple supplies.

Userlevel 7
Badge +2
I'm interested in making the current live usage more accessible - the page on the website is great but means physically going to look at it, so I'd like to have the current usage readily available on something such as my phone.

I think it would also be interesting to look at the historic usage on other ways such as day of the week and cross reference with freely available weather history to see if there are correlations temperature and gas usage.🆒


Our developers will love this feedback, Gary - keep it coming! Anyone else have any suggestions for how we could use your smart meter data?

Tim
Userlevel 7
Badge +2
Right then, @atilagiant, @SianiAnni and @Deanohol - it's great that you're interested in being part of any live-data trials we have planned. So that I can get you added to our list of interested customers, send me a PM with your account number, name and DoB. I'll make it happen! 🆒

Tim
Userlevel 7
Badge +2
Hi Tim,
I too would like the ability to download my energy usage data as a csv / xls(x) file to complement the data I can get from my weather station and my PV Inverter data.

Please sign me up for any trials that are going.
Many thanks

Peter


In that case, @Peter_Mat send over a PM with your account number, name and DoB. I'll get you added to our list of testers! 🆒

Tim
Badge
Hi,

I'm more of a configurer than a coder but I'm very keen to extract historical data either directly from the meter or via your back-end web services. I'm using Honeywell Evohome to control heating and water in my house and I'd like to be able to measure the impact of its use on my energy consumption.

Ideal for me would be a domoticz (https://www.domoticz.com/) integration - domoticz is already proving to be a great way to bring together a number of different smart home activities to visualise and control them together.

Do the dev team have an idea of when this functionality is likely to be implemented?

Reply