Pinned Solved

Can we get access to the live usage API?


Userlevel 2
Badge
Hi Ovo, I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

For instance (more sport related), Strava - https://strava.github.io/api/ and Fitbit - https://dev.fitbit.com/docs/
icon

Best answer by Tim_OVO 10 March 2017, 16:40

Hi Ovo, I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

Great question, Gary, and thanks for sharing it with us!

We definitely agree that an open API (Application program interface, for those not in the know), for our customers to access their smart meter data, is a brilliant idea. This is something we have thought about, and plan on making available in the future, so watch this space!

The possibilities for community projects are endless and we'd love to see some of the ideas that you OVO Forum users have - fire away in the comments sections below! ;)

Tim
View original

111 replies

A bit late to answer (just got a smart meter installed last week) but I think this would be a great idea. I would probably use the API to include data in a general home automation dashboard, integrating the electricity usage to see if any link to time spent at home or weather information. Happy to be involved in the early testing when it's being developed.
Userlevel 2
I'd also be interested in having a play with an API if/when it is available.

Could I also suggest simply making the data for a specified date range available as a downloadable CSV?

I would expect that this would be much easier to implement than an API (mainly because it wouldn't require the OAuth or similar authentication infrastructure that would be needed for the API), and would allow customers to analyse their own data in Excel or whatever without needing the programming skills required to use an API.

Richard.
Userlevel 2
Badge
That's great to hear, I'll keep a look out!

I'm interested in making the current live usage more accessible - the page on the website is great but means physically going to look at it, so I'd like to have the current usage readily available on something such as my phone.

I think it would also be interesting to look at the historic usage on other ways such as day of the week and cross reference with freely available weather history to see if there are correlations temperature and gas usage.🆒
Userlevel 3
I'm also interested in joining the early access program for the API. At present I'm using Loopenergy to monitor my gas and electric usage, and couple that with my Honeywell Evohome heating controller, I've got a fairly good monitor of overall usage and trends. At present I've a series of python scripts collecting data from both platforms and storing the metrics in an influxdb setup which I query and visualise graphically using grafana. I've also developed custom methods for adapting the overall control of the central heating system beyond what the controller already performs, adapting dynamically to usage patterns, weather, and overall energy consumption, for example, if gas usage increases without heat being called across any zone of the heating system, then the assumption is that the cooker is on and the heating then scales back on the upper floors to allow the increase in ambient temperature from the cooker to be taken into account. Plus I just like seeing what's going on (In work with some fairly big data platforms with work so it's kind of second nature really)

So yes, I'm hoping I can continue to collect data from the new smart meter and continue to use it how I do!
Userlevel 4
Badge
I've been monitoring internal temperature and external temperature for over a year, but the missing data set in my home fabric performance analysis is gas use. Ever since I got a smart meter I have periodically asked about the availability of a smart router, from which I could get usage data. I understand that data standards are still evolving, but downloading timestamp and kWh reading doesn't seem a lot to ask for:) I have sampled a few days of data by hovering over the usage graph and transcribing times and values into a spreadsheet, but obviously this takes far too long to be able to build up enough data to analyse. I have been successfully monitoring electricity usage (using an Open Energy Monitor emonTx) for a couple of months now, but the clever guys at OEM say that they don't know of any way of reading gas usage from my particular gas meter, although the technology is available to read gas usage from other kinds of gas meter.
So please add me to the list of people interested in a gas usage api.
Would it be possible to add a feature to analyse the gas smart meter data against the degree day data for my area.

This is a widely used tool to analyse the energy efficiency of building heating, as it allows usage to be normalised against the weather.
Userlevel 3
Sorry, I know I'm jumping the gun... but I'm:

a) Impatient
b) Curious

I've spent about an hour messing about to try and aggregate the realtime metrics for my Smartmeter usage, and although somewhat inconclusive (due to gaps in data going back to ovo it seems). I've done some reading up on how the SmartThings integration points work, and between that and some trial and error, got a small python script up & running which will log in, poll for active smartmeters on the account, and check the instant usage data against them (if present):

vagrant@vagrant:~/Scripts# ./ovo.py
Getting readings for GAS meter with mpan 7********4:
Estimated Annual Usage is 7144 M3
Demand : 11.693 KW
Cost/Hour: 0.3776839
Getting readings for ELECTRICITY meter with mpan 1********3:
Estimated Annual Usage is 7091.1 KWH
Demand : 2.564 KW
Cost/Hour: 0.3610112

I'm looking now at doing historical polling, so aggregating data over a set period of time, which seems easy enough to do as it's just serialised JSON in the results for example:

{"power":[{"startTime":1513036800000,"instantPower":0.8766,"tariff":0.1408,"price":0.12342528},{"startTime":1513036860000,"instantPower":0.8114,"tariff":0.1408,"price":0.11424512},{"startTime":1513036920000,"instantPower":0.8276,"tariff":0.1408,"price":0.11652608},{"startTime":1513036980000,"instantPower":0.8102,"tariff":0.1408,"price":0.11407616},{"startTime":1513037040000,"instantPower":1.6943,"tariff":0.1408,"price":0.23855744},{"startTime":1513037100000,"dataError":"NotFound"},{"startTime":1513037160000,"dataError":"NotFound"},{"startTime":1513037220000,"dataError":"NotFound"},{"startTime":1513037280000,"dataError":"NotFound"},{"startTime":1513037340000,"instantPower":1.6696,"tariff":0.1408,"price":0.23507968},{"startTime":1513037400000,"instantPower":1.7693,"tariff":0.1408,"price":0.24911744},{"startTime":1513037460000,"instantPower":1.8,"tariff":0.1408,"price":0.25344},{"startTime":1513037520000,"instantPower":1.9155,"tariff":0.1408,"price":0.26970240},{"startTime":1513037580000,"dataError":"NotFound"},{"startTime":1513037640000,"dataError":"NotFound"},{"startTime":1513037700000,"instantPower":0.7824,"tariff":0.1408,"price":0.11016192},{"startTime":1513037760000,"instantPower":0.7688,"tariff":0.1408,"price":0.10824704},{"startTime":1513037820000,"dataError":"NotFound"},{"startTime":1513037880000,"dataError":"NotFound"},{"startTime":1513037940000,"instantPower":0.8776,"tariff":0.1408,"price":0.12356608},{"startTime":1513038000000,"instantPower":0.7783,"tariff":0.1408,"price":0.10958464},{"startTime":1513038060000,"dataError":"NotFound"},{"startTime":1513038120000,"dataError":"NotFound"},{"startTime":1513038180000,"dataError":"NotFound"},{"startTime":1513038240000,"dataError":"NotFound"},{"startTime":1513038300000,"instantPower":0.8985,"tariff":0.1408,"price":0.12650880},{"startTime":1513038360000,"instantPower":0.833,"tariff":0.1408,"price":0.1172864},{"startTime":1513038420000,"instantPower":0.8089,"tariff":0.1408,"price":0.11389312},{"startTime":1513038480000,"instantPower":0.7962,"tariff":0.1408,"price":0.11210496}]}

I just wish that the signal from the meter was more reliable so not to have gaps in the metrics... (hence the NotFound events)
Userlevel 2
You already have an internal RESTful API that your node.js framework uses for callbacks withing the Usage page et al. And this JSON return is all that you need for a Node RED flow.

Just in case anyone is interested in doing this, the JS code in the graphics display package running on your browser does a callback to retrieve the JSON data which it uses to draw the pretty usage graphs. For this to work your session must have been properly authenticated with your account and passwork because the API does a security check to ensure that any request for your data has the correct session cookies so that only you can see your data. Notwithstanding this, once your browser has established a session, then you can execute a query of the following format in your browser to pull this data down. You can then open this in your favourite spreadsheet application and manipulate / store the data as your see fit.

where the MPAN comes from your profile and the FROM and TO fields are of the format YYYY-MM-DDTHH:00:00.000Z.
The API does not seem to limited the range size, so I do this once a month and use a script to load the data into my Home Automation MySQL database. Please don't abuse this or I am sure that the developers will apply extra validation. I personally think that 1 query a month is better than going on a couple of times a day as I was doing previously.

Advanced programmers could no doubt script the whole session connection to recover the session cookies. I am one, but I haven't bothered.
+1 again.


My use-case?

Alerting me when the underfloor heating has come on unexpectedly while we're out, due to the frost stat being triggered. I could then decide to turn on the GCH heating remotely, saving on total energy use.


How?

Use an Ovo API to monitor for when the kW rate jumps by more than a threshold, for a period of time. This would indicate that a high-load device has turned on.

I'd then check whether the house is occupied, e.g. by connecting to Tado's location service or similar.

Finally I'd check if the weather service suggests it's a cold day outside.

If nobody is at home, and the load has jumped, it suggests the underfloor heating has turned on due to the frost stat. Send an alert to mobile phones, so we can decide to switch on Tado remotely to mitigate the underfloor heating.


Tech stack?

Raspberry Pi, Tado, Home Assistant, Node-RED, Python.
Userlevel 6
Badge +2
Hi Ovo, I wondered whether the API that powers your live and historical usage page in the account menu is something you could look into opening up a bit so we can freely access our raw data and also perhaps it may stimulate some interesting community projects?

Great question, Gary, and thanks for sharing it with us!

We definitely agree that an open API (Application program interface, for those not in the know), for our customers to access their smart meter data, is a brilliant idea. This is something we have thought about, and plan on making available in the future, so watch this space!

The possibilities for community projects are endless and we'd love to see some of the ideas that you OVO Forum users have - fire away in the comments sections below! ;)

Tim
Userlevel 6
Badge +2
Right then, @atilagiant, @SianiAnni and @Deanohol - it's great that you're interested in being part of any live-data trials we have planned. So that I can get you added to our list of interested customers, send me a PM with your account number, name and DoB. I'll make it happen! 🆒

Tim
Badge
Hi,

I'm more of a configurer than a coder but I'm very keen to extract historical data either directly from the meter or via your back-end web services. I'm using Honeywell Evohome to control heating and water in my house and I'd like to be able to measure the impact of its use on my energy consumption.

Ideal for me would be a domoticz (https://www.domoticz.com/) integration - domoticz is already proving to be a great way to bring together a number of different smart home activities to visualise and control them together.

Do the dev team have an idea of when this functionality is likely to be implemented?
Badge
Tim - that absolutely works for me!
Userlevel 4
Badge

Advanced programmers could no doubt script the whole session connection to recover the session cookies. I am one, but I haven't bothered.


Further to @ErTnEc's git contribution, I have updated this to do the V2 API for historic consumption. I run the following script daily to download the previous day's half-hourly consumption into the MySQL DB that underpins my NodeRED installation: load_readings.py.
There's an issue with NodeRED HTTPrequest which causes logon problems so I haven't got the native NodeRED version working. Sorry. It's not worth my spending time on this because I will moving supplier when my contract ends. I really like OVO, but a 30+% price hike is just too much, when other suppliers offer better deals.


Thanks for this. I haven't managed to get your load_readings.py script to run on Windows, but as long as I'm already logged into MyOvo, pasting a handcrafted url from your script certainly returns the data, which is great because the url which I was using last winter no longer works! I copy the displayed data into a python script which then parses and outputs something human readable and graphable into a csv file.
Userlevel 2
Right then, @atilagiant, @SianiAnni and @Deanohol - it's great that you're interested in being part of any live-data trials we have planned. So that I can get you added to our list of interested customers, send me a PM with your account number, name and DoB. I'll make it happen! 🆒

Tim


Tim - I have sent you a message and am keen to get involved to.. This sort of process/analysis is right up my street and is what I do for a day job (sort of.. Different industry)
Hi
Just wondering if OVO are any closer to re enabeling the real time live usage part of the app.
thanks
Hi Lucy / Tom,

Am I too late to be added into the API trial ?

Has anyone got any datasets yet ?

Thanks
Steve
Hey there, I've just signed up as part of the community to see if I can join in on the trials - would you mind adding me if I send the required PM?

Thanks!
hi, Guys,

May I request access. Part of the reason I joined OVO was this very forum article. Whilst I am not on-board yet, on Oct 5 I switch and Oct 20th is my smart meter installation, I am keen to get access to this data as I am looking to create multiple apps that I can query with Alexa, integrate with home-assistant etc. I am keen to try and pre-empt energy usage based on weather, historical heating settings from the thermostat etc. and Shift hours. And finally, break down energy usage by correlating the information from my smart plugs.

There are probably loads of ideas out there for this type of data and it is great to see OVO embracing this way of working. As more third-party products come out with their own exposure, OVO will be seen as a leader and will hopefully bring more customers to you as a side benefit.

Can't wait to see what comes of this.

Chris
Userlevel 3
@ErTnEc, this is exactly what I want to be able to do. I use NodeRED to poll darksky.net for current weather readings. Can you send me your python script to play with?

Sorry I've had a hectic time lately and not had chance to pick this back up.

https://gist.github.com/PaulW/7f3ce29c3469757c26504b572597a45c

I still need to add in some extra error handling (specifically around initial auth requests as it takes a while for the authtoken to be fully active it seems) but it's a fairly simple process which people can easily build upon if need be.
+1 for this – or at least being able to download your raw data to be able to analyse and explore ourselves. Would be really useful and interesting.
Userlevel 2
You already have an internal RESTful API that your node.js framework uses for callbacks withing the Usage page et al. And this JSON return is all that you need for a Node RED flow.

However doing the page request to establish a valid session token is a bit of a PITA. It's a pity that you don;t faciliate direct access to your /api/live/meters/NNNNN/consumptions/aggregate read functions using a secret api key approach that many such system use.

Nonetheless, I'd rare do this the "approved " way, so can you sign me up for trials? Thanks.
Userlevel 2
@SianiAnni At the moment, I cludged this by running the query in my browser after creating my session interactively and then using a small perl script to upload this into my nodered db.

I haven' bother to retro-engineer the sequence of POST requests to establish fresh set of session cookies for each AJAX callback to work as retro-engineering this is all a bit of a dog, and iIt could take me a day to do this so there isn't the payback: one day's work to save a minute or so every month.

Which is why we really need to wait for the proper API if you want to automate this fully.
Userlevel 2

Advanced programmers could no doubt script the whole session connection to recover the session cookies. I am one, but I haven't bothered.


Further to @ErTnEc's git contribution, I have updated this to do the V2 API for historic consumption. I run the following script daily to download the previous day's half-hourly consumption into the MySQL DB that underpins my NodeRED installation: load_readings.py.

There's an issue with NodeRED HTTPrequest which causes logon problems so I haven't got the native NodeRED version working. Sorry. It's not worth my spending time on this because I will moving supplier when my contract ends. I really like OVO, but a 30+% price hike is just too much, when other suppliers offer better deals.
Userlevel 2
works a treat.
@jdenver, If you check, I've updated the script slightly. What I've found is that the Smart Meter sends its SMS message once a day about 02:00, so the consumption figures for the previous day are only available after that. The message occasionally fails for whatever reason, so you can miss a day, but it does retry at some point. So what I've done is to query the max DTS from my readings table and set the FROM to 30 mins after the last end. That way, the reading will catch up and you don't end up dropping data in your table.

I am note sure why the NodeRED native version fails to authenticate, but it is just easier to trigger the flow by a cron injector (actually I use node-red-contrib-pkjq as this is better maintained) and then exec the python script.

Reply

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

    Accept cookies Cookie settings