Energy monitoring - going beyond smart meters...

Update: I’ve just found that there is a Modbus interface listed in the OEM Shop.

If the hardware connection has been developed, then I must assume that there is some software support for it!

I’m adding it to the order about to be placed, and I’ll have a look at the software later.

That all sounds like it’ll keep you busy for a while.

Have fun!

I’m hoping this will keep US busy for a while. We have different skill sets and will need to collaborate!

Rest assured, there are few things I enjoy more than collaboration :-)

Tagging in @Tim_OVO and @Transparent  because they’ve both had/still have an interest in “missing data” and “estimation” which this comment is about…

Roughly speaking, “cleaning up” the data involves things like the following...

For electricity data, one combines actual meter readings (the ones taken at the end of each billing month, i.e the ones printed on bills) and usage data (“usage” is the only clue to what is going on during the month). Usage data are available online in My Ovo, Usage (both daily and half-hourly). Combining these leads to an upper bound and a lower bound, which vary with time, on what the meter reading could be at any time during the billing month.

The following graph shows these two bounds for my consumption, over the last 2 years. Any acceptable estimated meter reading has to lie between these two lines. They look like a single line because, on this scale, the orange line covers the blue:

 A higher or lower reading is logically and physically impossible.

There were a couple of days in May when DCC may have had problems, because Hildebrand’s Bright app is missing data too (albeit at slightly different times). Zooming in a lot:

The difference between these bounds, in kWh, is plotted in the graph below - it’s always within 1 kWh of zero, and the missing data in May results in the largest positive spike, to the heady heights of 0.6 kWh.

This is all done in an approximation where I gloss over the fact that the timings of daily and half-hourly usage data are not aligned. It’s ignoring this “apportionment” that causes the difference between my upper and lower “bounds” sometimes to have the wrong sign. But it isn’t hard to get very plausible estimates.

I’m moving on to look at the gas data (which is make more intricate by having to take out CV to convert energy usage in kWh into increments in meter readings of gas volume. The trick in dealing with missing data is to combine meter readings with usage, and this is where CV comes in.

But I’ve just been forcibly reminded that Ovo is still sometimes presenting me with negative values for daily gas usage. What on earth they’re playing at, I’ve no idea ...

… but the error boils down to a failure to make intelligent use of half-hourly usage data, when the midnight meter reading (the one that they don’t bother to record any more, because “they don’t use it”) and latest daily usage data get lost in the communications system.

If I sound exasperated, that’s because I am. Getting the processing and analysis of data right, even incomplete data, really isn’t as difficult as Ovo make it appear.

PS The reference to “the orange covering the blue” rings bells, but it would have been too distracting to leave the following aside where it was in the explanation above:

(Cue joke about watching snooker on a black and white tv. “Whispering” Ted Lowe. I’m showing my age, and a slightly mis-spent youth...)

Your progress is faster than mine @Simon1D - I’m still busy ordering components.

I’m building my Pi & eMonPi Shield boards into a configuration which includes an ‘always on’  5v UPS. Firstly the instructions state that a power-loss during operation can destroy the SD-card(!)

… but I’d also like to monitor what interesting anomalies occur when power fails and is then restored.

Here’s a photo of my well-pump meter taken on 4th March ‘21 when power was being restored from a damaged 11kV supply:

A mains supply hovering outside the permitted 216-253v can do all sorts of damage to equipment. In this case the fault remained for around 3 or 4 minutes whilst I took several photographs.

I’m only playing with numbers at the moment (which is easy for me) as a displacement activity from trying to understand what my new oscilloscope can tell me about optical pulse mis-counting (which is all quite far out of my comfort zone).

Of increasing urgency is the setting up a bit of temperature monitoring on various pipes - I would like to see how inefficient our old boiler is, by comparison with its replacement, before it’s ripped out a week tomorrow. Arguably this should be easy for a retired metrologist used to analysing temperature data sensed with microK precision (and at room temperatures, not cryogenic) but I was the theorist and had experimental colleagues to build the equipment and make it work ...

In a previous decade, other colleagues spent a LONG time fretting over the imminent reduction in mains voltage from 240 to 235 (their work depended on a 50 year old Van de Graaff generator that needed all the help it could get). In the end of course, mains voltage didn’t really change, only the allowed tolerance. But they wisely found a way not to rely on the ancient machine.

For the moment, I’ll just keep my fingers crossed re the risk of surges damaging my shiny new Emonpi things.

We seem to have very complementary skills @Simon1D 

So on a practical note, when you followed the online instructions to first set up your OEM hardware, did you plug in a monitor screen to your Raspberry-Pi via the HDMI port?

After all, you’ve bought the option without the integral LCD in the expensive aluminium case, so you can’t see these messages:

Do I assume you’ve bought the metal-encapsulated temperature sensors from the OEM shop?

This contains a Dallas DS18B20 chip, manufactured by Maxim,  which sends data down a single wire (called a ‘1-wire bus’). So if the metal cylinder or the length of wire is constraining you, then you can pick up the 3-pin chip itself from loads of other suppliers.

Aside: @Tim_OVO or @Jess_OVO - can you check with the OVO Foundation to see if they’re supporting any electronics/programming clubs with students in this coming academic year?

The equipment and strategies we’re describing here would produce some excellent projects of direct relevance to the fight against Global Warming and the company’s push towards Net Zero.

I did. I also downloaded the pre-built image and ran the Emonpi software on the old Pi-3 for a day or two before any hardware arrived, sitting in comfort at a desk with keyboard, screen and mouse. It made me rather unsure of how to configure things, but it works, and I haven’t got around to naming the feeds properly. With hindsight, I’d have been better to start afresh with a completely unconfigured system as soon as the hardware arrived.

Do I assume you’ve bought the metal-encapsulated temperature sensors from the OEM shop?

This contains a Dallas DS18B20 chip, manufactured by Maxim,  which sends data down a single wire (called a ‘1-wire bus’). So if the metal cylinder or the length of wire is constraining you, then you can pick up the 3-pin chip itself from loads of other suppliers.

I did buy one, yes, but I haven’t plugged that in yet, having little interest, at the moment, in temperature where the meters are.

I want to monitor 3 temperatures at once (e.g. flow and return in one place, another temperature somewhere else), and plan to use my existing 3 thermistors and more Raspberry Pi/Picos … (I’ve made up some nice long leads too...

Indeed we do - ideal for collaboration …

(but not till after a trip to the pub...)