Energy monitoring - going beyond smart meters...

Indeed it is. There’s a huge amount of stuff that is way beyond anything I’ve got my head around so far, and the monitoring of energy is only a small part of it all.

The heatpump monitoring stuff will surely be of interest to some on here, as will the monitoring of solar PV.

I use Home Assistant with modbus to get my solar data from the Solaredge inverter.

Thanks @Simon1D - In order to keep others informed (incl @Tim_OVO ) can you please give brief explanations of:-

a: the blob on the front of your Smart Meter which you refer to as the Optical Pulse Sensor

b: the contents of the black box with the sticky-up white wire

c: How you are getting those graphs from the black box… without using the phrases

• IP address
• protocol
• server stuff 

Hopefully that gives the Moderators a fighting chance of keeping up on where we’re heading with all this.

 Will do my best, @Transparent …

a: the blob on the front of your Smart Meter which you refer to as the Optical Pulse Sensor

 

 Like many/most/all smart electricity meters, mine has a lot of LEDs that flash from time to time. One of these relates to the energy supplied. Here’s a photo I took before attaching the “optical pulse sensor”:

Zooming in a bit

That black hole by the “1000 imp/kWh” label has a red LED that flashes once every time the total energy supplied goes up by another 0.001 kWh. There is more detail here https://shop.openenergymonitor.com/optical-utility-meter-led-pulse-sensor/

That sensor produces an electrical signal. Out of curiosity, I tried to measure the signal that came from one LED pulse with an oscilloscope. Here is the result:

This is not a feature of the standard OEM system. I (now) have my own PC oscilloscope.

That, and the signals from any other sensors are fed into a box of tricks called “EmonTx”, also described in more detail here

https://shop.openenergymonitor.com/emontx-energy-monitor-transmitter/

which broadcasts that information on a 433 MHz radio signal that can get through the brick wall to the black box on the other side.

b: the contents of the black box with the sticky-up white wire

 

The sticky-up white wire is just the radio aerial on this:

described here https://shop.openenergymonitor.com/rfm69pi-433mhz-raspberry-pi-base-station-transceiver/ which I attached to my Raspberry Pi3 (the only other thing inside that black box).

There was a one-time only process of setting things up, in which I told the Pi how to connect to my home Wi-Fi network (it remembers the network name and password, etc.)

This system is running all the time. I can look at it from my laptop by pointing a web browser at http://emonpi.local, I log in and am presented with a “dashboard” offering a summary of how things are.

OK. I confess that last sentence is a lie - I haven’t actually set up a “dashboard” yet, I just get a menu of inputs (an “input” has usually has sensor data, a value that changes every 10 seconds) and feeds (a “feed” is what you get if you keep track of an input over a period of time - i.e. timeseries data such as can be plotted:

and one of the things I can do with a feed is look at it. That is how the graph appears on the screen, and I can then zoom in, plot and compare multiple feeds, etc. to my heart’s content.

For the sake of others who come across this topic in the following months, may I point out that the Raspberry Pi is a British-designed micro-controller board. It uses the ARM processor family, originally created by Acorn Computers Ltd and most often found in smart-phones.

The Raspberry Pi Foundation has its own website with lots of background information and projects to get you started.

The two main suppliers of Raspberry Pi boards and add-on modules in the UK are

• the PiHut a dedicated online shop for Pi stuff!
• CPC/Farnell, which is a supplier of electronic components

Although the Model-4 is the latest iteration of the Pi, you will note that @Simon1D has achieved the above tests using the earlier/cheaper Model-3 version. That is still being kept available at just under £34 from CPC.

The Pi is a complete computer which can run a number of operating systems, although most users choose Raspberry Pi OS, derived from Debian Linux.

The board allows connection of a monitor (via HDMI), keyboard, mouse, network and solid-state drives. As you can see from @Simon1D’s illustrated use above, it can also be left running without a screen and keyboard attached. That makes it a very cheap and powerful way of gathering data from sensors.

Thanks @Transparent, but perhaps I should add that my “achievement” was really only to download the “image” of the carefully tweaked version of Raspberry Pi OS which has the Emoncms software pre-installed, and write that image to a microSD card.

I do like free software (free as in beer, as well as free as in freedom).

Those that prefer to buy a card preloaded with this image will find it here. If you only have the even earlier Model-2 Pi (no longer available to buy?) OEM say Emoncms will run on that too.

For what it’s worth, I downloaded and played with the Emoncms software for a day or two before deciding to buy the hardware (sensors, etc). In this way, it is possible to get some idea of the capabilities of the system (within a day of such playing, I was convinced that it was worth trying for real).

For instance, I worked out how to upload my own Ovo smart meter data and review it all in Emoncms. Here, for instance, is the not-quite-up-to-date electricity consumption (these are kWh per half-hour) over the past year

And the same for gas (kWh per half-hour):

[There are no MAXINT values in this period]

Such fun to be had …

I have been known on here to advocate ways to look at “usage”, other than kWh/half-hour (better perhaps to call this “demand”, and express it in W), such as meter reading (or “consumption”, in kWh). Inevitably, Emoncms can do the sums to convert one into the other:

I’ve no doubt it could look up the national grid’s CV data and recover gas volume in m^3, with a bit of effort. Perhaps not surprisingly for a monitoring system, Emoncms is not very tolerant of missing data (my gas data on My Ovo has a lot of gaps), and is very ready to decide that a feed is “inactive” and abandon processing it. This is why the gas kWh stops after a few months.

I now have no need to create my own software to review data as I want to. Emoncms appears to be already set up to do what I want. This gives me an extra incentive to solve Ovo’s problems with missing data ;-)

It is ages since I have used Modbus (both the protocol and I were a long younger...), but I am happy to help where I can. It is not a difficult protocol, and it very much follows a standard read / write access model. One of the things to be aware of is that it does not support consistent snapshots, so that needs to be handled on the application layer (nasty!). 

That’s good news @MrPuds - I’ve been reading up on Modbus for the past 3 months now and comparing the commands/data available from various energy meters.

I don’t understand SCADA software nearly well enough (yet). Most offerings seem vastly too complex for the level of energy information we’d like to have available for most Forum Members. I think we need something open-source (free) and supported by a group of enthusiasts.

Most SCADA software is expensive and includes ongoing licences/maintenance contracts. Ugh!

I spent about 2 hours yesterday reading through the various solutions being offered by the Open Energy Monitor project. I’m now at the point of ordering some bits, although I’ll be using wired connections, rather than the 433MHz link which @Simon1D has bought into.

At this moment in time I haven’t yet discovered if the OEM Project hardware/software has any possible option to run an RS485 (Modbus) channel.

Everything I read yesterday concerns the use of three input devices:

• an optical sensor reading meter ‘pulses’
• a current transformer
• a voltage sensor, which plugs into a 13A mains socket

In other words, this is doing only monitoring, and no controlling or configuring. What I’d also like to see is Modbus devices being implemented for

• reading data from DC circuits (eg PV Solar Panels)
• controlling devices on/off in response to software scripts

So that’s where I’m currently at.

And a note to @Jess_OVO & @Tim_OVO - my investigations yesterday have given me a couple of ideas in which these energy monitors could be used to do a Smart Meter Check. So that would be very useful tool to deploy for situations like the one we’ve recently been investigating.

Let’s not worry for the moment how the customer acquires the device and the cost. I already have some possible solutions.

But first let’s see if we can better understand these (relatively cheap) monitors and learn how to configure them to provide useful/interesting data.

I see you’re as easily persuaded as I was ;-)

I look forward, where useful, to comparing notes on our respective experiences …

Modbus isn’t of direct interest to me, so I didn’t follow much beyond that link, but they strike me as a very welcoming and helpful community of users at OEM, so if I were you, I wouldn’t hesitate to post your questions sooner rather than later, @Transparent.

BW

Oh I didn’t take much persuading, @Simon1D . I was already investigating a monitoring strategy using an Arduino rather than a Pi.

I’m on the OVO PowerVault trial of Storage Batteries, but the battery unit itself has no integral monitoring. Instead this is delivered via a ‘live’ graph from their website. But that graph incorporates digital filtering. So although I can sometimes momentarily see anomalies, such as drop-outs, these get ‘smudged’ as they are amalgamated with following readings.

I’ve also been monitoring a local substation for the past 3 years - a project which Western Power is now funding when the original grant from Ofgem ran out. There are eight community groups who have access to these smart transformers.

Due to that background, Western Power have kindly run a 3-phase feed to my house. Although I continue to use only single phase, I can have access to the other two for monitoring purposes. So that presents a whole host of useful data for investigation