Energy monitoring - going beyond smart meters...

@Simon1Dwrote:

For example, I would really like to know if the boiler is cycling.

And please note @earthpoint  that my gas boiler doesn’t cycle repeatedly, but I’ve implemented another solution for that problem which doesn’t require live monitoring.

That’s the benefit of a Forum like this. You can get to read different ideas and then decide which best fits what you’re looking for.

FWIW, my solution doesn’t require live monitoring either - the cylinder is being replaced next week, the new one has two heating coils instead of one (solar thermal panels too, maybe, in a few months), and they are more efficient, to boot. And, just because it’s obviously a good idea, I’ve asked the installer in the meantime, pending the possible installation of those solar panels, to connect up the second coil in series with the first. If I can increase the efficiency of heat transfer between boiler and hot water cylinder, why would I not do that? And yet, such details are, so far as I can tell, not standard practice.

Anyway. I’ll bet that solves the cycling problem. In the meantime, I’m in observe and interpret/understand mode. I can wait till next Wednesday for a more intelligent solution…

But this forum (and other fora elsewhere, specialising in different aspects) are where such ideas are worked out, tested, exchanged, and evolve.

The majority of component parts for my own Energy Monitor have now arrived:

 

 

1. emonPi shield for sensor inputs
2. Raspberry-Pi model B+ with additional GPIO connecting cable
3. LCD display 1602 with I²C interface (concealed underneath)
4. ModBus interface module with integral DC-DC converter
5. Serial converter connecting ModBus interface to Raspberry-Pi
6. Lithium battery charger and 5v DC-DC boost converter
7. Lithium cell, 3400mAh to maintain monitoring during a mains fault
8. Antenna, 433MHz; future expansion for remote sensors
9. Smart Meter pulse optical-sensor
10. Current transformer sensor, 100A max

Power-supply and connecting leads between modules not shown in this photograph. The pink bubble-wrap is an anti-static layer to prevent damage to sensitive electronics.

Wow it’s arrived - very exciting! 

I’m looking forward to hearing how this project evolves, now that you’ve got some hardware. If you’re able to get valuable, reliable data and therefore insight into your electricity usage, I’ll be sharing the news of this project throughout OVO to show what is possible. 

Yes this isn’t something we’ll expect our members to replicate, but it looks like powerful collaboration and DIY innovation to me!

Node-RED has some quite active modbus users and contributed nodes. 

Yes, I’ve looked at this in the past. The cost made it a little too rich for me to invest in since our needs are very simple. 

I’ve always meant to get round to making up a simple sensor that can count the LED flashes on our electric meter. Just never quite made it to the top of the to-do pile though 

There is something you could do for us all please @knight

Whilst researching the matter of Energy monitoring and data-logging over the past three years, I’ve several times come across online comments mentioning Node-RED.

It wasn’t relevant when my research concerned substation monitoring on the OpenLV Project, because Node-RED wasn’t certified for use in that environment.

But it could save the rest of us some time if you give us an overview and explain why Node-RED popped up in connection with monitoring.

Thank you.

Hello energy monitor pioneers,

I wanted to check in on this thread, after seeing some of the fantastic insight @Simon1D was able to generate in this post, leading him to his security alarm system. 

From this, I’m able to presume that your energy monitoring systems are in placed and working? 

Hi, sorry about the delay in responding. 

Node-RED is a “low-code” development tool. It lets people create processes for automation and information processing without needing much, if any programming knowledge. It uses a flow-diagram kind of approach where you draw out a sequence of tasks in an editor and connect the tasks together with lines. For the technically minded, this is called a “graph” (the things most people think of as graphs are actually “charts”  ).

We describe the process as a “flow” with “messages” “flowing” between the tasks (which are called nodes - hence that part of the name).

It is actually easier to understand when you see it than it is to describe it in words! There are some great introduction videos on YouTube for those that want to learn.

It started as a tool developed in IBM for working in the field with Internet of Things but has since flown the nest and was gifted to the JavaScript Foundation and is fully open source and free.

It has also been embedded in all sorts of places. For example, a lot of people who use Home Assistant for home automation use the Node-RED addin because they can do some really complex processing very simply without any code needed. A number of commercial systems now also have Node-RED embedded to enable users to create automations and data-processes without coding. Siemens, Hitachi and various others are doing this.

Node-RED is all based on JavaScript and it is quite easy for people (like myself) to contribute nodes. There is a thriving community of developers and thousands of contributed nodes that do all sorts of tasks whether interacting with external systems like Hive, Nest, Alexa, etc;  Comms services such as Telegram, Email, WhatsApp, etc and much more.

It comes with a very easy to use User Interface development tool called simply “Dashboard” that also requires no programming knowledge. Of course, that inevitably has some limitations once you want to get more complex processing and interfaces. That’s where my own main contribution comes in, “uibuilder” - which lets you use standard web development knowledge to easily build data-driven web applications.

I also developed a module that can be used in Node-RED and other Node.JS (a JavaScript-based server that Node-RED is built on) applications that talks to the Drayton Wiser smart heating controller. It lets you get the latest info, listen for changes (so that you can trigger other actions) and update settings.

Node-RED was originally built around the use of MQTT - which is a service that you can send messages to and allows other systems to subscribe to such that they get updates when information is updated. MQTT is lightweight, easy to install and works on very limited platforms such as Raspberry Pi’s (as does Node-RED) and is ideal for IoT systems including, of course, home automation. Many commercial and open source systems and services use MQTT as a message broker. For example, the Shelly remote switch modules have MQTT options. This allows smart IoT modules to have a standard way of exchanging information and Node-RED can take full advantage of this.

Here is a typical example of a simple flow in Node-RED that does some lighting control:

As you can see, it makes extensive use of MQTT (the pink and grey nodes), timers (the 3rd blue node). Many of the MQTT subscriptions are listening for messages created by other flows. Such as the daylight on/off message that is generated by another node that knows about sunrise and sunset. You can also see in the bottom orange node (which is some JavaScript code - you can easily include custom code in a flow) that the flow is also listening to updates from a custom sensor that I built that sends out information about temperatures, humidity and light levels. The light is turned on if the light level falls too low unless it is sleep time.

Here is another example that shows how simple it is to do something really useful. This sends a warning message to Telegram when the front door is opened or closed.

The actual output to Telegram is another flow that listens for a particular MQTT message. I use a lot of function nodes because I know JavaScript but if you don’t, you can use a few simple extra nodes.

Node-RED also has a thriving forum which is one of the friendliest forums on the Internet (and I’ve used a LOT of forums over the years) where beginners and experts alike share and ask for help. The core developers are always on there in addition to a bunch of other regular contributers like myself.

Tagging in @Tim_OVO and @Transparent

...

There were a couple of days in May when DCC may have had problems, because Hildebrand’s Bright app is missing data too (albeit at slightly different times). Zooming in a lot:

The difference between these bounds, in kWh, is plotted in the graph below - it’s always within 1 kWh of zero, and the missing data in May results in the largest positive spike, to the heady heights of 0.6 kWh.

 

The more I read about this, the more I am getting the feeling that the whole smart meeting system is poorly designed. There should not be any deltas in the whole system, it should all be based on meter readings. As you show nicely, the mixed use of both leads to huge complexity and uncertainty that could easily be avoided by only using meter readings *with decent resolution*. If a meter reading is missing, the uncertainty is completely bounded by the next two meter readings, but if usage data is missing, the uncertainty could span over very long times. 

I do agree. However, as they say,

We are where we are.

... and I can only speculate on the colloquial name for the waterway we've arrived in and how, precisely, our equipment might be lacking …

;-)