Rain gauge

[pkkrusty]

Hi all, I've been working on a simple rain gauge, and couldn't find anything already made that quite met my requirements, so here is the result of my muddling through it.

I'm using a tipping rain gauge easily found on aliexpress for about $16 shipped. https://www.aliexpress.com/item/2027467349.html

This is attached to a D1 Mini on D5 and GND

CounterDebounce should be set to around 200. For this rain gauge, interior area measures 11cm x 5cm, minus four small rounded sections in the corners. After some math, I came up with 54.7854cm^2. I used a 60ml syringe to slowly count the pulses, and got 36. Some more math means that each tip of the bucket equals .3mm of rain.

Thanks to some help on the Discord channel, I was able to write a set of rules that calculate the rainfall per hour and send a MQTT message to a Node-Red node that transforms it into a format where it gets written to a simple MySQL table for later post-processing.

My goal was to do as much pre-processing as possible within Tasmota itself, so I did the math to convert tips to mm of rain using the scale command. Rules are as follows:

Rule1
  ON System#Boot DO Var3 %mem1% ENDON
  ON Time#Minute|60 DO Backlog Counter1 ENDON
  ON Counter1#Data DO Var1 %value% ENDON
  ON Counter1#Data DO Var2 %value% ENDON 
  ON Counter1#Data DO Sub1 %var3% ENDON
  ON Counter1#Data DO Var3 %var2% ENDON 
  ON Counter1#Data DO Scale1 %var1%,0,10,0,3 ENDON
  ON Counter1#Data DO Backlog Publish closet/tele/rain {"DateTime":"%timestamp%","Amount":"%var1%","TotalTips":"%var3%"}; Mem1 %value% ENDON

Explanation:
On System#boot, var3 is re-assigned the previous Counter1 reading so the subtraction and calculation can happen
Every hour, the command Counter1 is executed which will read out the current value (note that BACKLOG is necessary here for the following Counter triggers to execute successfully)
When Counter1 is triggered, var1 and var2 are assigned the value of counter (because var1 is going to be altered)
Var3 (previous counter value) is subtracted from Var1 and Var1 is overwritten with that new value
Var3 is then written with the new (now old) value of Counter1
Var1 is then scaled to 30% which changes the difference (number of tips in the last hour) to millimeters of rain.
Then the data is published in a format that my Node-Red can deal with and mem1 (persistent through a reboot) is written with the latest Counter1 value so subtraction will work in case power is lost (because var1-var3 are not persistent).

Note that this all works because Rules that execute on the same trigger will execute sequentially. Also note that I couldn't wrap things in backlog because backlog introduces a 200ms delay between commands and in that time, other following triggered rules will have executed.

Node-Red is subscribed to the topic closet/tele/rain and then goes through a JSON node to turn it into a standard JSON object (that's probably possible within the Tasmota message but I couldn't figure it out) and then a function node turns it into a MySQL insert statement.

Function is as follows:

datevar=msg.payload.DateTime;
amountvar=msg.payload.Amount;
totalvar=msg.payload.TotalTips;
msg.topic="INSERT INTO `SensorData`.`RainData` (`DateTime`, `Amount`,`TotalTips`) VALUES ('"+ datevar +"','"+ amountvar +"','"+ totalvar +"');";
return msg;

On my Raspberry Pi I used the following guide to install and enable syslog logging to a MySQL database (which is much easier/faster to search and deal with than the CSV file I was using before.

https://pimylifeup.com/raspberry-pi-mysql/

From there, I created a new database for sensor logging and a new table for rain data.

[barbudor]

Hi @pkkrusty Thanks for the nice write up 👍
Your text will look nicer if you use code marks for Rules using the <> button

When it comes to storing data from sensors, there is a special kind of databases named Time Series Databases which most well known example is InflluxDB. Other exist like Prometheus, AWS Time Stream, ....
This can then be very easily interfaced with graphical tools like Grafana.
All those can run on your RPi, preferably using Docker.

MySQL is a general purpose relational database which is a bit heavy solution for data that always come in series with a timestamp. Time Series Databases are lighter and faster when it comes to handle data across large period of time. For example when you will have stored 1 or 2 years of rain data and want to draw graphs on long periods, you will have to get all the data from the database and have to handle many points. TSDB can process data on the fly and returns computed min, max, mean, ....

Also you don't need to have a JSON node if you select "Output: a parsed JSON object" in your MQTT subscriber node.

[pkkrusty]

Thanks for the ideas. I'll take a look at TSDBs.

JSON node: Yes, I see that now. What is causing issues for me is when I take out the quotation marks for the %timestamp% and %var1% and %var3%. I assumed taking them out would turn the JSON object into one where the Amount and TotalTips would be numbers instead of a number surrounded by quotes TotalTips:30 vs TotalTips:"30" but that doesn't seem to be working.

Unrelated to the project: What's the rationale behind running things like InfluxDB, Hassio, etc in a Docker container vs just installing straight onto my Pi?

[barbudor]

Docker makes managing upgrades easier.
You can download the image for the new version, then switch to it in a matter of second. If not happy revert to the previous once immediately.
You can also stage the new version and test it in parallel of the current "in production" version.
You can test a new software and remove it in a perfect way (100% sure that you will not leave any remaining package)

[sfromis]

Installing things in containers like with Docker also avoids potential conflicts if applications have incompatible prerequisites.

About JSON - it would make sense (be most logical) to leave numeric data as numbers, not as strings, meaning no quotes around. OTOH, the timestamp is not a number, and thus it must be quoted as a string.

[sillyfrog]

Awesome, thanks for this, I'm just setting it up now and it's looking great (using the same rain guage). I've made a minor change as per below to the rule:

Rule1
  ON System#Boot DO Var3 %mem1% ENDON
  ON Time#Minute|60 DO Backlog Counter1 ENDON
  ON Counter1#Data DO Var1 %value% ENDON
  ON Counter1#Data DO Var2 %value% ENDON 
  ON Counter1#Data DO Sub1 %var3% ENDON
  ON Counter1#Data DO Var3 %var2% ENDON 
  ON Counter1#Data DO Scale1 %var1%,0,10,0,3 ENDON
  ON Counter1#Data DO Event Rain=%var1% ENDON
  ON Event#Rain<0 DO Mem1 %var1% ENDON
  ON Event#Rain>=0 DO Backlog Publish rainfall/tele/rain {"DateTime":"%timestamp%","Amount":"%var1%","TotalTips":"%var3%"}; Mem1 %var1% ENDON

I found that sometimes when the device resets (eg: power gets disconnected), in my configuration, the latest number are not written to flash, this means the Amount becomes negative. The above change means if this happens, that value will be skipped (and the information lost, but it's going to be wrong anyway), but Mem1 will be updated. All other times (even if zero), the value is still sent via MQTT.

[pkkrusty]

I noticed a few times that I was getting negative numbers, and was never around to catch the problem in action. Sounds like you figured it out. Thanks. I'll wrap my head around your changes and edit the initial post.

[shanekuz]

Thanks for the great write up, i have built the same but for each tip of the bucket the count goes up 5 is thats what happening for others?

[pkkrusty]

No, just one. Have you messed with debounce timing? Initially I would get two counts each tip because the magnet would pass by, creating a rising edge and falling edge, so I adjusted the debounce to 200 I think so the system would ignore that falling edge.

[shanekuz]

thanks will take a look again

[shanekuz]

thankyou that was it, it was set to 0

[GasTurbineMan]

Great work! I ordered same unit and in process of getting it set up now. Just curious if you are using this rain gauge as your PRIMARY collection device (that is, just putting it out in the rain directly)? My concern with just putting this out in the rain directly is that is seems pretty shallow and wonder how much collected rain is lost due to splashing out? Somewhere I got the idea that this device gets installed in a "weather station" which has a collection pan somewhat deeper that is funneled into this unit. I guess I will find out - as my plan is to use this as the primary collection device.

Regarding the Rule1 and MQTT data, do you ever RESET the counter back to 0 ?

[pkkrusty]

Yes. I also wondered about rain splatter, but never did the work to measure it against a known gauge. I may try this coming rainy season. You could always just make your own bigger/deeper funnel and do the math in Tasmota, like you're suggesting. Would help protect the device from UV damage also.

You could reset the counter to zero (I probably will before our rainy season starts), but no real reason to. Note the comment from @sillyfrog that fixes an error with occasional negative numbers. I need to incorporate that into the write-up.