Dockerized Power Meter Reading
I've dockerized my Centos 7 meter reading application, if you want to check it out, it's over on my github account here.
I've created a docker-compose.yml file to build a set of 4 machines, with persistent storage. All you need us a software defined radio. I purchased mine on Amazon for less than $40.
Here I'll be describing a little how-to of how the containers work, and how to use them.
version: '3.1' services: weatherdb: image: mariadb restart: always environment: MYSQL_ROOT_PASSWORD: weatherdb ports: - 3306:3306 volumes: - weather-mysql:/var/lib/mysql networks: - weather grafana: image: grafana/grafana restart: always ports: - 3000:3000 volumes: - grafana-storage:/var/lib/grafana networks: - weather depends_on: - weatherdb metermaid: build: ./metermaid/ ports: - 1234:1234 volumes: #- metermaid-storage:/app - /dev/bus/usb:/dev/bus/usb privileged: True networks: - weather depends_on: - weatherdb weatherscript: build: ./weather/ restart: always networks: - weather depends_on: - weatherdb volumes: weather-mysql: grafana-storage: metermaid-storage:
The yaml is pretty straightforward actually, which surprised me about docker. As you can see I'm starting 4 machines.
weatherdb - a straight MariaDB host, here because this app doesn't need to be secure, and to simplify setup, I'm passing the password using an environmental variable. You can access this machine from a mysql client by connecting to
mysql -h$dockerhost -uroot -ppasswordon normal port 3306. The volume is a persistent type, you can tell because it's a name before the column, instead of a path.
grafana - this machine is also just an official grafana docker container from docker hub. The only difference I have made is persistent storage and a dependency on the MariaDB host. You can access this machine using
metermaid - metermaid is fairly specific docker container, I'll post the docker code below. It is a centos 7 official instance, that I install prerequisites on. This instance will start rtl-tcp and handle all connections to the radio, and saving of all meter data to mysql. The entrypoint is the metermaid python application.
Please note the metermaid machine needs to run with privileged, so it can see usb, which is mounted (
/dev/bus/usb:/dev/bus/usb). the metermaid machine will basically have root access, but being a specific purposed machine, meant for home users, this should be safe. It uses a network of its own and does no actual web calls. It's still safe to port forward 3000 to grafana, its a completely different machine with no privileges.
From centos:latest RUN yum -y update RUN yum -y install wget vim tar epel-release git nc tee tree && yum -y install rtl-sdr python34 python34-mysql python34-pip RUN wget https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz RUN tar -xvf go1.10.1.linux-amd64.tar.gz && mv go /usr/local/ RUN mkdir /projects/ RUN mkdir -p /app/bin && cd /app/bin ENV GOROOT=/usr/local/go ENV GOPATH=/projects ENV PATH=$GOPATH/bin:$GOROOT/bin:$PATH:/app/bin/ COPY code /app/bin/ RUN python3 -m pip install pymysql RUN /usr/local/go/bin/go get github.com/bemasher/rtlamr && /usr/local/go/bin/go get github.com/bemasher/rtlamr-collect RUN python3 -m pip install mysql-connector-python simplejson mysql-connector requests ENTRYPOINT python3 /app/bin/metermaid.py
- weatherscript - This machine is similar to the other custom instance, I'm just changing the entry point to a for loop to hit the weather API every 5 minutes, and store the data. I'll leave my API in there for now, but if you enjoy this script, please feel free to edit the script, and sign up for your own free account. I believe I used openweather API.
From centos:latest RUN yum -y update RUN yum -y install wget vim tar epel-release git nc && yum -y install rtl-sdr python34 python34-mysql python34-pip RUN wget https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz RUN tar -xf go1.10.1.linux-amd64.tar.gz && mv go /usr/local/ RUN mkdir /projects/ RUN mkdir -p /app/bin && cd /app/bin ENV GOROOT=/usr/local/go ENV GOPATH=/projects ENV PATH=$GOPATH/bin:$GOROOT/bin:$PATH:/app/bin/MeterMaid COPY code /app/bin/ RUN python3 -m pip install pymysql mysql-connector-python simplejson mysql-connector requests ENTRYPOINT bash /app/bin/run.sh
So how do I use this?
Decide where you want the machine to run, I have a home server so it was a decent choice since the machine is always running. The nice thing about docker is Host OS should not matter. These instructions are for a GNU Linux host. Specifically Centos 7.
sudo yum install docker docker-compose
Once installed grab my git repo here.
$ git clone https://github.com/jreiners/metermaid-docker.git $ cd metermaid-docker $ docker-compose -f metermaid.yml build --no-cache
This part takes a few minutes, and as you can see, it will download, and completely set up a working environment that will collect all readings it can see and saves them to persistent SQL. You can hit control-C if you can log into sql, and hit grafana on 3306(mysql) and 3000(grafana)
once it's running there are a couple finishing steps you need to do. Which is create the database, and two tables to hold your meter data and weather info.
[justin@localhost ~]$ mysql -h192.168.1.163 -uroot -pweatherdb Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 389 Server version: 10.3.7-MariaDB-1:10.3.7+maria~jessie mariadb.org binary distribution Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MariaDB [(none)]> create database metermaid; Query OK, 1 row affected (0.00 sec)
This stack will autorestart if anything stops working, and will even start on reboot if you make sure docker is in startup (
systemctl enable docker)
You can find your meter serial number on a sticker on the meter itself. Electricity, gas, and water meters all should have a white sticker. Please keep this in mind: sometimes you cannot find one or two meters. My water does not work at my place, the meter has a label, but it just never communicates in a way the software-defined radio can see it. This could be a different encoding, or frequency higher or lower than my radio can see.
Please let me know what you think!
If you run into issues, please make sure nothing else is using your SDR and it's plugged into the machine and working. on the host
metermaid needs the following two tables, and one database created on first boot. these will persist.
metermaid needs the following database, and two tables created. You can use the mysql client, or use the query editor in grafana.
create database metermaid
CREATE TABLE `utilities` ( `mPrimaryKey` int(6) unsigned NOT NULL AUTO_INCREMENT, `mId` varchar(50) DEFAULT NULL, `mType` int(2) DEFAULT NULL, `mTime` int(11) DEFAULT NULL, `mTotalConsumption` int(11) DEFAULT NULL, `mConsumed` float DEFAULT NULL, PRIMARY KEY (`mPrimaryKey`) ) ENGINE=InnoDB AUTO_INCREMENT=142010 DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED
CREATE TABLE `weatherdb` ( `id` int(11) NOT NULL AUTO_INCREMENT, `epoch` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(), `mintemp` float DEFAULT NULL, `maxtemp` float DEFAULT NULL, `currenttemp` float DEFAULT NULL, `windspeed` float DEFAULT NULL, `winddir` varchar(3) DEFAULT NULL, `text1` varchar(30) DEFAULT NULL, `text2` varchar(30) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB AUTO_INCREMENT=250 DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED
Grafana example queries:
SELECT mId as metric, mTime as time_sec, mConsumed as value FROM UtilityMon.UtilityMeter WHERE mId=5744xxxx and $__unixEpochFilter(mTime) ;
SELECT mId as metric, mTime as time_sec, mConsumed as value FROM UtilityMon.UtilityMeter WHERE mId=6592xxxx and $__unixEpochFilter(mTime) ;
Neighborhood Electric Meters:
SELECT mType, mId as metric, mTime as time_sec, mConsumed as value FROM UtilityMon.UtilityMeter WHERE mType=7 and $__unixEpochFilter(mTime) ;
Neighborhood Gas Meters:
SELECT mType, mId as metric, mTime as time_sec, mConsumed as value FROM UtilityMon.UtilityMeter WHERE mType=12 and $__unixEpochFilter(mTime) ;
My old metermaid machine was lost in an outage, so I need to redo my weather query. I'll update this page when I get it graphing with temp again in grafana.