The Hard'ack Technology Environmental Monitor

2020-01-27
Richard Heurtley
richard@heurtley.com

Introduction

This article is about a gas sensing assembly I built for the University of Vermont (UVM) Extension to be installed in a barn. The environmental monitor calculates parts-per-million (ppm) values for:

  • CH4: methane
  • CO2: carbon dioxide
  • H2O: water vapor
  • H2S: hydrogen sulfide
  • NH3: ammonia

Results are stored in a database and are also available in real-time over the Internet.

Pictures

Here are some pictures of the completed assembly. It is 9" tall not including the WiFi antenna, 8.5" wide, 8.5" deep including the splash shield, and weighs 4.75 pounds. Click on a small picture to produce a large picture:

Hardware

Controller

The controller is a Raspberry Pi 3 B+ running Raspbian Linux with a custom hat consisting of a Perma-Proto Hat board (Adafruit #2310) with two ADS1015 four channel A/D converter modules (Adafruit #1083) mounted on it. And, as long as I'm plugging Adafruit stuff, the BME680 module is an Adafruit #3660.

The Raspberry Pi has a built-in WiFi adapter but I'm using a USB WiFi adapter with a detachable antenna so the antenna can be mounted on the case top

Sensors

The gas sensors are mounted on the case door and are protected from projectiles and splashes by a sheet of polycarbonate. The environmental monitor is not designed to be exposed to weather. Most of the sensors are inexpensive MQ series resistive sensors sold by Zhengzhou Winsen Electronics Technology:

Gas Sensor Price Web Page Specifications
CH4 Winsen MQ-4 $5.00 link link
CO2 Winsen MH-Z14A $51.00 link link
H2O Bosch BME680 $23.00 link link
H2S Winsen MQ-136 $13.00 link link
NH3 Winsen MQ-137 $45.00 link link

In addition there are four 10K thermistor inputs.

Power Supply

The power supply section consists of the following components:

  • AC Power Supply: Mean Well RS-25-15: The input is 120VAC and the output is tuned to 13.5VDC to supply an intermediate power circuit to which the battery is connected. 13.5VDC is a constant voltage charge to the battery.
  • AGM Battery: NPP NP12-1.2AH: This maintains voltage on the intermediate power circuit when AC power is unavailable. This allows the environmental monitor to be unplugged and relocated quickly without the gas sensors cooling down.
  • DC Power Supply: Mean Well SD-15A-5: The input is 9.2VDC to 18VDC on the intermediate power circuit and the output is 5VDC to power the controller and sensors.

The AC power line is protected with a 120VAC 0.5A fuse. The battery's connection to the intermediate power circuit is protected with a 12VDC 2A fuse.

Switches

There are three switches:

  • AC Power: This disconnects the AC power line from the AC power supply. This is the same as unplugging the AC power cord.
  • Battery: This disconnects the AGM battery from the intermediate power circuit. The environmental monitor will still operate but there will be no battery backup.
  • Controller: This disconnects the controller and sensors from the DC power supply.

To turn the environmental monitor on, turn on the AC Power, Battery, and Controller switches in sequence. Data collection will start automatically.

To turn the environmental monitor off, first halt data collection, shut down Linux on the controller, and turn off the Controller, Battery, and AC Power switches in sequence.

Sensor Operation

The MH-Z14A CO2 sensor connects to the controller's serial port and returns digital ppm values. The BME680 H20 sensor connects to the controller's I2C bus and returns digital temperature, air pressure, and relative humidity. From these values the water vapor concentration in ppm is calculated. The MQ sensors vary their resistance in response to the concentration of the gas for which they're designed. The sensors' manuals each contain a calibration curve and correction curves for temperature and relative humidity. An A/D converter reads the voltage drop across a sensor's resistor. From the voltage the sensor's resistance is calculated and from the resistance the gas concentration in ppm is calculated.

The MQ-4 methane sensor's specification lists a minimum detection level of 300ppm. The calibration curves for all the MQ sensors were extended to 0ppm when the equations were derived. Methane concentrations from 0ppm to 300ppm may not be valid.

The water vapor concentration is derived from the current temperature, air pressure, and relative humidity. The range shown below is from (20F, 1Atm, 20%) to (75F, 1Atm, 90%).

Ranges:

Gas Sensor Low ppm High ppm
CH4 MQ-4 300 5000
CO2 MH-Z14A 0 5000
H2O BME680 450 16800
H2S MQ-136 1 200
NH3 MQ-137 5 500

Data

The data acquisition program, which is described more fully below, queries the sensors once per second. Values can be viewed in real-time by directly accessing the environmental monitor over the Internet. Five minute statistical summaries of the data are saved in the database.

The data acquisition program does a considerable amount of processing of the raw acquired data. In order to calibrate the sensors once the environmental monitor is located on site, raw and some intermediate values are saved in the database along with the final values. The list of values saved in the database, as of this writing, is:

Sensor Value Unit
BME680 Raw Temperature Celsius
BME680 Raw Temperature Fahrenheit
BME680 Raw Relative Humidity Percent
BME680 Air Pressure Hecto Pascals
BME680 Air Pressure Inches of Mercury
BME680 VOC Sensor Resistance Ohms
Power Supply At Perma-Proto Hat Volts
Power Supply At MQ Sensors Volts
Thermistor 0 Temperature Celsius
Thermistor 1 Temperature Celsius
Thermistor 2 Temperature Celsius
Thermistor 3 Temperature Celsius
Thermistor 0 Temperature Fahrenheit
Thermistor 1 Temperature Fahrenheit
Thermistor 2 Temperature Fahrenheit
Thermistor 3 Temperature Fahrenheit
CH4 MQ-4 Resistance Ohms
H2S MQ-136 Resistance Ohms
NH3 MQ-137 Resistance Ohms
CH4 MQ-4 Concentration ppm
CO2 MH-Z14A Concentration ppm
H2O BME680 Concentration ppm
H2S MQ-136 Concentration ppm
NH3 MQ-137 Concentration ppm
BME680 Adjusted Temperature Celsius
BME680 Adjusted Temperature Fahrenheit
BME680 Adjusted Relative Humidity Percent
Thermistor Average Temperature Celsius
Thermistor Average Temperature Fahrenheit

Configuration

The Hard'ack system is configured with text files. There are two primary configuration files that specify:

  1. Where and how to acquire data and where to store it in the controller.
  2. What to do with the data once it's been stored in the controller.

The environmental monitor's two primary configuration files, as of this writing, are:

  1. gather_config.txt
  2. scatter_config.txt

The primary configuration files include other configuration files one of which, as of this writing, is:

archive_config.txt

Data is organized two ways. In the controller it's organized by set and column. A set is a series of text files containing columns of numbers. A column is a column in a set. In the database data is primarily characterized by its position in the "location tree" described below.

Controller Data Text File Structure

Unless you're reading or writing Hard'ack configuration files you don't really need to know these details and can skip to the "Database Structure" section.

Data text files are named prefix_YYYYMMDD_HHMMSS.txt. Each set is given a unique prefix. In the environmental monitor set prefixes are defined in the archive_config.txt file. YYYYMMDD_HHMMSS is the UTC date and time the file was created.

The contents of data text files are tab delimited. The first column of each row is the UTC date and time the data in the subsequent columns was acquired. The format is "YYYY-MM-DD HH:MM:SS". Subsequent columns are columns numbered starting with zero and contain acquired data. Data text files contain no data context. There are no headers or titles to explain what each column contains.

The gather_config.txt file has many lines beginning with "SetXColY" that contain the specifications for the values to be stored in the files created for set X and column Y. Set and column numbers start with zero but the first (leftmost) column in a data text file is the row's acquisition date and time and not a column. The second column in a data text file is column zero.

Database Structure

In the database, data is primarily characterized by its position in a "location tree" of strict parent-child relationships. The location tree structure was created to model the interior of buildings. It doesn't work well outside that context. The layers of the location tree are:

Layer Parent Relationship Child Relationship
manager Commissioner of data acquisition. Manages one or more complexes.
complex Managed by its manager. Composed of one or more buildings.
building Part of a complex Composed of one or more floors.
floor Part of a building. Composed of one or more units.
unit On a floor. Composed of one or more rooms.
room Part of a unit. Contains one or more devices or things to measure.
device Something in a room to be measured. Colonized by one or more ppoints (sensors).
ppoint (Physical Point) A sensor mounted on a device. Produces one or more points.
point A sequence of numbers produced by the ppoint. May have a unit designation.

A device may be a boiler, a water tank, a pump, an electric power line, or the interior environment. A ppoint may be a gas sensor, a thermistor, a voltage or current sensor, or a register in a smart sensor. A point may also be a value calculated from other points.

An example of a point's designation on the location tree for the environmental monitor is:

LayerName
managerUVM Extension
complexPhilo Ridge Farm
buildingBarn 1
floorGround
unitCentral Hall
roomCentral Hall
deviceInterior Environment
ppointMQ4 CH4
pointCH4 Concentration

Units are not part of the location tree but the point illustrated has the unit "ppm".

Point data is stored as a series of database records each containing a statistical summary of the point's values for a period of time. The fields of a record are:

  • UTC date and time of the first sample
  • UTC date and time of the last sample
  • Number of samples
  • Highest sample value
  • Lowest sample value
  • Sample average value
  • Sample standard deviation
  • Sample median value

It is possible to store discrete samples in the database. In that case the first and last sample times will be the same and all values will be the sample value except standard deviation which will be zero.

The environmental monitor samples data once per second and stores five minute statistical summaries in the database. If there are no acquisition problems then each database record is a summary of 300 samples.

Hard'ack Project History

Many years ago Housing Vermont, a Burlington, Vermont based developer of affordable housing, asked me to develop a system to monitor the operation of the boiler rooms of various Housing Vermont buildings. I called the system "Hard'ack" which is a loose acronym of "Housing Vermont Data Acquisition" and the name of a popular ski hill in St. Albans, Vermont. Housing Vermont is no longer involved in Hard'ack development.

"Hard'ack Technology" is just a name I made up.

Hard'ack Hardware

The Hard'ack programs run under both Windows and Linux but Linux is the intended platform. The following controllers have been put into production:

Hard'ack Software

The Hard'ack system consists of a set of programs that run on the controller and another set of programs that run on an optional database server. The Hard'ack programs are written in the C language and run under both Windows and Linux.

Hard'ack Controller Software

Gather

The gather program queries sensors and produces numbers that are stored in a series of tab-delimited files on the controller's equivalent of a hard disk drive. A minimal Hard'ack installation runs just the gather program. Sensor specifications are entered into a configuration file that can become quite elaborate.

The gather program initially calculated temperatures from thermistors connected to Modbus A/D controllers. The program has since then evolved into a behemoth that can query a number of Modbus, BACnet, oBIX, serial, TCP/IP and I2C devices; do calculations on the results, and create statistical summaries of sampled data for a specified period.

The following devices are supported:

  • Anything using the Modbus, BACnet, or oBix procotols over a serial or network interface.
  • The Digispark Attiny85 development board running I2C-Tiny-USB firmware. This is mostly used to add an I2C interface to Windows computers.
  • The eGauge System's eGauge electrical power monitor.
  • The TI ADS1015 and ADS1115 I2C A/D converters.
  • The Analog Devices LTC2497 I2C A/D converter used in the Alchemy Power Pi-16ADC Raspberry Pi hat.
  • The Bosch BME280 and BME680 environmental sensors.
  • Any of the Winsen MQ series gas sensors.
  • The Winsen MH-Z14A CO2 sensor.
  • The ubiquitous DHT11 and DHT22 relative humidity sensors. (Poorly and on the Raspberry Pi only.)
  • The u-blox NEO-6 and NEO-M8 GPS receivers.
  • The Edinburgh Sensor Gascard NG series gas sensors. (TCP/IP only.)
  • The InterMet Systems iMet-XQ2 UAV sensor.

Scatter

The optional scatter program reads the tab-delimited files produced by gather and does things to them including:

  • Concatenation
  • Compression
  • Copying to one or more servers
  • Writing to a database
  • Writing to an MQTT server
  • Archiving
  • Deletion

The environmental monitor writes statistical summaries to a PostgreSQL database.

Beacon

The optional beacon program pings the database server once a minute and obtains the controller's public IP address on the Internet and the current password of the panel program, which is described next.

Panel

The optional panel program is an old-fashioned cgi-bin program invoked by an HTTP server. The panel program accepts a single password and then renders sampled data in real-time. Data is presented on several pages as defined by the gather program's configuration file.

Other Controller Software

Lighttpd

Lighttpd is a lightweight HTTP server suitable for running on smaller controllers. It invokes panel to render the real-time display.

OpenVPN

Remote management and secure database access is provided by running an optional OpenVPN client on the controller. The client connects to an OpenVPN server on the Internet. The Hard'ack networking system is described later.

Hard'ack Server Software

Nexus

Nexus is a large cgi-bin program invoked by an HTTP server. Nexus is the Hard'ack system's web graphic user interface (GUI) to the database. Typical nexus operations are:

  • Monitoring controller status.
  • Generating charts of acquired data.
  • Exporting acquired data into one of a number of standard data formats.

Dispatch

The dispatch program is another cgi-bin program invoked by an HTTP server. Dispatch handles requests from beacon running on controllers and returns the controller's public IP address and panel password.

Sentinel

The Hard'ack system has a system of alerts to warn about out-of-range or error conditions. E-mails are sent in response to an alert. The sentinel program periodically scans the database for alert definitions and processes them.

Other Server Software

PostgreSQL

PostgreSQL is the database engine used by Hard'ack to store data.

Apache

The Apache HTTP server is used to invoke the nexus and dispatch cgi-bin programs.

Hard'ack Networking

A full Hard'ack system has four networking tasks:

  1. Putting the web GUI server on the Internet so that it's publicly available.
  2. Having a controller access the database server in a secure manner.
  3. Securely accessing controllers for maintenance.
  4. Putting a controller on the Internet so that its real-time panel is publicly available.

There are many different ways to accomplish these tasks. My current setup relies on a Virtual Private Network (VPN) for tasks #2 through #4. A single server performs both database and web GUI functions.

Making a server publicly available on the Internet (task #1) is relatively easy. It requires friendly Internet access with either an unchanging static IP address or an arrangement with a third party outfit like NoIP.com to manage a changeable dynamic IP address. If the server is directly connected to the front-line firewall then opening ports 80 (HTTP) and 443 (HTTPS) and forwarding them to the server is all that needs to be done. If there are layers of firewalls between the Internet and the server then ports 80 and 443 need to be opened and forwarded from layer to layer to reach the server.

Making a controller publicly available on the Internet for maintenance and real-time panel access (tasks #3 and #4) are similarly easy if the Internet access is friendly. Opening port 22 (SSH) and port 443 (HTTPS) on the front-line firewall and forwarding the ports to the controller accomplishes these tasks. But you may not have control over the Internet access at some sites and the people who do have control of the front-line firewall may not want to open ports through it.

A common solution is to create an encrypted Virtual Private Network (VPN) that bypasses the issue of site-side firewalls entirely. I use a Ubiquiti EdgeRouter as a VPN server. The VPN server is attached to my front-line firewall and port 1194 (OpenVPN) is opened and forwarded to it. The VPN server defines its own network that is independent of all other networks. A second network interface on the VPN server is the physical manifestation of the independent network and the database/web GUI server is connected to it.

This puts the database/web GUI server behind two layers of firewalls: My front-line firewall and the VPN server. To accomplish task #1 the front-line firewall forwards ports 80 (HTTP) and 443 (HTTPS) to the VPN server and the VPN server then forwards the two ports to the database/web GUI server.

When a controller starts up it creates a connection to the VPN server and establishes its presence on the independent network. The controller now has direct access to the database server accomplishing task #2.

Anyone logged into the database/web GUI server has direct access to all the controllers and can log into them using the SSH protocol. This accomplishes task #3. Another way to access the controllers for maintenance is to install an OpenVPN client on one's travel computer and connect to the VPN.

Task #4, making a controller's real-time panel publicly available on the Internet, is more intricate. To be secure the real-time panel must use the encrypted HTTPS protocol. The standard port for HTTPS is 443 but that port is being used by the web GUI server. A further complication is that there may be several controllers and a single public IP address. There are a few ways to do this. My solution is to use non-standard ports, a unique non-standard port for each controller.

The unique port for each controller is forwarded by the front-end firewall to the VPN server. The VPN server then does Source Network Address Translation (NAT) to route the port's traffic to the controller over the VPN. This makes the VPN server a middleman between a user's workstation and the controller. Requests are sent from a user's workstation to the VPN server and relayed to the controller. Replies are sent from the controller to the VPN server and relayed to the user's workstation.

Here's how it works in detail:

  1. A user at a workstation enters a controller's URL into a web browser. For this example I'm using "https://site1.mydomain.com:6789". The Domain Name System (DNS) converts the name "site1.mydomain.com" into the public Internet IP address used by the database/web GUI and VPN servers. The non-standard port is 6789. The web browser sends an HTTPS GET request to the IP address and port. The request is sent to whatever front-line firewall is at the user's site.
  2. The user's site's front-line firewall sends the request to the public Internet.
  3. My front-line firewall receives the request and forwards it to the VPN server.
  4. The VPN server receives the request and, using Source NAT, modifies it to make it appear that it originated from the VPN server. The request is addressed to the controller's public IP address, which is known to the VPN server, and sent back to my front-line firewall.
  5. My front-line firewall sends the request to the public Internet.
  6. The request is received by whatever front-line firewall is at the controller's site. Because of the VPN the request bypasses the firewall's rules and is sent to the controller.
  7. The controller receives the request and produces a reply. The reply is addressed to the VPN server and sent to the controller's site's front-line firewall.
  8. The controller's site's front-line firewall sends the reply to the public Internet.
  9. My front-line firewall receives the reply and forwards it to the VPN server.
  10. Using Source NAT the VPN server restores the original request source, the user's workstation, and sends the reply back to my front-line firewall.
  11. My front-line firewall sends the reply to the public Internet.
  12. The user's site's front-line firewall receives the reply and sends it to the user's workstation.
  13. The user's workstation receives the reply and the web browser renders a login form.