The Sonoff TH10/TH16 are 90-250V (50/60hz) temperature and humidity monitoring WiFi smart switches supporting a maximum current of 10A (2200W) or 16A (3500W) with two purpose built sensors, one for temperature and one for humidity. Compared to most retail Wifi enabled switches they are extremely cost effective with an online price under USD 10. The switches are designed to be controlled by a free IOS or Android app but what makes them perfect for developing your own HA system is that they use an ESP8266 chip which means you can flash them with custom firmware and do just about whatever you want with them!

Recently (2018) I bought a couple of new TH16’s and found that a lot had changed since I first used them so here are are the steps currently required to FLASH, program and use a Sonoff TH16 as a Home Automation smart switch and temperature sensor controlled by PHP or FHEM.

Sonoff TH16

You can purchase the Sonoff TH16 at all your favourite online retailers including Amazon and eBay, whilst it is possible to build your own sensors if you want to monitor temperature or humidity then it’s easiest to also purchase the corresponding Sonoff sensor, for temperature I used the Sonoff Temp Sensor-DS18B20 which costs about USD 5. The temperature sensor is waterproof.

FLASH Firmware with ESPEasy

ESPEasy is open source firmware for ESP8266. “The ESP Easy firmware can be used to turn the ESP module into an easy multifunction sensor device for Home Automation solutions” in our case the ESP module is within the TH16, the ESPEasy firmware replaces the code supplied with the TH16 with the ESPEasy firmware version you choose to flash.

To flash the TH16 with new firmware you need to enable serial communication between a Windows PC and the TH16, you can do this by using a USB to TTL dongle CH340G (FT232RL or PL2303 based will also do) these are easily obtainable online. The dongle connects to a USB port on your PC creating a serial interface. You then connect the dongle to the TH16.

Before you can connect the dongle you must open up the TH16 and solder a header to the power, TX and RX pins on the TH16 circuit board. Header pins are easily available online, you will also need a soldering iron, solder and a steady hand.

Note, you may need to swap TX and RX round. The Sonoff circuit board version used here was v2.1

The currently advertised stable firmware is version 120 which is at least 2 years old, I recommend you flash the TH16 with the latest MEGA dev firmware which has nightly builds available from the github at https://github.com/letscontrolit/ESPEasy/releases

Download the ESPEasy_mega-XX.zip file and extract it. You will see various firmware binary images, we need a 1MB ESP8266 binary image so the DEV image ESP_Easy_mega-2018XXXX_dev_ESP8266_1024.bin is the image we will use to flash the TH16.

We need software to communicate with the TH16 and flash the new firmware to the 8266 chip, the esptool software supplied with the ESPEasy firmware no longer works with the latest versions of the Sonoff TH smart switches (v2.1). Download the nodemcu flasher here

For 32 bit Windows : https://github.com/nodemcu/nodemcu-flasher/blob/master/Win32/Release/ESP8266Flasher.exe
For 64 bit Windows : https://github.com/nodemcu/nodemcu-flasher/blob/master/Win64/Release/ESP8266Flasher.exe

When you have all the software saved and the TH16 connections ready start the ESP8266Flasher. Now holding down the tall white button on the TH16 plug the TH16/USB dongle into your PC and release the tall white button this will configure the TH16 8266 chip for flash mode.

The nodemcu firmware programmer (ESP8266Flasher) should now detect the serial COM port connected to the TH16. If the port is not detected check in Windows device manager that the USB dongle is detected as a serial device and has been assigned a COM port.

Click on the Advanced tab and configure the following settings

Click on the Config tab and select the 1MB ESP8266 binary image from the downloaded ESPEasy image folder

Now go back to the Operation tab and click the blue FLASH button to begin the flash process, a progress bar will be displayed and the process will take a couple of minutes to complete.

When the flash operation is complete you can disconnect the TH16 from your PC.

ESPEasy WLAN Configuration

You now need to power on the TH16 from the mains, so you first need to connect a mains cable (90-250v) to the device and power it up

When the TH16 is powered on after flashing it becomes a Wireless Access Point to allow you to connect to it and configure initial WLAN settings for the device. With a smart phone or other device connect to the WLAN name ESP_Easy_0 with the password configesp.

You should see a browser page asking for WLAN credentials. Select the WLAN you want the TH16 to connect to and enter the password. The TH16 should connect to the WLAN and inform you which IP address it has been assigned. I recommend you create a DHCP reservation for this address so that it is permanent.

You can now manage and configure the TH16 from a web browser using the url http://x.x.x.x where x.x.x.x is the IP address assigned to the TH16.

If you do not see the ESP_Easy_0 WLAN then the flashing process was not successful. Attempt to flash the device again, some users report that you should press and hold the tall white TH16 button and continue to hold it until the flashing process is complete.

ESPEasy Configuration

The TH16 is now powered on, connected to the WLAN with the DS18B20 temperature sensor plugged in. Note that the sensor plugs in using a small jack plug, the plug should push in and click into the socket. With my TH16 the hole in the casing prevented the sensor plug from clicking firmly into place and was not detected. I had to enlarge the casing hole in order for the sensor to connect properly and be detected.

Open a browser and enter the url of the TH16 to open the web based configuration : http://x.x.x.x

First give the device a name by clicking the Config tab and entering a Unit Name. If you are using FHEM (or another supported HA system) click on the Controllers tab and edit the credentials for your HA Controller server. For FHEM select FHEM HTTP from the Protocol list and enter the IP address of your FHEM server and the port used by the FHEM ESPEasy bridge (usually 8383 is used).

Click on the Devices tab

I have three devices configured

1. Generic System Info is reporting the WLAN signal strength – this is used as a presence controller by FHEM.
2. Switch Input is reporting the state of the built in power switch which is controller by GPIO 12
3. Environment DS18b20 is reporting the state (temperature) of the DS18b20 sensor which is controlled by GPIO14
   1. if the sensor is not detected by the TH16 make sure if is connected properly

To configure these devices click the edit button and select the device type you want to configure from the drop down box, here is my configuration for the temperature sensor

Make sure you check the enabled box before you save the device. You can see that I am also sending this data to the controller (FHEM). The interval determines how often the device value will be updated. Explore the other configuration options available, you may want to add an access password or an NTP time server for example.

With the three devices configured and working you will see the corresponding values displayed, switch status (1=ON, 0=OFF), temperature etc.

Retrieving Data and Controlling the Switch

The easiest way to retrieve data from the TH16 is via an http request that returns JSON data, simply browse to

http://x.x.x.x/json

Here you will see a nicely formatted json array of status data for the TH16 including the switch status and temperature sensor value

"TaskValues": [
{"ValueNumber":1,
"Name":"Temperature",
"NrDecimals":2,
"Value":29.50
}]

This makes retrieving data from the TH16 really simple if your are using PHP for example you can do something like this to get the data using curl and return it as a php array.

$_data=$this->curlAjaxRequest('http://x.x.x.x/json');

	public function curlAjaxRequest($_url)
	{

		try {
			$_curl = curl_init(); 
	
			$_header[0] = "Accept: text/xml,application/xml,application/xhtml+xml,"; 
			$_header[0] .= "text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5"; 
			$_header[] = "Cache-Control: max-age=0"; 
			$_header[] = "Connection: keep-alive"; 
			$_header[] = "Keep-Alive: 300"; 
			$_header[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7"; 
			$_header[] = "Accept-Language: en;q=0.5";
			$_header[] = "Pragma: ";
			
			curl_setopt($_curl, CURLOPT_URL, $_url); 
			curl_setopt($_curl, CURLOPT_USERAGENT, 'My Home Automation'); 
			curl_setopt($_curl, CURLOPT_HTTPHEADER, $_header); 
			curl_setopt($_curl, CURLOPT_ENCODING, 'gzip,deflate'); 
			curl_setopt($_curl, CURLOPT_AUTOREFERER, true); 
			curl_setopt($_curl, CURLOPT_RETURNTRANSFER, 1); 
			curl_setopt($_curl, CURLOPT_TIMEOUT, 10); 
			
			$_object = curl_exec($_curl); // execute the curl command 
			curl_close($_curl); // close the connection 

		}
		
		//catch exception
		catch(Exception $e) {
			
			// do nothing with $e
			return false;
		  
		}
		
			return json_decode($_object,true);

	}

We can also control the TH16 using JSON commands. For example to turn the switch on or off using GPIO12 we can use

ON : http://x.x.x.x/control?cmd=GPIO,12,1
OFF : http://x.x.x.x/control?cmd=GPIO,12,0

Again using curl you can send these commands via PHP.

FHEM Configuration

If you are using FHEM there is already a FHEM module that supports the ESPEasy firmware. To configure FHEM you need to enable the ESPEasy bridge, this is done with the following commands

define espBridge ESPEasy bridge 8383
attr espBridge authentication 0
attr espBridge combineDevices 0
attr espBridge group ESPEasy Bridge
attr espBridge room ESPEasy

You can see here we enable a bridge called espBridge listening on TCP port 8383.

If you have configured FHEM with autocreate enabled you should now see the TH16 switch automatically configured when it next sends it’s data update. It will appear in FHEM under ESPEasy.

I configured FHEM with the following configuration for each TH16

# ESP3 - OFFICE
define ESPEasy_ESP3_OFFICE_Switch_input ESPEasy x.x.x.x 80 espBridge ESP3_Switch_input
attr ESPEasy_ESP3_OFFICE_Switch_input IODev espBridge
attr ESPEasy_ESP3_OFFICE_Switch_input Interval 300
attr ESPEasy_ESP3_OFFICE_Switch_input devStateIcon on:ios-on-green:on off:ios-off:ff absent:10px-kreis-rot:statusRequest .*:ios-NACK:check
attr ESPEasy_ESP3_OFFICE_Switch_input event-on-change-reading .*
attr ESPEasy_ESP3_OFFICE_Switch_input eventMap /gpio 12 on:on/gpio 12 off:off/status gpio 12:check/
attr ESPEasy_ESP3_OFFICE_Switch_input group ESPEasy Device
attr ESPEasy_ESP3_OFFICE_Switch_input icon ge_wht_steckdose
attr ESPEasy_ESP3_OFFICE_Switch_input presenceCheck 1
attr ESPEasy_ESP3_OFFICE_Switch_input readingSwitchText 1
attr ESPEasy_ESP3_OFFICE_Switch_input room ESPEasy
attr ESPEasy_ESP3_OFFICE_Switch_input setState 3
attr ESPEasy_ESP3_OFFICE_Switch_input stateFormat {ReadingsVal($name,"presence","") eq "absent" ? "absent" : ReadingsVal($name,"Switch","")}

define ESPEasy_ESP3_OFFICE_System_Info ESPEasy x.x.x.x 80 espBridge ESP3_System_Info
attr ESPEasy_ESP3_OFFICE_System_Info IODev espBridge
attr ESPEasy_ESP3_OFFICE_System_Info Interval 300
attr ESPEasy_ESP3_OFFICE_System_Info group ESPEasy Device
attr ESPEasy_ESP3_OFFICE_System_Info icon WLAN_Status.1
attr ESPEasy_ESP3_OFFICE_System_Info presenceCheck 1
attr ESPEasy_ESP3_OFFICE_System_Info readingSwitchText 1
attr ESPEasy_ESP3_OFFICE_System_Info room ESPEasy
attr ESPEasy_ESP3_OFFICE_System_Info setState 3

define ESPEasy_ESP3_OFFICE_Temperature___DS18b20 ESPEasy x.x.x.x 80 espBridge ESP3_Temperature___DS18b20
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 IODev espBridge
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 Interval 300
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 group ESPEasy Device
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 icon temp_temperature
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 presenceCheck 1
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 readingSwitchText 1
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 room ESPEasy
attr ESPEasy_ESP3_OFFICE_Temperature___DS18b20 setState 3

I can now also use FHEM to control the switch by clicking on the GUI or using the commands

set ESPEasy_ESP3_OFFICE_Switch_input on
set ESPEasy_ESP3_OFFICE_Switch_input off

Conclusions

Even the cheapest WIFI enabled smart switches are at least three or four times expensive as the Sonoff and that does not include any kind of sensor. Using the TH16 with ESPEasy software is a really cheap and reliable way to deploy WiFi smart switches and sensors in your Home and if you are an electronic engineer then there are many more options available for building and connecting your own sensors to the TH16.

Resources

• • https://www.itead.cc/sonoff-th.html
  • https://www.letscontrolit.com/wiki/index.php/ESPEasy
  • https://www.amazon.com/s/ref=nb_sb_noss_2/134-0162784-3379579?url=search-alias%3Daps&field-keywords=sonoff+th16
  • https://github.com/letscontrolit/ESPEasy/releases

Install Docker

Installing Docker is a no brainer

curl -sSL get.docker.com | sh
sudo usermod your-username -aG docker
reboot

Install docker-compose

Docker-compose makes building, starting and stopping Docker containers really simple.

apt-get -y install python-pip
pip install docker-compose

Build the container

Pull the container build files from github and run docker-compose build to build the container image.

git clone https://github.com/gaiterjones/docker-rpi-fhem
cd docker-rpi-fhem
docker-compose build

Start the container

docker-compose up -d

Connect to FHEM

Enter the following url into your browser to connect to FHEM.

http://my.pi:8803/fhem

Update FHEM by entering with the update command.
Restart FHEM by entering the shutdown restart command.

FHEM is now running and up to date.

You can start configuring it or copy your existing FHEM config or editing the fhem.cfg file in ./fhem. This is a Docker volume on the host mapped to /opt/fhem in the container. Docker host volumes will persist when you bring down the container (docker-compose down) or clear the volume caches (docker-compose down -v) so you will not lose your configuration or logs when the container restarts or is rebuilt.

Customising

Take a look at docker-compose.yml

# RPI
# FHEM CONTAINER
#
version: "3"

services:
    server:
        build: ./build/
        hostname: fhem
        domainname: home.com
        privileged: true
        ports:
          - "8083:8083"
          - "7072:7072"
        expose:
            - 8083
            - 7072
        volumes:
            - ./fhem:/opt/fhem
            #- /dev/ttyUSB0:/dev/ttyUSB0
        networks:
            - server
        restart: always
        healthcheck:
           test: ["CMD-SHELL", "echo 'QUIT' | nc -w 5 localhost 7072 > /dev/null 2>&1 && echo 'FHEM OK'"]
           interval: 15m
           timeout: 10s
           retries: 3
networks:
    server:

You can see we are configuring global ports 8083 and 7072 on the host as well as exposing them on the container. I have an SSL certificate installed on my Pi and use Nginx as a frontend proxy to my Docker containers, this lets me connect to FHEM using SSL

https://my.pi/fhem

To remove the global host ports delete or comment out the ports config leaving just the expose config. Nginx can be configured to proxy to fhem using the following configuration:

# FHEM
location /fhem {
  proxy_pass http://fhem_container_1:8083/fhem;
}

The container runs in privileged mode which gives it access to the hosts hardware and devices. If you are using external USB devices connected to the Pi you must add them as a volume in the container i.e.

– /dev/ttyUSB0:/dev/ttyUSB0

Make sure the fhem user running in the container has the same user id as your host user, in this case uid 1001.

	 	 
user@raspberrypi:/dev# id your-user-name	 	 
uid=1001(your-user-name) gid=1001(your-user-name) groups=1001(your-user-name),27(sudo),996(docker)	 	 

This will ensure there are no permission problems when the container tries to access devices on the host.

For example you want to extract the main product info, price, title, SKU etc and all the product attributes names and the product category. This might require multiple product / collection calls and when you multiply this by a search across 1000’s of products can become slow.

Magento Product Attribute Search and Filter

I wanted to provide a quick and easy way to search and filter magento products by attributes, in this case by colour, so customers could quickly narrow product selection down by selecting a colour and a filtering by category.

To extract the data I developed my Magento Product Data and Attribute exporter. This let me initially see how many attributes I had to work with, in this case how many different product colours were already configured.

To make colour selection simpler I created a new Magento attribute product_primary_colour and added a primary colour to each product so that I ended up with about 16 main colours that would provide a match to all the existing product attribute colours.

Once I had all my exported product data in a tab delimited text file I quickly realised that it would be relatively simple to search and filter this data via ajax requests.

I had initially thought it would be more effective to do the search within Magento using product collections but the speed of the search against the delimited txt data file made me think again. When I then added cached search results using Memcache I was happy that this solution would work.

The benefits of using the flat txt file are that it is quick and can run completely external from Magento in another webapp or in WordPress or Facebook.

The Magento attribute search and filter is easily customiseable and can search the data created by the export script filtering results from any of the product data columns in the export file. This could be colour, size, type etc, there are no limits.

If your product data is changing regularly simply export the data regularly via a cron job to keep it up to date.

if you need to filter your data by a completely new attribute simply pull the export data into Excel, create your new attribute data and corresponding Magento attribute and import the data back into Magento. Now you can export this data and easily search and filter it.

Here is an example of the attribute search running in an iframe using the Magento test product data. Here I have exported the color attribute, and am filtering against existing product categories like Womens, Mens and Shirts (note Mens matches Womens, some refinement needed there!). This will also easily run in Magento when setup in a static block containing the iframe data.

You can see how quickly the data is filtered and returned, try a search for Red to see some data. Click around the categories and check the debug footer which shows if the results were cached or not and how long the ajax query took to run.

Data from Magento DEV Store running Magento 1.9.x

Memcache is a bit like Jesus, all things to all men. Blasphemy aside, I love using Memcache for PHP application solutions. It is versatile, simple to use and very fast.

This year I have used memcache for a whole bunch of great solutions. I wrote a memcache database wrapper class to automatically use memcache to cache database queries – this increased the page loading time of a website considerably.

I used memcache for simple counters, i.e. page visits and media plays – much faster than using a database.

I used it  to cache media playlists in memory for quick lookup and matching of current playing tracks on a live darwin media stream and I used it to cache Magento product data used on eBay listings.

Today I have been using it to cache logging data (eBay product views) that can be quickly retrieved and displayed live using Ajax. My PHP logging class can be called from any PHP app to log any sort of data using memcache for super fast cached storage and retrieval.

The class logs data in arrays and saves them to memcache, the data can be set to expire whenever you like, the client then sets a timerange i.e. 30 minutes and queries memcache for all log data stored in that time range and displays it in real time using Ajax requests.

Because all the logging is done in memory it is very fast to log and to extract the data – much faster than using a database or writing to a logfile. At the moment I am not interested in storing the data so when the cache TTL expires the data is gone, but If required the data could be extracted asynchronously via a cronjob to a more permanent storage medium.

This method of caching log data and viewing it live could be used for any data you like not just web related stats, e.g. live orders, live product stocks, live tweets, etc. etc.

In the example below you see live logging data for the last 30 mins from this WordPress blog – the logging class is being called by WordPress every time a page loads, the iframe is loading a simple webpage that uses jQuery and Ajax to retrieve the log data every 10 seconds. Look at all those pesky bots!

I  developed Magento External for use with eBay listings as a solution to listing many products with multi language descriptions controlling all the content from a central database – Magento.

Magento External lets you produce simple product description pages using data from your Magento shop database. If you are selling the same products across multiple retail platforms such as Amazon and eBay it makes sense to keep your product data centralised, and if you already have a Magento shop then using the information you already have in Magento for external sites makes a lot of sense.

For eBay listings you can then also update your product info from your Magento admin, making it far easier to manage product listings.

The code utilises memcache to cache requests reducing load on your db.

• multi language
• custom javascript menus
• cross browser support (cross domain xdr supported)
• custom images and header images
• memcache support
• supports simple, grouped, configurable products

Here is an example eBay listing running here in an iframe. To see a live Magento external eBay listing send me an email.

Facebook is a great social media marketing tool, first goal – get those likes. Second goal, improve your Edgerank to reach more customers with your meaningful Social Media content.

You thought that Googles Pagerank was complicated, read on…

Courtesy of Postrocket.

I am a great fan of the Apple Darwin Media Streaming server and use it for streaming live mp3 audio content. (by live I mean content that is heard concurrently by all listeners).

I use the jQuery Player media player which is a great cross browser compatible media player and have adapted it to work with Darwin audio streams. The latest version is specifically designed to provide audio content via a Facebook page tab. The application uses PHP and jQuery to provide a Facebook Tab cross browser compatible media player that can play live darwin media streams. It includes the ability to require page likes to listen to content,  display playlist information from an XSPF playlist, including track name, links to further content i.e. buy the track and a simple track like counter.

You can see it running as a Desktop Facebook Tab application here and a standalone demo is available below. Mobile version is also available.