Home Automation – My Way

**Please note – this is not a ‘how to’ guide, just a bunch of notes and tales of the fun I have had getting stuff working 🙂

Home automation: other names include Smart Homes, Smart Devices etc, sounds like fun so I dived in about a year ago (2019).

There are a few different connectivity options for smart things – stuff like “ZigBee”, “Z-Wave”, “WiFi” to name the most popular ones. Generally speaking, WiFi devices will connect to your home WiFi network and be controlled via an app on a phone or pc etc, a remote server usually in China, and possibly a voice activated device like ‘Alexa’ or ‘Google Home’ which again will use a remote server, either Amazon or Google.

ZigBee and the less common Z-Wave devices usually connect to a hub (a small plug-in device) and form their own private ‘mesh’ networks, the hub will then be connected to your internet and thereon to things like Alexa and other apps for control. In the beginning I bought on price – the cheapest smart stuff was generally WiFi but was not found much on things like bulbs – these were mostly ZigBee. So we ended up with a mix of ZigBee and WiFi devices, several apps and of course an Alexa device or two.

It worked pretty well for the first year, did most of what I wanted although it was hard to configure with simple things like the porch lights needing work doing in the Philips Hue hub, Smart-Things app AND the Alexa routines just to get the functionality I wanted. It was a real pain and very fragmented/messy way of doing stuff. Most users would be happy with just the Alexa routines and/or maybe the Hue app ‘scenes’ etc. but not me – I wanted it tidy and very configurable, and I wanted to be in full control.

When starting out, we replaced a lot of bulbs with smart bulbs (ZigBee ones from Innr), socket controllers were WiFi ones (cheaper) – these were used to turn stuff like decorative/comfort lamps on/off where the bulbs could not be replaced with smart ones. Also at Christmas, all the lights etc were controlled by 4 smart sockets and various routines run by Alexa. Most of the bulbs I fitted were ‘ambience’ types – these are white but you can adjust the shade of whiteness from very warm to very cold – somewhere in between is a setting that mimics the old tungsten bulbs perfectly – we wanted these as many plain white LED bulbs are a nasty shade of white.  We had no interest in the rainbow colour (RGB) bulbs so did not add any.

Starting out

Switches were replaced with multi-function smart devices, these look like this when fitted…

The original switch is still fitted behind the smart switch, it is left turned on as smart bulbs only work when left on. Smart switches have quite a few benefits – they can be fitted anywhere (it’s Zigbee and battery powered so no wiring), they can have multiple uses – you can control many things with them, they are also removable(cordless) so you can use them where you sit for example. The ones I used were connected via ZigBee and made by Philips as part of their ‘Hue’ range.

That’s pretty much how it stayed for a year – we had 19 bulbs, 5 smart sockets, A ‘Hive’ smart heating control and the Alexa device to shout at 😉

A year later

It was all still working, mostly, we had some issues with a few bulbs behaving badly – they would switch on randomly, usually at night, not often but probably once a month. It seems this can be pretty normal for some of these devices and is well known.

Home Assistant and the Raspberry Pi

After a lot of research, looking at different control systems, one called ‘Home Assistant’ popped to the top of the research pile and it was this that I decided to look into using. Home Assistant (found here) is open-source software so is not tied to any major company like Google or Amazon etc. The software runs on a Raspberry Pi – these are tiny, bare computers that look like this…

It’s about the size of a credit card and 15mm deep – they are tiny yet boast some serious processing power and connectivity – Ethernet, WiFi, USB & HDMI plus others.  They are also surprisingly cheap (£50 at time of purchase). The software storage is a tiny Micro-SD card, add a power supply and you have a functional computer.

The Home Assistant software was fairly easy to set up, I followed this guide on YouTube but also watched others to get a better idea 🙂  Then the fun started – I needed to learn how to programme things so they did what I wanted. There are a few ways of doing this in Home Assistant, I chose to use an add-on called ‘Node-Red’ – this a visual, almost drag-and-drop way of setting up control actions. A simple one might look something like this…

The simple action shown above turns off some lights in the mornings as it gets lighter outside. To figure it out, it helps to view it as a series of questions/answers, from left to right…

Has the light level changed? – This checks the light level outside using a simple motion sensor under the porch.

Is it early morning? – This stops the lights turning off say if it gets dark in the afternoon.

Are the lights on? – Only turn off if they are on, it stops the control sending repeated ‘off’ commands.

Is the light level above my set limit? – Compare the level to a preset number.

Lights off! – The final command to actually turn the lights off.

These “Sequences” as they are called can get pretty complex but walking through in a logical manner enables you to easily see whats going to happen, most of the time 😉  I have also started to learn a bit of the JAVA and JSON programming languages as these are needed sometimes for more complex things, it’s all part of the fun.

Home Assistant had been running ok for a while, my learning was progressing and I now had several “Flows” configured – these are groups of sequences placed together to make things easier to find later – you could have flows for say Living Room, Kitchen, Outside etc and put sequences that are relative to each area in them for instance.

Problems

I then ran into a problem – the SD memory card died. This was surprising as it was only 4 weeks old but I had read that using them as a storage device in a computer really hammers them and they will die sooner or later. Luckily I had a similar SD card in the car dash-cam so that was put to work as a temporary fix – one big downside to home automation this way is that if the controller or your hub dies, you have no control at all – you cannot even put a light on easily if it’s a smart bulb – bear this in mind when planning smart home designs.

So having got it all back up and running again on the temporary SD card – this took about two hours as I made sure I kept backups, I decided things needed to be made a little more sturdy. Some reading showed that the best option is to fit a small SSD drive, these are very cheap now and commonly used in laptops etc.

I got a Western Digital 120Gb drive for only £18 on Amazon which is silly-cheap really. Naturally the SSD is bigger than an SD card so a way of connecting it is needed, luckily there are many ways, I chose an SSD piggyback mount which fits onto the Raspberry Pi directly….

That solved the mounting, but we then needed a bigger case….

The new case also meant the Pi would get a bit warmer so we add a cooling fan which piggybacks on the top of the Pi…

When it’s all screwed together, it makes an impressive little stack of things, fits nicely in the steel case and is a pretty powerful little computer…

All good fun so far…..

Installing the Home Assistant OS on an SSD

So I now had a jumbo Raspberry Pi, I still had to re-install the software and restore the programming flows and data to the new SSD.

It took a few YouTube videos to do this, it involves some pretty Geeky commands being sent to the Pi in order to get it to start up without an SD card and to use the SSD instead. It all went well though and a couple of hours later the Pi was running ok. However, none of my controls worked anymore, it appeared it was all happy, no errors in the logs, but nothing did anything – light switches did nothing and the software could not turn on a light etc.

This was panic time – carry on or put it back to SD card?? I sought a lot of help on this one, spent about 5 hours sweating over a hot laptop and generally burning up the internet.

Now, in order to get the Pi to talk to lightbulbs and other smart devices, you need to fit something called a “ZigBee dongle” (or Z-Wave if using that protocol) – a small device that plugs into a USB port on the Pi and contains a small radio transmitter / receiver that sends commands to the devices and receives messages back from same devices about their status etc. The one I chose was the “ConBee II” which seemed to be well recommended.

A few more hours were spent, much coffee was consumed and a repeat topic started coming up on searches – USB3 and WiFi – It seems there is a known issue with the new USB3 high-speed connections and things like cordless mice, keyboards and of course ZigBee dongles! Some nice person sent me a link to a published white-paper from Intel that explained it all.

It all boiled down to the fact that USB3 seems to run at the same frequency as the smart devices do; 2.4 GHz which is also the same as general WiFi. When in use the USB3 device or its connectors can emit interference strong enough to cripple some devices – the ZigBee dongle is one of these and it is usually plugged in to the same computer as the offending USB3 devices!

Why had the issue sprung up now when the USB3 ports were there before? Simply because they were not being used but the SDD drive uses USB3 to connect to the Pi main board so the USB3 bus was now extremely busy and emitting a whole mess of interference.

Clearly a fix was needed, the answer popped up with some more searching – take the ZigBee dongle out of the USB port on the Pi and fit it to a USB extension cable, then fit the cable into a slower USB2 port – the Pi has two USB3 and two USB2 ports.

This fixed the problem instantly and with 15 minutes, all the bulbs and devices had reconnected and we were in business again. It’s been running for a couple of days now and seems stable, the SSD is also much faster than the SD card as well. Now that it’s fitted, the SSD mounting board has a little blue LED on it that flashes when the drive is used – this LED highlights why the SD cards suffer, it really gets a lot of use!

Expanding the control reach

We have a panel heater in an outbuilding, it’s a garden games room so we keep the frost off and turn it up when we want to use the room. I fitted a so-called ‘smart heater’ which has remote control of its settings via Alexa etc and it used a WiFi signal for its control – this was paired with an app on the mobile phone and via servers in China most likely, allowed control remotely.

It’s a reasonable heater but the signal was very unreliable and disconnected every couple of days, this meant having to go out to the room, put the heater back into ‘pairing mode’ then going through the re-connect process every time – not very smart at all.

Something better was needed so I added one of these…..

It’s a Temperature/Humidity sensor and used ZigBee as its connection method, if you remember from earlier, almost all of our smart stuff is ZigBee now. It’s a great little device and the signal has no problem reaching the house network. That means we could now read the temperature in the outbuilding, adding a ZigBee smart-socket means we can control the panel heater on & off without using the useless WiFi app & signal.

Some simple coding in Home Assistant gives us a basic thermostat action and so far it seems to be extremely workable, controlling the temperature to within 1 degree Centigrade.

As the panel heater is a known power usage of 2kW – it was then a simple matter of adding some maths and a display in Home Assistant to show us the real cost of running the heater – smart. It’s just a calculation based on the time the heater spends turned on each cycle plus our known cost per kWh of electricity. This only works for devices where the power usage is fixed though.

Back indoors

Things were running well since the upgrades, it was pretty stable and doing what it should.

Time to move the Raspberry Pi off the coffee table where it had been sitting 🙂 I shut it down, moved it a little to somewhere not so obvious and re-started it.

It all seemed to start well, looking at the dashboard showed things to look pretty normal, apart from the fact that I had no control over the lights etc (again)! There were no errors in the system logs. After an hour or two of head-scratching, I realised I had seen this effect before – the ZigBee dongle signal was being interfered with.

ZigBee runs on the same frequency as 2.4Ghz WiFi does – they have to share it but WiFi is the stronger of the two signals, when I moved the setup earlier, it was now closer to my WiFi router, only by about a foot but closer than it was. I then tried putting the dongle on the floor (its on a USB extender cable) and it all started working again – that proved the interference theory.

I moved the whole lot upstairs, it needs an ethernet connection to the network, not WiFi so I’m limited to where I can put it. The result was – no signal again, some bulbs worked but not all.  Back downstairs with it, this time on the opposite side of the TV stack, it was about 5 feet away from the WiFi box and seemed to be a lot happier now, all devices were back working.

Conclusion – being very low power, ZigBee is extremely sensitive to interference and may need trying in several places to find the best option. This may be the reason why the smart-home support forums have a lot of posts like “some of my bulbs have stopped working…..” and “I can’t turn the heating on anymore….” – it could possibly be that the place the ZigBee hub is placed in is only allowing a weak signal out.

All good fun, not 😉

Controlling the Sky TV box

I had some success today with Node Red – this is the visual control building software for Home Assistant.

Since starting I have been struggling to discover a way to tell if a Sky-Q box is turned on or off – “why do i care?” you might ask. Well, I use the fact that the Sky-Q box is turned on as a pretty good indicator that we are in the house and probably watching TV – this info is then used to control various lights that we want to turn on automatically when it gets dark outside and other things.

Anyway, the Sky-Q box sadly uses one command to turn it on and off, it’s a toggle; makes no difference when you use the remote – if it fails to turn on you simply press the button again, but in the world of automation the system sends a command and if the device ignores it then all bets are off as we now have no idea if its on or off.

On a device with an On command and an Off command it’s not so bad as we can just send another ‘on’ command etc

After many days of searching I stumbled upon a page that had some interesting info on Sky boxes – it seems they have a web server in them that can dish out information about the box if asked – nice.

If you have a Sky box and browse to this address:- http://xxx.xxx.xxx.xxx:9006/as/system/information  (replace the xxx.xxx.xxx.xxx with the IP address of your box), you will get a response back that is just a page of plain text. I won’t put a picture up of mine as it might contain sensitive info, but in that text there is this line:-

   "activeStandby" : true,

This is the secret I needed, ‘true’ means it’s in standby mode and ‘false’ means it’s turned on.

In Node Red I used an “HTTP node” to send the request and a “Switch node” to check the reply for either “activeStandby”: true or “activeStandby”:false. Once that was known it was easy to choose what I wanted to do with the info in my lighting control flows. This little problem has been bugging me for about 4 weeks and the fix in the end was so simple yet nobody else seemed to know about it 🙂

Controlling windows, real ones not the software 😉

Things are working pretty well, I have added more motion sensors to control lights where possible, I have also added remote control blind ‘pullers’ to the blinds in the living and dining rooms – having these close and open based on daylight and time is a massive smart-home boost. The blind motors are a neat little box that mounts on the wall and simply pulls the blind chain. They contain a large rechargeable battery with an expected 6 month life and the Zigbee control circuitry.

RGB failure

I did have a weird failure today though, one of the RGB controllers for the living room LED light strips was permanently showing as on even though the actual lights were off. Sounds harmless but many automation routines are based upon the current state of a light or switch etc, if that state is not what it seems then things really get screwy 🙂

I checked all the routines as i had been tinkering a bit recently, nothing wrong there, nothing showing in the history logs which was a bit odd. Turn the light off in Home Assistant and it immediately pops back on, even though the real light was off all the time.

In the end it needed the power to the RGB controller cycling off/on which reset it and all returned to normal. My guess is that the controller had crashed and was repeatedly sending out “I am on” messages on the Zigbee network when it was really off. Home Assistant was doing its job correctly and telling me it was on but I was looking for the fault in the wrong place. I replaced the cheap RGB controllers in Late 2021 and the issue never returned.

Interaction with other hobbies

In-between messing with the home automation, I dabble with ham radio as a licensed operator, my callsign is 2E0FWL, it’s a fun pastime.

I had not been on-air for a good while as I was busy with the decorating and stuff, I turned the radio on last week for a play and to my horror was welcomed with 100% interference and very little else! The display on the radio looked like this…

The display shows radio signals as a visible display, the yellow vertical bars are man-made interference. A normal view would be mostly black with some feint blue bars showing other stations transmitting. The blue ‘fog’ between the yellow is a mass of broad-frequency noise.

This needed fixing clearly. The first thing to do is check if it’s outside as well – solar panels, phone masts, and other electrical gubbins can all create this noise, using a portable receiver, it was easy to test by tuning in to the noise then going outside – the noise stopped about 10 feet from the house – a clear sign it was coming from my house.

Next step is to turn things off – I killed the power to the light circuit and the noise went away, this showed it was coming from a light somewhere. Next we turn the power back on and start turning things off one by one – the thing with ‘smart’ bulbs is that they are always on, they just do not light up when off 🙂

It turned out it was the power supplies used for the RGB LED strips in the living and dining rooms, turning these off fixed the problem. The supply modules i bought were slimline units as i wanted to hide them easily in the dropped islands. They look like this…

Very slim and tidy but oh so noisy. I bought some bigger units which look like this…

Looking at various circuit diagrams and inside the two styles of supply shows that the slim ones have an awful lot of components missing – mostly filtering and smoothing – hence the terrible noise emitted. The new ones have a lot more ‘guts’ in them and are completely silent in terms of radiated interference. My conclusion is that the first ones are total junk and in terms of certification alone should not be sold at all, chances are they have never even been tested/certified and were just more of the “Chinese junk” that we get so much of now.

The price for both types was almost identical as well !

Voice assistants

We’ve all heard of them, they come in different flavours – Alexa, Google, Siri etc. They are pretty useful things most of the time, like all tech they can have issues but are generally fairly reliable and successful at voice recognition.

They all have one thing at least in common – they rely totally on on a massive server running in some remote building to process the voice commands we make. This is pretty safe but for some, one part of home automation is to do everything inside the home, this means no external servers in China etc just to turn a light on. Until fairly recently a workable offline voice assistant was pretty much impossible, it just needed too much processing etc to turn a voice command into text or action like turning a light off.

I recently started playing with a project called “Rhasspy” – its an open-source voice assistant project that can be run on a Raspberry Pi and will connect to smart home controllers like Home Assistant. Getting it up and running was a nightmare as it is fairly advance Linux/Pi stuff but I managed it in the end. The Pi has a small add-on called a “microphone hat” – this gives the Pi a sound input and also an output that will connect to a small loudspeaker – the Pi now has a voice, well there are a lot of voices to choose from.

It’s still being set up – it takes some tuning, but early results are very promising and it will respond almost as well as Alexa to home automation commands, not bad for a tiny computer like a Pi. It will probably never be able to search the internet like Alexa can as that will just take too much work and it also breaks the “offline” rule – not much point making an offline voice assistant go online.

I have slowly been working on the flows and commands in Node Red to enable it to connect to Home Assistant – this is the most fascinating part and really gives an insight into just what Alexa is really having to do in the background when you want a light turned on!  A small part of my flow is shown below, it’s all experimental at present but getting to point where I can actually use it now.

In use, there is a two-way communication between the device and Node-Red, it starts with a wake-word, then an intent recognition, followed by an acknowledgement and finally a termination.  It’s fast, and pretty accurate but I doubt it will give the professionals a run for their money, after all they invested millions in development and hardware whereas this project is running on a cheap Raspberry Pi and doing it all itself 🙂

Nearly a year later and the Rhasspy Assistant is still running – we use it for all control of the smart home stuff now and only use Alexa for internet queries like football scores etc. – something Rhasspy will likely never be able to easily perform.

2022 Update

It’s all still running well, we have added some more smart sockets, a couple more motion sensors to control kitchen lights and replaced the outdoor floodlight with a set of 4 LED ones – a total of 24 watts replace one 50 watt halogen bulb and the garden now looks like a football stadium.

Rhasspy is still doing the voice control, Alexa does a lot less now.