Home automation trigger house events based on my location

I have always been on the lookout for tracking devices which enable me to automate events in my house based on my location.
cargps

The obvious route I first took was to scan my network to check if my iphone is in range. Seemed a trivial solution that should work easily. I was in for a surprise. I found it works sometimes and is therefore not reliable and unusable. The problem lies in the fact that the iphone goes to sleep to save power. Also my solution was based upon polling my network to detect if my iphone had registered through DHCP. The delay of my iphone sleeping and the polling model means my automation script would trigger way after I had entered my house, not wat I was looking for.

What alternatives can I find:

  • iphone wifi detect when in range of home (as described not very succesfull)
  • bluetooth iBeacon
  • iPhone gps tracking using owntracks

Both the iBeacon and the gps tracking using owntracks where somewhat succesfull but both do have some accuracy issues. Maybe this is a subject for another time.

Recently I came across an alternative: a mobile tracking device which is very easy to install. It is battery powered and can last 3 or 12 years depending on the model.

Track your car using openhab and Komma EYE!

The technology is based on a low power module that you hide in your car. There are no connections needed because the module is battery based. The module can be purchased at http://www.komma-trace.nl/. Send them en email, there is no shop on their site.

And the product has a supporting website at: http://trackeye.eu/Animation/en/index.html

But using a website to view the module location is not the preferred way to integrate with my home automation system.

I use an openhab setup. http://www.openhab.org/ I like to connect devices to openhab using a networking protocol. My preferred way to integrate at the moment is to use a messagebus like mqtt. So I decided to develop a small program to request the module location and post the information on mqtt. Openhab can then process the information and store the vehicle location. Scripts can then be run using the openhab scripting model. Openhab is pretty good at scripting but is often a lot of trouble to integrate special hardware. Using mqtt as a generic interface I get best of both worlds.

But how do I get the gps location? I logged into the supporting website and found is was developed as a single page application (SPA) in ASP.Net. SPA’s are difficult to screenscrape because they are not based on a simple HTTP POST model. I decided to contact the developer and was pleased to hear that they are working on an mqtt interface but this is not production ready yet. The alternative they currently have is a REST service. Excellent!

After some emails with Komma Trace I was provided with credentials to connect to the REST interface and receive the GPS location information.

Here is an example of the type of call that needs to be done:
http://flexws.appserver.dk/flexws/latestsutmessage/form/

The resulting data is JSON and can easily be processed using c# and .Net 4.0, Add a reference to System.Web.Extensions.
This code can run on mono and there ia a simple mqtt client on unix to send the data to openhab. But first the c# code:


using System;
using System.Net;
using System.IO;
using System.Collections.Specialized;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Globalization;

namespace GPSTrack
{

   class Program
   {
      static void Main(string[] args)
      {
		 // Please provide your device specific guid here. This can be requested from Komma Trace. 
         KommaWeb loWeb = new KommaWeb("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxx");

         if (loWeb.DataReady)
         {
            Console.WriteLine(loWeb.ToJSON());
         }
         else
         {
            Console.WriteLine(string.Format("Error: {0}", loWeb.LastAction));
         }
      }
   }

   public class KommaWeb
   {
      public bool DataReady;
      public string LocationText;
      public string Temperature;
      public double Latitude;
      public double Longitude;
      public DateTime LastPosition;
      public DateTime LastUpdate;
      public string LastAction;

      public KommaWeb(string psGUID)
      {
         LastAction = "Init";
         DataReady = false;

         LastAction = "Connect to site flexws.appserver.dk";
         String lsBaseUrl = "http://flexws.appserver.dk";

         HttpWebRequest loHTTPRequest = createRequest(lsBaseUrl + string.Format("/flexws/latestsutmessage/?guid={0}&tz=Europe%2FAmsterdam&mask=", psGUID));

         LastAction = "And get data";
         StringBuilder sbResponse;
         sbResponse = getResponseText(loHTTPRequest);

         LastAction = "parse JSON answer";
         Gpsmessage result = new System.Web.Script.Serialization.JavaScriptSerializer().Deserialize<Gpsmessage>(sbResponse.ToString());

         DataReady = true;
         Latitude = result.sutmessage.gpsinfo.lat;
         Longitude = result.sutmessage.gpsinfo.lon;
         LastPosition = result.sutmessage.ts_full;
         LastUpdate = LastPosition;
         Temperature = result.sutmessage.temp.ToString();
		 
		 // Retrieving the location is another story... use google location services...
         LocationText = "";

      }


      /// <summary>
      /// Summary in string format
      /// </summary>
      /// <returns></returns>
      public override string ToString()
      {
         string lsRegels = "KommaTrace" + System.Environment.NewLine;
         lsRegels += string.Format("Location: {0}", LocationText.ToString(CultureInfo.InvariantCulture)) + System.Environment.NewLine;
         lsRegels += string.Format("GPS: {0};{1}", Latitude.ToString(CultureInfo.InvariantCulture), Longitude) + System.Environment.NewLine;
         lsRegels += string.Format("LastUpdate: {0}", LastPosition.ToString(@"yyyy-MM-dd HH:mm:ss")) + System.Environment.NewLine;
         lsRegels += string.Format("Temperature: {0}", Temperature) + System.Environment.NewLine;
         return lsRegels;
      }

      /// <summary>
      /// Summary in string format, plain and simple....
      /// </summary>
      /// <returns></returns>
      public string ToJSON()
      {
         string lsRegels = "{" + System.Environment.NewLine;
         lsRegels += @"""objectlocation"": {" + System.Environment.NewLine;
         lsRegels += string.Format(@"""description"": ""{0}"",", LocationText) + System.Environment.NewLine;
         lsRegels += string.Format(@"""temperature"": ""{0}"",", Temperature) + System.Environment.NewLine;
         //lsRegels += @"""gps"": {" + System.Environment.NewLine;
         lsRegels += string.Format(@"""latitude"": {0},", Latitude.ToString(CultureInfo.InvariantCulture)) + System.Environment.NewLine;
         lsRegels += string.Format(@"""longitude"": {0},", Longitude.ToString(CultureInfo.InvariantCulture)) + System.Environment.NewLine;
         //lsRegels += "}," + System.Environment.NewLine;  // GPS
         lsRegels += string.Format(@"""lastupdate"": ""{0}""", LastPosition.ToString(@"yyyy-MM-dd HH:mm:ss")) + System.Environment.NewLine;
         lsRegels += "}" + System.Environment.NewLine;  
         lsRegels += "}";
         return lsRegels;
      }
	  
      private HttpWebRequest createRequest(String psUrl)
      {
         HttpWebRequest loRequest = (HttpWebRequest) System.Net.WebRequest.Create(psUrl);
         loRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322)";
         loRequest.CookieContainer = new CookieContainer();

         // Expect100 does not behave well when using POST data
         //
         System.Net.ServicePointManager.Expect100Continue = false;

         loRequest.CachePolicy = new System.Net.Cache.RequestCachePolicy(
            System.Net.Cache.RequestCacheLevel.NoCacheNoStore);

         return loRequest;
      }

      private StringBuilder getResponseText(HttpWebRequest poRequest)
      {
         return getResponseText(poRequest.GetResponse());
      }

	  // Copy stream to stringbuilder...
      private StringBuilder getResponseText(WebResponse poResponse)
      {
         StringBuilder loData = new StringBuilder();

         byte[] buf = new byte[8192];
         Stream loResStream = poResponse.GetResponseStream();
         int count = 0;
         while ((count = loResStream.Read(buf, 0, buf.Length)) > 0)
         {
            loData.Append(Encoding.ASCII.GetString(buf, 0, count));
         }
         loResStream.Close();

         return loData;
      }

   }
	
   // The structures are for storing the device data.
   
   public class Echotag
   {
   }

   public class Gpsinfo
   {
      public double lat { get; set; }
      public double lon { get; set; }
      public int speed { get; set; }
      public int ttf { get; set; }
      public int radius { get; set; }
   }

   public class Gsminfo
   {
      public string cellid { get; set; }
      public int rssi { get; set; }
      public string lac { get; set; }
      public string mcc { get; set; }
      public string mnc { get; set; }
   }

   public class Sutmessage
   {
      public int ts { get; set; }
      public string date { get; set; }
      public string time { get; set; }
      public string nodeid { get; set; }
      public int tno { get; set; }
      public string iccid { get; set; }
      public string us { get; set; }
      public int ta { get; set; }
      public int temp { get; set; }
      public int version { get; set; }
      public int fwversion { get; set; }
      public Echotag echotag { get; set; }
      public Gpsinfo gpsinfo { get; set; }
      public List<Gsminfo> gsminfo { get; set; }

      public DateTime ts_full
      {
         get {
               DateTime epoch = new DateTime(1970, 1, 1, 0, 0, 0, 0).ToLocalTime();
               return epoch.AddSeconds(ts);
            }
      }
   }
   
   public class Gpsmessage
   {
      public Sutmessage sutmessage { get; set; }
   }

}

And how to we send this to Openhab? I use a python script because I was able to find an mqtt library for python.


import paho.mqtt.publish as publish
import json
from subprocess import check_output


outcar = check_output(["/opt/local/bin/mono", "/var/regent/KommaTrace.exe"])
#print outcar

data = json.loads(outcar)

print json.dumps(data["objectlocation"]["description"], indent=4, separators=(',', ': '))
print json.dumps(data["objectlocation"]["temperature"], indent=4, separators=(',', ': '))
print json.dumps(data["objectlocation"]["longitude"], indent=4, separators=(',', ': '))
print json.dumps(data["objectlocation"]["latitude"], indent=4, separators=(',', ': '))
print json.dumps(data["objectlocation"]["lastupdate"], indent=4, separators=(',', ': '))

# provide your mqtt credentials here:
loauth = {'username':"***username***", 'password':"***password***"}
publish.single("kommatrace/car/location",  json.dumps(data["objectlocation"]), hostname="openhabserver.local", auth=loauth, port=8883, retain=True, qos=0)

In openhab I have an item to receive mqtt messages:

String Car_location    "Car"  (GPS)   	{mqtt="<[openhabserver:kommatrace/car/location:state:default]"}

And some items to store the data:

Number Car_temp        "Temp [%s]"     	(GPS)
Number Car_latitude    "Latitude [%s]" 	(Status)
Number Car_longitude   "Longitude [%s]" (Status)
String Car_position    "Car [%s]"     	(Status)
String Car_lastupdate  "Time [%s]" 		(Status)

And I have a rule to process the data:

rule "Car Position"
when
    Item Car_location received update
then
    logInfo("Car", "Start.")

    var String json = (Car_location.state as StringType).toString
    //{"latitude": 52.0*** "lastupdate": "2015-10-22 14:54:15", "longitude": 4.****, "description": ""}
	
    var String lat  = transform("JSONPATH", "$.latitude", json)
    var String lon  = transform("JSONPATH", "$.longitude", json)
    var String pos  = transform("JSONPATH", "$.description", json)
    var String temp  = transform("JSONPATH", "$.temperature", json)
    var String last  = transform("JSONPATH", "$.lastupdate", json)

    postUpdate(Car_latitude,  lat)
    postUpdate(Car_longitude, lon)
    postUpdate(Car_positie, pos)
    postUpdate(Car_temp, temp)
    postUpdate(Car_lastupdate, last)
end

Happy coding!

Next time, how to plot location on map?

How to design floor tile plan with Inkscape

We have a spacious floor where we want to lay some tiles. The choice can be overwhelming. What colour, what size, how does the tile feel, how thick is it, questions questions.

Put too many small tiles on a large floor and the result can be distracting. Put very large tiles on the floor and the result can be a bit cheesy.

Why not go for some different patterns….

Well here comes the challenge, how many tiles do I fit in a room with a slanted pattern, or using two different sizes of tiles…

This seems like a simple question which can be answered by a computer! I have the floor plans in a pdf format, perhaps using inkscape I can read the floor shapes and use these as a base.

I found Inkscape has a way of programming extensions using python, excellent!

Rectangular Tiles

Tiles regularFirst challenge: If I know how wide and high a tile is, and I know how much grout joint is needed I can then calculate and draw the tiles. On my mac I can create an extension by writing a python program and copying the code to the folder:

/Applications/Inkscape.app/Contents/ Resources/share/inkscape/extensions

You need to place a python file and an inx file describing the extension.

Here is my first extension (inx file):

<?xml version="1.0" encoding="UTF-8"?>
<inkscape-extension xmlns="http://www.inkscape.org/namespace/inkscape/extension">
<_name>Tile rectangular</_name>
<id>nl.esweb.tile.rectangular</id>
<dependency type="executable" location="extensions">tile01.py</dependency>
<dependency type="executable" location="extensions">inkex.py</dependency>
<param name="tilewidth" type="int" min="5" max="100" _gui-text="Tile width in cm?">20</param>
<param name="tileheight" type="int" min="5" max="100" _gui-text="Tile height in cm?">20</param>
<param name="tilespacing" type="int" min="1" max="100" _gui-text="Tile spacing in mm?">16</param>
<param name="eenheid" type="int" min="1" max="1000" _gui-text="Eenheid 1cm is x meter?">1</param>
<param name="layer" type="string" _gui-text="Use layer with name?">Tile</param>
<effect>
<object-type>all</object-type>
<effects-menu>
<submenu _name="Tiles"/>
</effects-menu>
</effect>
<script>
<command reldir="extensions" interpreter="python">tile01.py</command>
</script>
</inkscape-extension>

There are some parameters and a reference to a tile01.py program. See attached zip file for the full source code.

Copy the files to the inkscape extensions folder and start up inkscape. Draw a box on your page, the extension needs this to know where to dray the tiles. Select the box and open the extensions menu, choose the Tile menu and the newly placed extension with name “Tile rectangular”.

A window will open and ask you the named parameters. Sorry for some left over dutch words but I could not always find a translation. Use google translate! Select Ok and the box will be filled with regular tiles, see example above!

Full Tiles Rectangular source here.

How is it done?

Well using inkscape python extension inkex you can add objects to a new layer.

newtile = inkex.etree.Element(inkex.addNS('rect','svg'))
newtile.set('x', str(runningx + spacingpx))
newtile.set('y', str(runningy + spacingpx))
newtile.set('width',  str( tilewidthpx))
newtile.set('height', str( tileheigthpx))
newtile.set('style', formatStyle(tilestyle))
layer.append(newtile)

Using a few loops to test if we are still within the bounds of the square we can then work across and down to fill the whole square.

Halfsteens Tiles

Tiles HalfSteens

Another tile pattern: In dutch Halfsteens. Half stone or Half layout? Well this is what it looks like actually:

Also a classic pattern seen often in de Boerderij.

Code was pretty easy, each new row is offset by half a tile and repeat.

 

Here is the code Tiles Half Layout.

But now some more complex patterns.

Tiles Big and small

Tiles Big and SmallWhen you mix a larger tile and a smaller tile you can get spectacular designs. Often this involves a lot of extra cutting of the bigger tiles to accommodate for the small tile. But this wastes a lot of material. Therefore the adjusted big and small tiles can be used.

Can work really well where the large tile has a base colour and the smaller tiles add fresh colours.

Here is the code Tiles Big and Small

Tiles one way or the other

Tiles Up and outAnother classic pattern. Found often in Italy using terra cotta.

Here is the code Tiles Om en Om

But the next tile had me puzzled. I thought I might have to get my school books out to calculate triangles and circles to measure correct movement between rotated tiles. But I figured I could use turtle plotting! I never thought I would get to use the technology.

 

Fish pattern of Vissegraat

Tiles VissegraatIn the next diagram you can see the visgraat pattern.

The best way I could describe this was using turtle commands.

turtle.setheading( 45)

turtle.pendown()
turtle.forward( tileheigthpx)

turtle.right(  90)
turtle.forward( tilewidthpx)

turtle.right( 90)
turtle.forward( tileheigthpx)

turtle.right( 90)
turtle.forward(tilewidthpx)
turtle.penup()

And there you have it. Set heading 45 degrees. pendown, go forward, turn right etc… And we have base tile tilted 45 degrees. Perfect!

Want the code Tiles Vissegraat.

Be creative

I hope these pages can help you plan your floor tiles.

And for the dutch readers here are some search words to help find this document.

Tegels kunnen ook met deze extensie ingedeeld worden. Wij gebruiken het voor boeren plavuizen of estrikken. Terracotta wordt het ook wel genoemd of gebakken klei tegels.

Draytek Vigor blues

Draytek used to have top technology, way in the past. But nowadays I keep bumping into trouble.

A few years ago I had a new Ziggo internet connection at home with, at the time, a super speed of 120 Mbit/s. It was so fast, I was super happy. After installation I found that Ziggo does not provide fixed ip addresses on the super fast access. The only route they have is a much more expensive business option.

I decided to go the other route of dual wan setup because I still had my old and trusty adsl connection with fixed ip.

I then went looking for a device which has dual wan capability and the option of fast internet access. My choices were very limited and I came up with the Draytek Vigor 2920.

After some initial setup troubles (every router has these) I managed to get it up and running nicely. For many years it ran without flaw.

Recently though my connection was becoming slower and slower. Sometimes timeouts would occur. I hooked up my laptop behind the vigor and bang speed to the max.

I updated firmware to latest version, no luck..

It seems the device is “out of support”.

Well, no choice but to upgrade to a new dual wan device. In comes the Cisco RV042G. After install I immediately had super speed, but what a hassle to configure everything..

I setup port forwarding and protocol rules to make sure smtp traffic is processed to the correct route. But to no avail I just cannot get these rules, ports and firewall working together.

Maybe swap the Cisco for a new Draytek vigor, I don’t think so…..

Only one option left…. Scrap the dual wan setup. Will make life so much easier!

Will go for hosted email and website, be gone dual wan setup. Your making my tech life too complicated.

Synology Spamassassin

Recently I converted my mail server to the Synology mailserver. I use a setup with IMAP accounts for storing all my emails.

Getting a running mail server converted to Synology went quite well and quickly. I must say I am impressed with the result.

What I do find scary is the backup facility. I really need to check if I can restore this backup easily. But how do I do this, I do not have a spare Synology to try this on. Maybe a project for a rainy day, a nice virtual image with the Synology software would be nice!

But now the subject at hand. After a few days I was again receiving spam messages, something I had overcome on the previous mailserver using assp. This should be a problem easily tackled on the Synology I thought.

Just go into the UI and check “Enable SpamAssassin filter engine” and boom, no spam.

The engine did not catch much… Okay maybe it needs further setup. Just go into Settings and enable auto learning?

Well as I have found before, Synology is a looker but often not a performer. I need to dig into the internals.

I was amazed to find on the web that spamassassin is bundled with some amazing tools. One that solves another of my long wished for features. How to you manage spam as a mail user without needing to access a mail server.

The solution is simple, each user on the mail server need to have a standardized Junk email folder in their IMAP store. On the mail server we regularly schedule to update a bayesian index on every users Junk folder. Now when new spam mail comes in and is not detected a user only needs to mark it as spam on his iphone, the mail is moved to Junk and the server indexes this every 2 hours or so to prevent this spam from coming in again.

Theory is great but now the setup!

The tool is called sa-learn and is called in the following way:

/var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/

The last parameter is the magic trick to look at all the Junk mailboxes. Great, only it does not work on Synology….. 🙁

This is due to perl configuration. But fear not, others have found a solution, you need to call it as follows:

perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/

Problem is this will probably break on the next major DSM update but we will cross that bridge when we come to it.

The tool has some more tricks, so what I do is create the a script with name /opt/spamscan.sh with the following contents:

#!/bin/sh
date
echo "Scanning Spam Folders(s)"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
echo "Updating SpamAssassin DB"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --sync
echo "Backing up SpamAssassin DB"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --backup > /opt/spamassassin_backup.bak
date

Use chmod on the script file to allow executing. chmod u+x /opt/spamscan.sh chmod g+x /opt/spamscan.sh chmod o+x /opt/spamscan.sh And I schedule this using /etc/crontab: 0 */2 * * * root /opt/spamscan.sh >> /var/log/spam_learn.log Wow, now all ready and done. You need to be patient. Having 20 Junk mail messages is not good enough for bayesian statistics. About a month later I had a few hundred but was not seeing all my spam being detected. Hmm BIG disappointment! Lets dig further. If you use your mail client to look at messages you can see there are added headers in the format of: X-Mailscanner. I found that some messages have a line X-Mailscanner-Spamcheck: not spam, SpamAssassin (not cached, score=3.6, required 5, BAYES_99 3.50, RDNS_NONE 0.10) but others do not. This is not working as I expected. I also found that the Mailscanner has 5 running instances on my Synology all taking about 50 Mb of memory, I have only limited mails coming in every day so I need to tone this down, the Synology only has 512 Mb… Okay lets dig into this and find the config files for SpamAssassin and Mailscanner. You can find these in /var/packages/MailServer/target/etc/MailScanner. But be careful changing this file, on next reboot it will be overwritten bij de config template. That is wat needs to be changed. So get your self a cup of coffee and use your favorite editor to update the file /var/packages/MailServer/target/etc/template/mailscanner.template, there are hundreds of options. Also nice to know, logging for the mail server is sent to /volume1/@maillog/ and not /var/log. The items I modified are:

%org-name% = <your name here>-
Max Children = 3
Always Include SpamAssassin Report = yes
Multiple Headers = add
Place New Headers At Top Of Message = yes
Log Spam = yes
Log Non Spam = yes

And now after running a while the results are much better. When a spam messages gets through, the headers will explain why, usually because spam score is not high enough. I add this email to Junk and the next identical spam message will always have a higher score until it is recognized correctly. I am a happy camper.

Thank you Synology for providing the platform and thank you internet for helping to configure the damn thing!

EnOcean for Home Automation

I have been looking for ways to automate simple things in my house. Some technologies seem impressive but when I started using them I was often disappointed at the reliability. The new technology from EnOcean seems to solve many issues. Find out how I interfaced some EnOcean modules using .Net framework and c#.

read more…

Storing

Door technische storing van de hardeschijf zijn de sites een week down geweest. Tot mijn grote schrik bleek mijn foto archief niet in de backup te zitten… Gelukkig is alles weer terug na het inschakelen van bedrijf Yodata in Ede. Daar is alle informatie van de onleesbare schijf terug gezet. Een knap staaltje Data Recovery! (voor meer informatie zie www.yodata.nl) Nu is alles weer up and running.