Ian's random blog

Repair, Reuse and Recycle

Sun, 09 Nov 2025 00:00:00 +0000

How I created a spare monitor

I’m big on reuse, repair and recycling which is one of the reasons I volunteer at Glasgow Repair Cafe – I blame being exposed to “Stig of the Dump” at an impressionable age. As a result I struggle to throw out broken things let alone things that still work. However, space limitations mean that I do occasionally have to tidy out some stuff. I discovered that I have way too many old laptops that will come in useful sometime recently. So I installed a light weight linux on two of them and donated one of them to the repair café (where we have several laptops that really should be retired as they lack some keys), the other one is an old ThinkPad which I will use in my workshop/garage so that I don’t get sawdust in my work laptop.

Two of them proved to be beyond repair (or wouldn’t talk to the network with a modern linux), this left me looking at a pile of potential e-waste. And, while they didn’t work as computers any more there were still some useful bits on them. There was also the question of what to do with all the nice stickers I had on the back of them.

The Build

I decided that I could use an(other) 2nd monitor, and since the screen was still good I thought I could reuse a. The first step is to disassemble the laptop and extract the display. You then search for the serial number on the back (in my case NT156FHM-N31) which lead me to an Amazon or E-Bay page where for about £20 I could buy a pair of small PCBs. The thin ribbon cable coming out of the screen goes in one side and a USB-C cable an HDMI cable go in the other. Plugging the other ends in to the relevant ports on my laptop and hey presto I had a 2nd monitor. Unfortunately (but unsurprisingly) it was set to Chinese, but it turns out Google lens can do sufficient real time translation to allow me to figure out the language settings. After that it has a menu system that is remarkably similar to ever TV I’ve used in the past decade.

I could have stopped there but it looked a little rough and was a bit fragile. So, I toyed with 3D printing an enclosure or building one out of wood (this one is William Gibson’s fault). But life got in the way, so about 6 months ago I was looking to tidy up enough office space to start thinking about dping something when I found an LCD screen and some PCBs. This is when I had the break through I already had an enclosure that was exactly the right size and shape for the screen – the laptop lid. If I removed it from the hinges (which I saved as they are sure to be useful one day) it would clip back together nicely and make the screen nice and safe. I just needed to work out how to hold it upright, here I found an old microphone stand that I had picked up in a charity shop thinking I could build an angle poise style light out of it. With the addition of a VESA monitor bracket plate and a ball head mount I had a working monitor which was height and angle adjustable.

The final problem was where to put the control boards, again I should really build them an enclosure, but for the time being they are attached with stand offs to the back of the case. I also had to cut a small hole in the bottom of the case to pass the ribbon cable through but as its soft plastic this was easy enough.

Now that it is finished and in regular use as my main second monitor it is working really well and for about £30 I can’t complain. If I ever need to take it apart I will probably drill some new holes and move the control panel to be vertical and nearer the edge as where it is at the moment is a little fiddly to turn it on and off. I have another partly disassembled laptop which might well find its screen going the same way as I can always use more displays around the house.

Health and Employment Update

Sat, 08 Nov 2025 00:00:00 +0000

It appears that I haven’t posted anything here for a year! There are some mitigating circumstances though.

TL;DR; I didn’t die and I have a new job.

The Build Up

Coincidently, about a year ago I started to struggle with my mental health. Initially I didn’t worry too much as the onset of winter and the clocks going back always adversely affects my mood, and living in Scotland makes this worse as it gets lighter later and dark earlier than when we lived further south. Normally, I would expect for my mood to pick up in the new year when the days lengthen and its safe to go outside again. But this year it didn’t, I don’t know why, maybe working at a university wasn’t as much fun as I had hoped it to a. Certainly it turns out that just being called a research software engineer didn’t make as much of a difference to my quality of life as I had hoped it would. I struggled on until Easter time (which was clearly a mistake with hindsight), when it became clear to everyone and even me that I needed to take some time off. So anyone who was hoping to meet up with me at GISRUK I’m sorry but I really couldn’t face travelling or meeting people. I spent much of April in bed sleeping while I waited for a bigger dose of anti-depressants to kick in (this can be between 6 and 10 weeks, on top of the fortnight to wean off the previous ones).

I also started taking Manjaro to help with my weight and diabetes along with a tailored exercise program, so by the end of May I had lost 10 kilos and was feeling pretty good about life in general. Then …

The Incident

As many of you will have heard I suffered a significant health issue in early June when my colon decided that it had had enough and decided to start bleeding unexpectedly. I went out for dinner one evening with some friends from university and the next thing I really knew was I was in a critical care ward with the staff explaining to me again and again that I was unwell. Between the loss of blood and morphine I was handling this information very poorly. This confusion wasn’t helped by the fact that I knew the Royal Victoria Infirmary (RVI) in Newcastle very well from my student days as we used to use its historic corridors as a (dry) short cut to and from the university most days. This hospital was clearly much newer and shinier than the one I remembered. It turns out the old one was knocked down and replaced by student flats some years ago.

I had to come to terms with a) having nearly died (though I don’t remember), b) no longer having a colon and an Ileostomy and having a scar from waist to sternum (held together with massive staples) and c) having a stoma bag attached to me (most of the time).

Anyway, after 10 days or so I was considered well enough to move to a normal ward, though I promptly let my blood pressure collapse and was moved back to ICU for some magnesium, but eventually I was well enough to transfer back to Scotland. I can’t say I recommend travelling up the M74 in an ambulance but it was good to be back near to home, especially for Lesley.

After another week on a high dependency ward in East Kilbride (since I had transferred from ICU as there were no “normal” beds available for me to take in Newcastle) I had to go into an HDU bed (I know). I was easting normally, finally, and could walk from one end of the ward to another (lying in bed for a fortnight really kills your muscle tone) and I was allowed to go home.

Early the next week I went back for a stoma clinic checkup and the nurses took one look at me and decided that maybe I should visit the ambulatory emergency unit (despite, or because, needing a wheelchair to get there). They promptly readmitted me as my blood pressure was down below 80, which explained why I wasn’t feeling too chipper. I spent a few more days in hospital being given IV magnesium and antibiotics to sort out my blood pressure and inflammatory markers. Things could have been worse, the guy in the next bed had a gall stone the size of a tennis ball taken out of his intestines as it had bored its way out off his gall bladder. The surgeons were very excited and had taken pictures of it to show him.

Recovering, took much longer than I expected though all the medical staff kept saying how tough an operation was. This meant that I wasn’t well enough to travel to Mostar for FOSS4GEU which again was a shame (especially since I had been organised enough to actually book travel and hotels in advance). So again sorry if you were hoping to catch up with me there.

The Recovery

My contract at Glasgow ended at the start of October, which was about when my doctors were becoming happy for me to go back to work. I had half heartedly filled out some applications for new university jobs and dumped my CV on various job sites. Though to be honest I was really looking forward to being able to say in January that clearly I was unemployable and should consider retirement. Sadly (or happily) I got a call from a recruiter who seemed desperate to hire me (but couldn’t really say why). He was offering a quite silly amount of money with the only catch being the need to incorporate. So, I agreed to an interview and it turned out to be with a system integrator who was working for His Majesty’s Land Registry (HMLR) who maintain the English and Welsh cadastral register. Quite why they were so keen to hire me was still unclear but I said yes to a six month contract. When I started at the HMLR it became clear why they wanted me, they are trying to update part of the register (the only vectorized part) from a proprietary format that was developed by Envitia (who I had worked for back in 2011-15) and were trying to convert to a PostGIS DB using GeoTools to build a Java tool.

Adjustable, draggable circles around markers in leaflet

Tue, 05 Nov 2024 00:00:00 +0000

I recently needed to implement a way of showing the uncertainty of a marker in a web map (the final application isn’t ready yet) and while I found a bunch of tutorials and stackoverflow answers that covered parts of this I couldn’t find a single page on how to do it. So I’m going to write it up here so that next time I need to do this a search will find this.

TL;DR; You can see a simple demo here, and here’s a picture to show you what I’m talking about.

There are 3 markers (in my app anyway) and they are represented using a coloured flag. At its simplest the user can just pick a position and if they are happy with that move on to the next step (in the image above this is the blue flag). If however they are less sure of the location they can double click on the marker which will add a 100m circle (in a matching colour) around the point (the green flag above). Clicking on the circle will display a popup which allows them to adjust the radius of the circle (the red flag above). Also at any time the user can click on a flag and drag it (and it’s circle) to a new position on the map.

function addMarker(n) {
    name = n;
    if (lastClick) {
        L.DomUtil.removeClass(map._container, `${lastClick}-flag-cursor-enabled`);
    }
    lastClick = names[name];
    if (!positions[names[name]]) {
        L.DomUtil.addClass(map._container, `${names[name]}-flag-cursor-enabled`);
        map.on('click', setMarker)
    } else {
        map.panTo(positions[names[name]].getLatLng());
    }
    if (Object.values(positions).length == 3) {
      // proceed to remainder of the app
    }
}

When the user clicks on one of the buttons, addMarker is called, this clears the previous cursor if it was set and then if this colour of marker hasn’t been set changes the cursor to match the flag we are adding or pans to that marker if it has been placed on the map. Then we attach the setMarker method to the map’s click event, so when they click we can place a marker there.

function setMarker(e) {
    map.removeEventListener("click", setMarker, false);
    if (!positions[names[name]]) {
        var icon = L.icon({
            iconUrl: icons[names[name]],
            iconSize: [30, 30],
            iconAnchor: [20, 27],
        });
        lat = e.latlng.lat;
        lon = e.latlng.lng;
        //Add a marker to show where you clicked.
        var latlng = L.latLng(lat, lon);
        var marker = new L.marker(latlng, {
            icon: icon,
            title: labels[names[name]],
            draggable: false,
            clickable: true,
            autoPan: true,
        }).addTo(map);
        marker.on('dblclick', (e) => {
            addCircle(e.target);
        });
        marker.on('click', mapClickListen);
        /* this is all to work around dragend triggering a click
         * see https://gis.stackexchange.com/questions/190049/leaflet-map-draggable-marker-events
         */
        marker.on('dragstart', function(e) {
            console.log('marker dragstart event');
            marker.off('click', mapClickListen);
        });
        marker.on('dragend', (e) => {
            e.target.dragging.disable();
            //pointsValid();
            marker.on('click', mapClickListen);
        });
        positions[names[name]] = marker;

    } else {
        map.panTo(positions[names[name]].getLatLng());
    }
    L.DomUtil.removeClass(map._container, `${names[name]}-flag-cursor-enabled`);
}

Again, we check if the marker has already been placed (I’m paranoid) and if not go ahead and create a flag icon with the correctly coloured image. The IconAnchor makes sure that the base of the flag pole is placed on the click location. Then we create a new Marker using the icon at the latitude and longitude of the click, we set it to be clickable but not draggable. The idea is (was) that the user should need to click on a marker to move it, which works but the end of the drag seems to click on the marker again so it never turns off. So I found a workaround from 2016 to the bug and which was apparently fixed some years ago but I can’t seem to make it work. Any way the key step is the add addCircle to the double click event.

function addCircle(marker) {
    name = names[marker.options.title];
    if (!circles[name]) {
        const popup = document.createElement("div");
        popup.innerHTML = 'adjust radius ';
        const spinner = document.createElement("INPUT");
        spinner.id = name;
        spinner.setAttribute("type", "number");
        spinner.setAttribute("min", "10");
        spinner.setAttribute("max", "200");
        spinner.setAttribute("step", "10");
        spinner.setAttribute("value", "100");
        spinner.addEventListener('input', function(e) {
          circles[e.target.id].setRadius(e.data);
        });
        popup.appendChild(spinner);
        circle = L.circle(marker.getLatLng(), 100, {
            color: colors[name],
            fillcolor: colors[name],
            fillopacity: 0.5,

        }).bindPopup(popup).addTo(map);

        marker.on('drag', (e) => {
            circles[names[e.target.options.title]].setLatLng(e.latlng);
        });
        circles[name] = circle;
    }
}

As is usual we check if there is already a circle with this name before we create one. Then we generate the popup that allows the user to change the radius of the circle. This is a basic HTML div with a spinner (or number as HTML calls them for some reason), we can set the minimum, maximum and step for this in the HTML as per normal. The trick then to add a function to the input event that is fired when ever the user clicks the spinner (or uses their up/down arrows on it). The remaining issue is to make sure that the right circle gets updated when the input changes, to do this I set the id of the spinner to the same name that keys the dictionary of circles (these keys match the markers too). Then we can simply use the setRadius method of the CircleMarker that we are about to create. Then we create the CircleMaker and attach the popup to it and add it to the map. Finally, when we drag the matching marker we want to move we need to change the centre of the circle. This needs a function for the drag event to do this, it’s slightly more complex to look up the required circle from a marker due to makers having nice long titles so there’s a double lookup, if I had millions of markers this might be a problem but with 3 it’s fine.

And that’s all there is too it, that’s not to say I like JavaScript or web development but in the end its not too hard!

How to reproject features in QGIS

Fri, 09 Aug 2024 00:00:00 +0000

I came across a brilliant thread on Mastodon by Sarah Dal discussing how she worked out what the most remote post box in the UK was. But there was one remark that bothered me, she said that to convert her lat, lon points to OSGB (EPSG:4326 to EPSG:27700 for the nerds) she had to go to the Ordnance Survey site to find some software to do this. Since she already had the points in QGIS this seemed like an unnecessary side quest to me. When I commented that she could have done this in QGIS she said that she always seemed to get it wrong.

I’m confused by this (and by the many other people who ask questions on gis.stackexchange.com about the same thing) so I’ve put together this video showing how I would do this. Basically, you right click on the layer you want to reproject and select export->Save features as and then just change the drop down box to the projection you need. QGIS will then save the features and add that layer to your project. You shouldn’t see any difference other than the colour of the points will change (as QGIS assigns a new random colour to the new layer).

And yes, I really should have split the Northern Irish post boxes out into a separate file and projected them to the Irish grid but this is just a demo.

Adding a spell check to QGIS

Tue, 16 Jul 2024 00:00:00 +0000

Adding a Spell Check to QGIS

(Or what to do on a rainy bank holiday in Glasgow)

This Monday was a local bank holiday in Glasgow (or at least the university) as a remnant of when the whole town took a train to Blackpool in the same two weeks so that the ship builders and steel works could stop in a coordinated fashion. As is required in the UK the weather was awful so I stayed in and being bored looked at my long list of possible projects. I picked one that has been kicking around on the list for a while adding a spell checker for QGIS. As a dyslexic I have spell checking turned on in nearly every program I enter text into including vim, InteliJ and my browser. So I have always felt that what QGIS really needed was a way to spell check maps before I printed them at A3 and put them on the wall.

Back in 2019 North Road wrote a iblog post about custom layout checks and ended it with a throw away comment “It’d even be possible to hook into one of the available Python spell checking libraries to write a spelling check!”. I came across this when I was trying to see if there was an easy way for my students (many of whom have English as a second language) to avoid handing in projects with glaring (i.e. I can see them) spelling errors in the title. So I stuck the link on my backlog, until the proverbial rainy day came along.

Implementation

Obviously I’m the last person who should be allowed to write spell checking software, but the joy of open source is that for things like this someone else has almost certainly already done it. So a quick duck-duck-go found me installing pyspellcheck which seemed like it would do what I want. It has a pretty easy interface in that once you’ve created a spell checker object, you can just pass in a list of words and it will return a list of (probably) misspelled words and a method to give the most likely correction and another method to give you list of other possibilities. Armed with this I could create a method to find and check all the text elements of a print layout.

@check.register(type=QgsAbstractValidityCheck.TypeLayoutCheck)
def layout_check_spelling(context, feedback):
    layout = context.layout
    results = []
    checker = SpellChecker()

    for i in layout.items():
        if isinstance(i, QgsLayoutItemLabel):
            text = i.currentText()
            tokens = [word.strip(string.punctuation) for word in text.split()]
            misspelled = checker.unknown(tokens)
            for word in misspelled:
                res = QgsValidityCheckResult()
                res.type = QgsValidityCheckResult.Warning
                res.title = 'Spelling Error?'
                template = f"""
                <strong>'{word}</strong>' may be misspelled, would
                '<strong>{checker.correction(word)}</strong>' be a better choice?
                """
                possibles = checker.candidates(word)
                if len(possibles) > 1:
                    template += """
                    Or one of:<br/>
                    <ul>
                    """
                    for t in possibles:
                        template += f"<li>{t}</li>\n"
                    template += '</ul>'
                res.detailedDescription = template
                results.append(res)
    return results

And in theory, that was that! But I’m pretty sure that my students (and everyone else) probably didn’t want to cut and paste that into the console every time they wanted to spell check a map. So, I looked at how to package this up for QGIS. I built a plugin (using the plugin builder tool), but then things got a little tricky as I can’t see any way for a plugin to add itself to the print layout rather than the main QGIS window (please let me know if it is possible), and it seemed unintuitive to make people press a button in one window to effect another one, besides the whole point of being a QgsAbstractValidityCheck was that the method is automatically run on print. So I didn’t need most of the plugin code or did I? On further thought I did, there is a need for some GUI as the user can pick which language they want to use in the spell check. pyspellcheck can spell check English, Spanish, French, Portuguese, German, Italian, Russian, Arabic, Basque, Latvian and Dutch (so if those are your language then please test this for me). I also thought that providing the option to supply a different to the default personal dictionary might be useful. So that made use of the dialog that pops up when you hit the plugin.

But it turns out you can’t register a class method as as a QgsAbstractValidityCheck since it gets confused when QGIS calls it later. So I had to move my checker method outside the plugin class. But then I couldn’t access the language and dictionary that was set in the GUI! Some more searching gave me the following code:

  _instance = plugins['qgis-spellcheck']
  checker = _instance.checker

Whereby I can pull out the named plugin and grab it’s spell checker, which was created in the plugin’s __init__ method. I seem to have a small issue that the user’s profile is not set when that runs which messes up where the personal dictionary is put (again if you know how to fix this let me know).

Future Work

Ideally, I’d like the spell checker to scan and highlight the text in the boxes as I typed but I fear that is beyond my understanding of the QGIS/Qt interface. Next highest on my wish list is for the list of spelling issues to be non-modal so I can cut and paste fixes into the text box, rather than having to memorise the correct spelling, close the window and then type it in (again answers on a github issue).

I’m sure all sorts of things will come up once people start using it, so as usual issues and PRs are welcome at https://github.com/ianturton/qgis-spellcheck.

Is GeoJSON a spatial data format?

Sat, 11 Nov 2023 00:00:00 +0000

Is GeoJSON a good spatial data format?

A few days ago on Mastodon Eli Pousson asked:

Can anyone suggest examples of files that can contain location info but aren’t often considered spatial data file formats?

He suggested EXIF, Iván Sánchez Ortega followed up with spreadsheets, and being devilish I said GeoJSON.

This led to more discussion, with people asking why I thought that, so I instead of being flippant I thought about it. This blog post is the result of those thoughts which I thought were kind of obvious but from things people have said since may be aren’t that obvious.

I’ve mostly been a developer for most of my career so my main interest in a spatial data format is that:

it stores my spatial data as I want it to,
it’s fast to read and to a lesser extent, write.
It’s easy to manage.

One, seems to be obvious, if I store a point then ask for it back I want to get that point back (to the limit of the precision of the processor’s floating point). If a format can’t manage that then please don’t use it. This is not common but Excel comes to mind as a program that takes good data and trashes it. If it isn’t changing gene names into dates then it’s reordering the dbf file to destroy your shapefile. GeoJSON also can fail at this as the standard says that I must store the data in WGS:84 (lon/lat), which is fine if that is the format that I store my data in already, but suppose I have some high quality OSGB data that is carefully surveyed to fractions of a millimetre and the underlying code does a conversion to WGS:84 in the background and further the developer wanted to save space and limited the number of decimal places to say 6 (OK, that was me) when it gets converted back to OSGB I’m looking at centimetres (or worse) but given the vagaries of floating point representation I may not be able to tell.

Two, comes from being a GeoServer developer, a largish chunk of the time taken to draw a web map (or stream out a WFS file) is taken up by reading the data from the disk. Much of the rest of the time is converting the data into a form that we can draw. Ideally, we only want to read in the features needed for the map the user has requested (actually, ideally we want to not read in most of the data by having it already be in the cache, but that is hard to do). So we like indexed datasets both spatial indexes and attribute indexes can help substantially speed up map drawing. As the size of spatial datasets increases the time taken to fetch the next feature from the store becomes more and more important. An index allows the program to skip to the correct place in the file for either a specific feature or for features that are in a specific place or contain a certain attribute with the requested value. This is a great time saver, imagine trying to look something up in a big book by using the index compared to paging through it reading each page in turn.

After one or more indexes the main thing I look for in a format is a binary format that is easy to read (and write). GeoJSON (and GML) are both problematic here as they are text formats (which is great in a transfer format) and so for every coordinate of every spatial object the computer has to read in a series of digits (and punctuation) and convert that into an actual binary number that it can understand. This is a slow operation (by computer speeds anyway) and if I have a couple of million points in my coastline file then I don’t want to do 4 million slow operations before I even think of drawing something.

Three, I have to interact with users on a fairly regular basis and in a lot of cases these are not spatial data experts. If a format comes with up to a dozen similarly named files (that are all important) that a GIS will refuse to process unless you guess which is the important one then it is more of a pain than a help. And yes shapefile I’m looking at you. If your process still makes use of Shapefiles please, please stop doing that to your users (and the support team) and switch over to GeoPackages which can store hundreds of data sets inside a single file, All good GIS products can process them by now, they have been an OGC standard for nearly 10 years. If you don’t think that shapefiles are confusing go and ask your support team how often they have been sent just the .shp file (or 11 files but not the .sbn) or how often they have seen people who have deleted all the none .shp files to save disk space.

My other objection to GeoJSON is that I don’t know what the structure (or schema) of the data set is until I have read the entire file. That last record could add several bonus attributes, in fact any (or all) of the records could do that, from a parsers view it is a nightmare. At least GML provides me with a fixed schema and enforces it through out the file.

When I’m storing data (as opposed to transferring it) I use PostGIS, it’s fast and accurate, can store my data in whatever projection I chose and is capable of interfacing with any GIS program I am likely to use, and if I’m writing new code then it provides good, well tested libraries in all the languages I care about so I don’t have to get into the weeds of parsing binary formats. If I fetch a feature from PostGIS it will have exactly the attributes I was expecting no more or less. It has good indexes and a nifty DSL (SQL) that I can use to express my queries that get dealt with by a cool query optimiser that knows way more than I do about how to access data in the database.

If for some reason I need to access my data while I’m travelling or share it with a colleague then I will use a GeoPackage which is a neat little database all packaged up in a single file. It’s not a quick as PostGIS so I wouldn’t use it for millions of records but for most day to day GIS data sets it’s great. You can even store you QGIS styles and project in it to make it a single file project transfer format.

One final point, I sometimes see people preaching that we should go cloud native (and often serverless) by embracing “modern” standards like GeoJSON and COGs. GeoJSON should never be used as a cloud native storage option (unless it’s so small you can read it once and cache it in memory in which case why are you using the cloud) as it is large (yes, I know it compresses well) and slow to parse (and slower still if you compressed it first) and can’t be indexed. So that means you have to copy the whole file from a disk on the far side of a slow internet connection. I don’t care if you have fibre to the door it is still slow compared to the disk in your machine!

How much fun was FOSS4g 2023?

Tue, 18 Jul 2023 00:00:00 +0000

Review of FOSS4G 2023

Introduction

FOSS4G 2023 was held in Prizren, Kosovo and there were 800+ attendees from 120+ countries. This was a great attendance despite Kosovo not being the easiest place to get to, especially if you were me. I had booked flights to and from Belgrade, Serbia because they were a relatively good times for me (I am missing being only 1 hour from a London airport). It looked like an easy 4 hour drive from the airport to Prizren. Then about a fortnight before the conference Kosovo closed it border to Serbian registered cars! A subsequent check all showed that most of the hire companies wouldn’t let you drive one of their cars over a border (and apparently despite the Serbian Government’s claims they considered Kosovo a separate country). So I fell back on the 9 hour overnight bus trip, which wasn’t as bad as it could have been but did mean I started the conference already somewhat sleep deprived.

Sort of Historical Side Note

When I was at school learning Geography all of this area was called Yugoslavia and it was behind an “iron curtain” so we knew little about it and cared even less (it was not on the exam). In history it was covered by the break up of the Ottoman Empire and the meddling of Bismarck as an “honest broker” at the 1878 Congress of Berlin (as I recall he rigged the Balkans in such a way as to start World War I, but history O-Level was a long time ago). So you can already see that there is a long a complex history and geography in the region. It became more complex in the 1990s when Yugoslavia broke up in a series of bloody conflicts, which I remember from the evening news. Kosovo only really came to my attention in 2008 when I was walking through New York with James Macgill and his kids and we saw a large group of cars flying large red flags with double headed eagles on them (and the one thing I retained from my history lessons is the ability to spot a double headed eagle at 200 metres). These were Kosovans celebrating the declaration of independence being proclaimed.

So squabbling about car number plates was pretty peaceful really. Fortunately, for us most of KFOR (the UN/Nato peace keeping force) had gone home, except for some in the Northern predominately Serb areas where they were out keeping the piece over disputed elections for city mayors. This meant we could use their old base as a conference venue, which was great except for the lack of lifts.

Cool talks

This is a list of talks I found particularly interesting or fun and that stuck in my mind long enough to remember for this post. There were lots of great talks that I either failed to see because the room was too full or was too hot for my brain to have taken in the details, so I’m looking forward to the videos coming out so I can refresh my memory later.

Neiene Boeijen Let’s put it on the map!
- Interesting discussion on why graphic designers shouldn’t make maps and geographers shouldn’t design user interfaces
Benjamin Trigona-Harany Aircraft trajectory analysis using PostGIS
- Explains how PostGIS handles trajectories which are really just a Linestring with MZ values on the points where M increases along the line
- We can create a foreign table based on a foreign “server” that converts to a REST API call in the background this allows real time mapping filtered by the bounds of the map and any other filters the API can handle.
Jody Garnett and Andrea Aime GeoServer used in fun and interesting ways & State of GeoServer
- As usual Andrea and Jody were presenting way too many talks including this useful review of what GeoServer is capable of in production and a review of new features in GeoServer for this year.
Iván Sánchez Ortega Gleo
- I was going to say this is a new WebGL mapping library, but apparently Iván has been going on about this for 8+ years now. Any way there is lots of object orientated style javascript code in there so you can extend your semi-transparent markers with bouncing. I don’t think it’s quite cool enough for me to start learning JavaScript or WebGL but the demo with 10,000 randomly coloured bouncing map markers is fun in a <blink> sort of way.
Sanghee Shin Let’s defense my country using FOSS4G!
- Discussed how South Korea is modernising its military mapping with GeoServer, PostGIS, Cesium and OpenLayers, which makes me feel better about knowing North Korea is using GeoServer and PostGIS too.
Andrea Aime Processing and publishing Maritime AIS data with GeoServer and Databricks in Azure
- Andrea standing in for Nuno discussed how to handle 1.5 Billion records in GeoServer. Looking at the AIS ship location data over 7 years at second intervals (I think) and how you could get GeoServer to display the relevant data in a fast way.
Logan Williams Investigating war crimes, animal trafficking, and more with open source geospatial data
- Logan is a Bellingcat reporter who uses open source tools to look at open source data to help confirm stories. For example he described how they looked at tracking a GRAD missile launcher from Russia to Ukraine before Malaysian Airliner was shoot down. This included spotting scuffs on the road and distinctive power lines in posted dash camera footage of the convoy.
- What ever you do don’t play GeoGuesser against these guys as they are going to be very good.

Drawing a line on a GeoTools Map

Mon, 08 May 2023 00:00:00 +0000

Drawing a line on a GeoTools Map

As anyone following the dismal news out of the UK recently will know we had some sort of royal extravaganza down south in England this weekend, so with nothing better to do (and an extra day off) I decided to do some playing with GeoTools. This was mostly motivated by the work I had already done in answering this gis.stackexchange.com question about how to draw a line on top of a map. It seems that there isn’t much about how to do this in the GeoTools documentation.

So I scratched my head and dredged up what I could remember about swing and particularly about the GeoTools swing module and came up with some code that answered the immediate question which was why the OP got many layers in the map rather than one. I decided that this might be useful for other people so I tidied the code up and created a small project that contains a DigitizerAction and a Digitizer class which is the actual tool. It’s pretty simple all it does is place a series of dots on the screen and then generates a LineString which is added to a list, which is then used to generate a new FeatureLayer and removes the old layer (if it exists).

I finished up by adding a little demo program which adds a draw button to the toolbar and displays the US States for you to draw over.

All you do is click the draw button, then each click on the map will add a point to the current line, a double click finishes the line and forces a redraw of the screen with the line now in red.

Details

For anyone who’s trying to create a new tool for their swing application with GeoTools here is a little more on how it works.

Action

We need to create an Action to tell swing what we plan to do, I called mine DigitizerAction and made it extend MapAction which Michael Bedward (the original author of the gt-swing module) helpfully provided to save us typing (or pasting) in a lot of boiler plate code. All I have to provide is some code to initialise the action’s icon, name etc and an actionPerformed method to set the tool up to actually do something when the button is clicked.

Tool

Again to save time and effort the Digitizer tool extends CursorTool which extends MapMouseAdapter so we don’t need to worry about how to listen to the mouse’s movements or how to get a real world position from a mouse click. Much of the code is either set up or book keeping. For set up we need to generate a FeatureType for the Features we’ll be building later, and a style so they show up on the map (a more advanced tool might let the user override that default style). The only other thing to take care of in the constructor is setting the cursor to a simple cross hair.

   public Digitizer() {
        SimpleFeatureTypeBuilder b = new SimpleFeatureTypeBuilder();
        b.setName("LineFeature");
        b.add("line", LineString.class);
        SimpleFeatureType TYPE = b.buildFeatureType();
        featureBuilder = new SimpleFeatureBuilder(TYPE);
        geometryFactory = JTSFactoryFinder.getGeometryFactory(JTSFactoryFinder.EMPTY_HINTS);
        style = SLD.createLineStyle(Color.red, 2.0f);

        ImageIcon imgIcon = new ImageIcon(getClass().getResource(ICON_IMAGE));
        cursor = new Cursor(Cursor.CROSSHAIR_CURSOR);
    }

The class also has some fields that we’ll need later lastX and lastY which is the position of the last click, an ArrayList of Coordinates which are the real world positions of the current line, and a list of SimpleFeatures which hold the lines that we have already drawn. We need to keep track of the previous features as the display layer is recreated every time we add a line.

The actual interaction with the user all occurs inside the onMouseClicked method which is called each time a mouse button is clicked.

    public void onMouseClicked(MapMouseEvent e) {

        if (e.getClickCount() > 1) { // was it a double click
            drawTheLine(positions);
            first = true;
        } else { // add a new point
            DirectPosition2D pos = e.getWorldPos();

            positions.add(new Coordinate(pos.x, pos.y));
            // Put a marker at each digitized point
            Graphics graphics = (Graphics2D) ((JComponent) getMapPane()).getGraphics().create();
            int x = e.getX();
            int y = e.getY();
            if (!first) {
                graphics.drawLine(lastX, lastY, x, y);
            }
            first = false;
            lastX = x;
            lastY = y;
            graphics.fillRect(x - 3, y - 3, 6, 6);
        }
    }

Here we first check if the user has clicked twice within the system time limit (so it is a double click), if it is we call the drawTheLine method on the positions list to add the line to the screen, we’ll look at that in a moment. We also reset the first flag to say that the next click (if there is one) is the first in a line.

If the user only clicked once (or this is the first click of a double click) we will go through the else branch, where we get the world position of the click and add that to our list of coordinates. We then grab a graphics from our map pane to draw a temporary mark so the user knows we’re listening and have seen their click, here we need the X and Y pixel coordinates. If this is not the first point in the line we draw a line from the last point (lastX, lastY) to the current point (x, y) and then we draw a point at the current point (x, y). We also make a note of this point for next click to be the start of the line and note that we now have previous point by making first false.

The last remaining step is to draw lines on the map as SimpleFeatures, this is done in drawTheLine where we generate a new LineString using the GeometryFactory, we then add that LineString to the FeatureBuilder and create a new SimpleFeature from it. Note that we leave the id set to null so it generates a new FID for each feature. The feature is then stored in the features list, the existing layer is removed from the map (to prevent us ending up with an ever increasing number of map layers). Then we create a new FeatureLayer using the DataUtilities.collection method to convert our list of SimpleFeatures to a SimpleFeatureCollection and applying the style we made earlier. Finally, we reset the positions list to empty to be ready to store the next line for the user.

Usage

If you want to use this in your own map you can simply do the following:

    mapFrame = new JMapFrame();
    mapFrame.enableToolBar(true);
    JToolBar toolBar = mapFrame.getToolBar();
    DigitizerAction d = new DigitizerAction(mapFrame.getMapPane());
    JButton btn = new JButton(d);
    toolBar.addSeparator();
    toolBar.add(btn);

You can then get the list of features by using d.getFeatures() if you want to provide a save function or to send them to some other store.

Some Open Source Tools I've Found Recently

Sat, 21 Jan 2023 00:00:00 +0000

I use open source tools nearly all the time and I’m always on the look out for new and useful tools to help make my life easier, so in this post I’m going to list some of the ones I’ve found recently.

MailHog

MailHog is an email testing took for developers. That doesn’t sound like much but when you are trying to configure a program to send you email notifications then it can be a life saver especially if you don’t happen to have an open SMTP server (or you are using AWS SES which requires all the addresses to be pre-validated in sandbox mode). You simply run MailHog on your local machine (or add the docker container to your swarm) and then set that machine as the SMTP server and the port to 1025. Then when ever your program sends an email you can check the web page at port 8025 to see the mail that was sent.

MailHog is released under an MIT licence and does the one thing it does very well.

Paperless-ngx

I am a book collector, especially with technical books and since discovering Humble Bundle some years ago it has only got worse. At last count I had over 300 technical e-books and I have read fewer than 20 of them completely (and those are mostly Manning ones that I bought as a paper book and got the e-book for free). This is actually fine as mostly I own the books as reference material to help me solve problems when they come up in the future. The problem is remeasuring which books I own and knowing which one will be helpful to solving the current problem. I was always planning in the back of my mind to write a “simple” python program to read the books and generate an index I could search when I had a problem to solve, but I never got very far with it.

However, I recently came across Paperless-ngx which is a document management system that transforms your physical documents into a searchable online archive so you can keep, well, less paper. So I think the developers think that it is for managing paper documents but I’m using it more like an Electric Monk:

“The Electric Monk was a labour-saving device, like a dishwasher or a video recorder. Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; Electric Monks believed things for you, thus saving you what was becoming an increasingly onerous task, that of believing all the things the world expected you to believe.”

― Douglas Adams, Dirk Gently’s Holistic Detective Agency

Paperless is easily installed using Docker and provides an easy to use web interface. I simply pointed it at my books folder and it (over the next day or so) read all my books for me and created an index so I can search all of them by title and content. It also allows me to tag them (and is slowly learning to suggest tags for new books) so I can limit searches to specific groups of books. It also has a consumption directory that I can save new documents into and it will automatically ingest them into the system. I’m pretty sure it can do all sorts of other clever things but I haven’t fully explored it (or read the manual). Any way I can now buy new e-books with a clear conscience as I know the books will get read now.

Pandoc

I’m easily distracted when writing so I prefer to use vim to do my writing so I don’t spend half the morning playing with the styling of my headings and leave myself 10 minutes to finish the document. However, the management team are very much into nicely formatted documents in word or Google Docs, so I write my documents in markdown. I also keep my daily journal/work log in markdown so it’s easy to cut and paste from that to a report, but to convert that markdown to a format that I can import in Google docs or word (or whatever) I use pandoc which describes it self as your swiss-army knife for documents. Its fantastic and can even generate the right type of wiki markup to allow me to paste formatted text in to what ever weird format Basecamp uses. Sadly, it still can’t match whatever Atlassian use for their wiki editor in Jira but you can’t have everything (though I believe we may have found a Jira plugin that reads markdown).

But even better I can read in a word doc and convert that to markdown (it’s best if the person generating the file knows how styles work but it’s easier to reformat the markdown than the word document.) You can even use it to generate presentations for you.

It can even produce PDF which means that I can print out the recipe for tea directly from the markdown file I store on github (or when I get organised to put a spare laptop/raspberryPi in the kitchen directly display it).

Give it a try next time you get a document in a format you can’t handle.

Moving...

Sun, 04 Dec 2022 00:00:00 +0000

Moving

If you’ve been following me on twitter you may have noticed that I’ve gone a bit quiet there, and possibly other places that you expected to see me like stackexchange and the GeoServer lists.

There are two reasons for this, first I’ve abandoned my twitter account, partly because of Elon Musk and co but mostly because there is now a critical mass of people I know and people I find interesting on Mastodon. So I can now be found tooting at http://fosstodon.org/@ianturton for my open source ramblings and snark, and at http://mastodon.scot/@ianturton for more random social stuff.

That second address gives a clue as to the other reason I’ve been quieter than usual, I’ve moved to Scotland this autumn as well. Many people have asked does that mean I’ve changed jobs or something but no Astun was a mostly remote company when I joined and has become a fully remote one during the pandemic so I realised I could live any where I wanted (subject to the constraints of Brexit) so why did I continue to live in the Tory heartlands of southern England when I disagreed with most of what it stood for.

There were many reasons to move to Scotland, it’s where my wife’s family are, it’s where we met and got married (30 years ago). But also, it has a proportional electoral system for it’s government so there is a chance to finally have people I voted for making decisions. It has higher taxes and better public services and finally, a good chance of getting an EU passport back in my lifetime.

Moving house is a stressful process, especially when you are as uninterested in planning as I am, but from a sudden idea in August, we had sold our house by September and moved out in November. In Scotland we found a house we liked and had completed the purchase 20 days later! Now we’re just waiting for the removal men to have time to deliver our belongings next week and things should settle down by the new year.

So I should be back to having an office with an ergonomic workstation rather than my current laptop on a dining table in an AirBnB that I’m currently using and I’ll be back to full speed coding and writing then.