Tutorial: How to download the LIDAR datasets from the UK Environment Agency website

Lidar is a technique for displaying the shape of the ground using pulses of laser light.  The results have been widely used to discover Roman monuments, as they can process them to omit modern buildings, trees, etc.  I have been interested in this ever since I discovered some Lidar images of the seabed showing the submerged ruins of the Roman fort of Walton Castle at Felixstowe.

Most of the United Kingdom has been surveyed using Lidar, and the resulting datasets are now freely available for download on the Environment Agency / DEFRA website.  If you can download them, then you can pull them into a tool like QGIS, and turn the data into images.   But this website is not well organised.  I have never been able to work out how to download anything!

Partly this is because I used my Android mobile much of the time.  Just don’t.  You won’t be able to get it to work.  Instead go to your trusty PC and open your browser.

1. Go to the Defra Home page, and search for LIDAR

Go to https://environment.data.gov.uk/ and put LIDAR in the search box.  You currently (July 2019) get 20 results, which look like this.  (Click on the image for a larger image)

The LIDAR Composite DTM and DSM materials are what you want, taken at various resolutions.  DSM is the raw data.  DTM removes surface objects like trees and houses.

2. Click on LIDAR Composite DTM – 1m

This takes you to a waffle page.  At the bottom are various links:

The “Dataset links” do not work – any of them.  On enquiry I was told that:

You need to open the WMS/WFS links within the GIS software in order for the data to load, please refer to the following FAQ:

https://support.environment.data.gov.uk/hc/en-gb/articles/360000921077-Why-do-I-get-a-Query-Parameters-must-include-Request-error-

Apparently “WMS” is is a way to get the datasets from the GIS tool directly.  Not what I have in mind here at the moment.

The one you want is the “Survey Download” icon, which I have highlighted in red.

3.  Click on Survey Download

This takes you to https://environment.data.gov.uk/DefraDataDownload/?Mode=survey, which after a long pause builds the following inscrutable screen:

Note how, cunningly, the pane marked “Download your data” only refers to *uploading*!

Ignore everything in the grey box under “Select your area”.  I have no idea what it is for, other than to confuse.

The bit you need is the square.  But… NOT YET!  If you hover over it, the mysterious tooltip “Polygon” will appear.  It is, in fact, a tool to draw an area on the map.  We’ll use it in a moment.

4.  First, zoom into the area that you want to look at

This bit is fairly obvious.  Use the “+” icon to zoom, and drag the map around.  Once you get far enough in, a grid will appear with references on it. If you know the reference, you can enter it in the search box, although I notice this sometimes does not work.

I’m using the area off Felixstowe, so I get to this.

Until you are zoomed in, you can’t do anything.  You can only download datasets for small areas, you see.  But this is probably enough.

5.  Draw a polygon on the map of the area for which you want Lidar

Now at last you can click on the “polygon” button.  It turns blue.  Now you can draw.  (This frankly can be pretty tricky too.)

  • Hover over the map at one corner of wherever you want to draw.  A tooltip will come up telling you to click to start.  Do so.  Nothing will seem to happen.
  • Now move the mouse.  A red line will follow you.  Click again for that corner.
  • Repeat until your polygon looks right, then double-click to save.

It will now look like this:

Note my polygon on the map.  But … also note that, cunningly, some extra stuff has appeared underneath the drawing tool, where you were not looking!  And partly off the page – so scroll down.

Now, at last, you have something you can download.  Hit the down arrow underneath “Download your data”.

There will be quite a pause – and then a new menu will appear!

What this lists is the various different types of dataset.  In fact it lists the lot, of all sizes and resolutions.  Whatever you choose, you get a link in blue, which I have highlighted.

The link is to a zip file.  In Chrome, just click on it to download to the Downloads folder; in IE, right-click and choose “Save target as” in the usual way.  Either way you will end up with a Bathy-Coastal-Multibeam-2013-TM33nw.zip file on your PC.

I’m more interested in the DTM 1 meter stuff, so I will redo this for that:

What are these various types of file?  Well who knows?!  I believe I want DTM anyway.

6.  Unzip the dataset

How you use the dataset is a different question, but I will give you what I found out.

First, you need to unzip the dataset.  I use 7Zip on my PC, and right-click, 7-Zip, and extract to folder.  So…

That created a folder Bathy-Coastal-Multibeam-2013-TM33nw in that directory.

I’m more interested in the DTM 1 meter stuff, so I get a download of LIDAR-DTM-1M-TM33nw.zip, and unpack it to a folder LIDAR-DTM-1M-TM33nw.

Inside the new folder are a bunch of .asc files.  These together make up the dataset.

Next, you need a GIS tool to view this stuff.

7. Import into QGIS

I found this very hard to do, but here’s some notes on what I did.  I worked it out based on this tutorial for an older version: http://geophyswithsnuffler.blogspot.com/2015/11/processing-uk-environment-agency-lidar.html

First, I installed the latest version of QGIS from the download site, which for me was 3.4.5.  Look for the “long term stable release” stuff, and ignore the rest.  This installed fine, and created a folder on my desktop, labelled QGIS 3.4, and an icon, “GRASS GIS 7.6.0”.  Now … do NOT try to start that icon.  Instead drag it into the folder, and forget about it.

Next open the folder, and double-click on the QGIS Desktop icon, again ignoring the GRASS thing.  This will open something you can work with.

Next, create a project by Project -> New.  Then do Project -> Save, and choose a name for your QGZ file – I used my own name.

Next, you need to import the dataset.  Raster -> Miscellaneous -> Merge brings up a daft dialog box headed “Merge”.

Click on the “…”, and you get another daft dialog box headed “Multiple Selection”.  Click on “Add”, and browse into the folder LIDAR-DTM-1M-TM33nw.

Select all the files in the folder, and hit “Open”.  They will all appear in the “Multiple Selection” box.

Now hit “OK”.  You’ll be back at the Merge dialog box.

You’ll want to save the resulting .tif file, so under “Merged” there is “Save to temporary file” – hit the “…” next to that and choose “Save to file”, and then pick a name.

Your “Merge” dialog will now look like this:

Don’t twiddle anything else.

Now hit Run, and go and make a cup of coffee.  It takes a while.

When it finishes, it will pop up “Algorithm ‘Merge’ finished”, and look like this:

Hit “close” to get rid of the dialog box.  You now have some results.

You can use the mouse to drag it around, and zoom in.  The results are likely to look… disappointing.

On the left side is a box “Layers”.  If you right-click on “Merged”, and choose “Properties”, you get stuff that you can play with.  Select “Symbology”, and you can change the “Render type”.  You can change it to “Hillshade” (whatever that is), and hit “Apply” and you get more details:

But that’s as far as I could get.

However, it IS more than I knew before.

Bibliography

Some links that I found useful:

  • http://apps.environment-agency.gov.uk/wiyby/151365.aspx – overview of the datasets
  • https://www.gislounge.com/what-is-a-shapefile/ – the datasets are “shape files”
  • https://www.gislounge.com/shapefile-viewers/ – possible viewers
  • https://houseprices.io/lab/lidar/map – Easily the best way to  view Lidar.  Only works on Chrome tho.  Based on the 1m DSM data.  Actually better than anything I got from this!

Leave a Reply