Tutorial: How to download the LIDAR datasets from the UK Environment Agency website

Lidar is a technique for displaying the shape of the ground using pulses of laser light.  The results have been widely used to discover Roman monuments, as they can process them to omit modern buildings, trees, etc.  I have been interested in this ever since I discovered some Lidar images of the seabed showing the submerged ruins of the Roman fort of Walton Castle at Felixstowe.

Most of the United Kingdom has been surveyed using Lidar, and the resulting datasets are now freely available for download on the Environment Agency / DEFRA website.  If you can download them, then you can pull them into a tool like QGIS, and turn the data into images.   But this website is not well organised.  I have never been able to work out how to download anything!

Partly this is because I used my Android mobile much of the time.  Just don’t.  You won’t be able to get it to work.  Instead go to your trusty PC and open your browser.

1. Go to the Defra Home page, and search for LIDAR

Go to https://environment.data.gov.uk/ and put LIDAR in the search box.  You currently (July 2019) get 20 results, which look like this.  (Click on the image for a larger image)

The LIDAR Composite DTM and DSM materials are what you want, taken at various resolutions.  DSM is the raw data.  DTM removes surface objects like trees and houses.

2. Click on LIDAR Composite DTM – 1m

This takes you to a waffle page.  At the bottom are various links:

The “Dataset links” do not work – any of them.  On enquiry I was told that:

You need to open the WMS/WFS links within the GIS software in order for the data to load, please refer to the following FAQ:

https://support.environment.data.gov.uk/hc/en-gb/articles/360000921077-Why-do-I-get-a-Query-Parameters-must-include-Request-error-

Apparently “WMS” is is a way to get the datasets from the GIS tool directly.  Not what I have in mind here at the moment.

The one you want is the “Survey Download” icon, which I have highlighted in red.

3.  Click on Survey Download

This takes you to https://environment.data.gov.uk/DefraDataDownload/?Mode=survey, which after a long pause builds the following inscrutable screen:

Note how, cunningly, the pane marked “Download your data” only refers to *uploading*!

Ignore everything in the grey box under “Select your area”.  I have no idea what it is for, other than to confuse.

The bit you need is the square.  But… NOT YET!  If you hover over it, the mysterious tooltip “Polygon” will appear.  It is, in fact, a tool to draw an area on the map.  We’ll use it in a moment.

4.  First, zoom into the area that you want to look at

This bit is fairly obvious.  Use the “+” icon to zoom, and drag the map around.  Once you get far enough in, a grid will appear with references on it. If you know the reference, you can enter it in the search box, although I notice this sometimes does not work.

I’m using the area off Felixstowe, so I get to this.

Until you are zoomed in, you can’t do anything.  You can only download datasets for small areas, you see.  But this is probably enough.

5.  Draw a polygon on the map of the area for which you want Lidar

Now at last you can click on the “polygon” button.  It turns blue.  Now you can draw.  (This frankly can be pretty tricky too.)

  • Hover over the map at one corner of wherever you want to draw.  A tooltip will come up telling you to click to start.  Do so.  Nothing will seem to happen.
  • Now move the mouse.  A red line will follow you.  Click again for that corner.
  • Repeat until your polygon looks right, then double-click to save.

It will now look like this:

Note my polygon on the map.  But … also note that, cunningly, some extra stuff has appeared underneath the drawing tool, where you were not looking!  And partly off the page – so scroll down.

Now, at last, you have something you can download.  Hit the down arrow underneath “Download your data”.

There will be quite a pause – and then a new menu will appear!

What this lists is the various different types of dataset.  In fact it lists the lot, of all sizes and resolutions.  Whatever you choose, you get a link in blue, which I have highlighted.

The link is to a zip file.  In Chrome, just click on it to download to the Downloads folder; in IE, right-click and choose “Save target as” in the usual way.  Either way you will end up with a Bathy-Coastal-Multibeam-2013-TM33nw.zip file on your PC.

I’m more interested in the DTM 1 meter stuff, so I will redo this for that:

What are these various types of file?  Well who knows?!  I believe I want DTM anyway.

6.  Unzip the dataset

How you use the dataset is a different question, but I will give you what I found out.

First, you need to unzip the dataset.  I use 7Zip on my PC, and right-click, 7-Zip, and extract to folder.  So…

That created a folder Bathy-Coastal-Multibeam-2013-TM33nw in that directory.

I’m more interested in the DTM 1 meter stuff, so I get a download of LIDAR-DTM-1M-TM33nw.zip, and unpack it to a folder LIDAR-DTM-1M-TM33nw.

Inside the new folder are a bunch of .asc files.  These together make up the dataset.

Next, you need a GIS tool to view this stuff.

7. Import into QGIS

I found this very hard to do, but here’s some notes on what I did.  I worked it out based on this tutorial for an older version: http://geophyswithsnuffler.blogspot.com/2015/11/processing-uk-environment-agency-lidar.html

First, I installed the latest version of QGIS from the download site, which for me was 3.4.5.  Look for the “long term stable release” stuff, and ignore the rest.  This installed fine, and created a folder on my desktop, labelled QGIS 3.4, and an icon, “GRASS GIS 7.6.0”.  Now … do NOT try to start that icon.  Instead drag it into the folder, and forget about it.

Next open the folder, and double-click on the QGIS Desktop icon, again ignoring the GRASS thing.  This will open something you can work with.

Next, create a project by Project -> New.  Then do Project -> Save, and choose a name for your QGZ file – I used my own name.

Next, you need to import the dataset.  Raster -> Miscellaneous -> Merge brings up a daft dialog box headed “Merge”.

Click on the “…”, and you get another daft dialog box headed “Multiple Selection”.  Click on “Add”, and browse into the folder LIDAR-DTM-1M-TM33nw.

Select all the files in the folder, and hit “Open”.  They will all appear in the “Multiple Selection” box.

Now hit “OK”.  You’ll be back at the Merge dialog box.

You’ll want to save the resulting .tif file, so under “Merged” there is “Save to temporary file” – hit the “…” next to that and choose “Save to file”, and then pick a name.

Your “Merge” dialog will now look like this:

Don’t twiddle anything else.

Now hit Run, and go and make a cup of coffee.  It takes a while.

When it finishes, it will pop up “Algorithm ‘Merge’ finished”, and look like this:

Hit “close” to get rid of the dialog box.  You now have some results.

You can use the mouse to drag it around, and zoom in.  The results are likely to look… disappointing.

On the left side is a box “Layers”.  If you right-click on “Merged”, and choose “Properties”, you get stuff that you can play with.  Select “Symbology”, and you can change the “Render type”.  You can change it to “Hillshade” (whatever that is), and hit “Apply” and you get more details:

But that’s as far as I could get.

However, it IS more than I knew before.

Bibliography

Some links that I found useful:

  • http://apps.environment-agency.gov.uk/wiyby/151365.aspx – overview of the datasets
  • https://www.gislounge.com/what-is-a-shapefile/ – the datasets are “shape files”
  • https://www.gislounge.com/shapefile-viewers/ – possible viewers
  • https://houseprices.io/lab/lidar/map – Easily the best way to  view Lidar.  Only works on Chrome tho.  Based on the 1m DSM data.  Actually better than anything I got from this!

Share

15 thoughts on “Tutorial: How to download the LIDAR datasets from the UK Environment Agency website

  1. Actually the interface described is quite intuitive for Remote Sensing software. This is how most download pages for RS data work

  2. You did not actually download LiDAR, you downloaded a Digital Terrain Model (DTM) created based on the LiDAR. LiDAR is a collection of points, the returns from the scan. it contains some extra information that the DTM does not include such as reflectance ratio which can help you understand what the laser actually reflected from. Now to create the DTM (also known as DEM, digital elevation model) whoever created made some assumptions, such as which points are ground, something that per the ASPRS guidelines should be included in any downloadable layer, and they used some geostatistical interpolation method such as Kriging or IDW to create the DEM. Since I have never used said dataset I have no idea if they are right or wrong.

    Should you wish to see the actual LiDAR point cloud, the format you should download is LAS or its compressed version LAZ. LiDAR software is expensive, QGIS is not proper LiDAR software but rather GIS software with LiDAR extensions. Merrick MARS costs $4,995 per license, so unless your company can afford it you will not use it. CloudCompare is an open source solution you can install so you can visualize and navigate the point cloud. Be also aware that because of the size of LiDAR datasets displaying them will test our computer’s ability: a general purpose PC that is fine for coding will struggle running LiDAR, a machine with a solid state drive and a lot of RAM is recommended

  3. Thank you very much for this. It is all rather confusing to the layman, as I can witness, and more info is very helpful!

  4. Following on from ikokki’s excellent point, the Point Cloud Library is also really good, if you want to do more advanced things like 3D semantic segmentation. It has some impressive commerical and academic backers and is free, but, you will probably have to compile it yourself and you need to know C++ to use it, though there are very good tutorials. http://pointclouds.org/ Also, unless you have a GPU, you are going to experience a lot of pain trying to do anything meaningful with las/laz formats, though you can certainly extract info from DSM and DTM using QGIS and a CPU.

  5. Can I extract canopy layer from the digital terrain model or maybe from the surface terrain model?

  6. It astounds me that GIS people think this is ‘intuitive’. It’s like they have no idea of how an actual intuitive webpage works. There is nothing intuitive about a box telling you to upload something (no clues as to what) when you go to a page called ‘download your data’. Environment.gov.uk seems to have been put together by a sadistic sociopath with a deep hatred of people who just want to see a dtm.

  7. HI Roger. Well done on getting through this minefield of how to bring up even an image of the ground topography. I’m trying to create my local golf course for a golf simulator (TGC 2019) and trying to get LIDAR data is a nightmare. At least you’ve given me an outline of the ground which may get me further toward my goal.

  8. After using’ layers ‘ the option of contours produces very good results which I can use.

Leave a Reply