Comparing different flight elevation management methods

Good day, I am currently doing a research project whereby I aim to evaluate and account for mapping product (DEM, DTM and Orthoimage) accuracy and precision (Relative and absolute) based on 3 datasets collected using 1. No Terrain following, 2. Using Terrain following and 3. Using 2 staggered flights at constant altitudes. A key factor was to use a site with a steep slope to force a difference in the quality of collected data and see which approach will yield the best results. i.e. this site has a elevation difference of 60 metres between top and bottom.

I used Map pilot pro to do flight planning and based on the GSD provided here for each mission I planned to compare the GCP and CP sigma and RMSE of the different flight expecting the project with a constant GSD to produce the best results overall.

The flight plan parameters are as follows using a Phantom 4 Pro V2:

  • 53m Altitude at takeoff position for a GSD of 1.5cm/pixel
  • 80% Forward and Sidelap
  • UAV speed of 4.2m/s
  • Normal Aligned mission

The site consists of GCPs spread out at about 50m grid intervals and CPs between each GCP formed polygon. All points were surveyed using RTK-GNSS within 20mm precision before applying a confidence interval however, in this example I used the default 20mm)

Here are the quality checks for each project:

  • Terrain Follow: Note, a flight line was missed and replaced with a constant altitude flight line, I tested with and without it and the difference was negligible IMO.

What I currently don’t understand is how the GCP and CP sigma and RMS can be better than the expected precision and accuracy stipulated by PIX4D based on the average GSD for a project. i.e. 1-2 times planimetric and 2-3 times altimetric. Furthermore, one would expect the terrain follow mission to yield the best results and secondly the staggered flight if I’m not mistaken but the results show otherwise.

How would you establish the final relative and absolute accuracies and precisions of all 3 surveys to then compare the results and establish the reasoning for the variation?

E.g. Are the CP results for say the constant altitude flight better than the terrain follow one because the control and tiepoints are matched in significantly more images due to the elevation change?

I know there are explanations available online but I’m struggling to come to a conclusion when establishing what exactly the GCP sigma and RMSE represents in PIX4D? Is it the relative precision and accuracy of the control network? and then the CP sigma and RMSE relate to the absolute accuracy of the model when evaluated against the GCPs?

Moreover, I was having issues when trying to process the flights when I use the ‘as-staked’ precision of the GCPs which were all between 6-17mm. After initial processing the GCPs are added and marked and when I reoptimize the point cloud disappeared.

Also, is it safe to assume that the ‘Theoretical accuracy’ at a tiepoint or GCP/Checkpoint states the relative precision of the point when a ‘free-net adjustment’ is performed?

Lastly, how does the amount of initial 2d matches and 3d densified tiepoints in the dense cloud affect the model accuracy and precision?

Best Regards

Hi Francois,
This quite a project and well thought out. I applaud you. I had to think about this overnight before replying as there is much to take in.

I believe that if you are wanting to test the accuracy of outputs as a function of flight elevation (or GSD) then I do not think this is the best way to go about it. I think you are better served to find a flat area then fly it at 40m, 80m, and 120m. This will keep all other variables constant.

The problem I see with some of your data is that, while the GSD is increasing with altitude, the overlap is also increasing. Thereby, you have induced another variable into your project. I know you stated that overlap was 80% forward and side, but it appears that was set from the takeoff point. So as the hillside falls away the overlap will increase. I can also see this in your data, particularly how you marked your GCPs and CPs. In the flight that did not use terrain following you marked many points with well over 100 images. In some cases well over 120 images. If you compare this with the terrain following I can see that you only marked 20-40 images. This tells me that the overlap is significantly greater in the no terrain following. There is simply more overlap and more images.

I hope this helps.

Hi Mike,

I managed to achieve excellent(expected) results which provided for interesting comparisons for my research when adjusting the GCP precisions to a 95% confidence level, showing how small changes in the GSD can lead to large volumetric residuals when the surfaces are compared over the entire site. Seems the best way to learn how software functions is spending countless hours of trial and error.

Best Regards

I would enjoy seeing your results if you are ok with sharing. You could post here or send it to me in a private message.

Hi Mike, I will share it soon just awaiting evaluation. I have another query though. Today I surveyed a mine pit and stockpile area in bright sunny conditions. I adjusted the camera settings accordingly and the images look good when I view them from the SD card, however when I import them to PIX4D Mapper it seems the brightness is a lot more than the original images? Does Pix4D only preview the images like this or change it when processing? Please see attached screenshot as it is the first time I see this or realize it.

Essentially, the value of each pixel is obtained as a weighted average of the pixels in the original images that correspond to this particular pixel. In addition, color balancing is also being applied. This means that Pix4Dmapper will try to adjust the intensity of the colors in each image so that the images fit better together. The goal of this operation is to produce a more visually pleasing result.

1 Like