How good is the RELATIVE accuracy of PIX4D outputs WITHOUT GCPs?

First of all, I don’t think this question is strictly related to Mapper, it’s more of a general photogrammetry question, but this seems to be one of the more popular sections of the forum with lots of knowledgeable users so I’m hoping I can get some insightful responses here.

Secondly, I am aware that GCPs are required to achieve a high level of absolute accuracy, and for my employers survey projects, we always use GCPs. I have also gotten the impression from my online research that GCPs help with relative accuracy as well.

However

My personal experience after processing 30+ survey projects in PIX4Dcloud Advanced and poring over each Quality Report leads me to believe that the gains in relative accuracy due to GCPs are minimal. . My conclusion could just be due to me misunderstanding the Quality Reports, which is why I’m asking this question.

My company is hoping to start using PIX4Dinspect to create digital twins of cell towers. We have a client who already has a partnership with another photogrammetry processing service (let’s call them “Acme”). Sales people from Acme have told me that GCPs (specifically, Aeropoints) are a part of their best practices for achieving highly accurate digital twins. From my experience and (admittedly limited) knowledge, this seems highly unnecessary given what kind of information is actually being extracted from the digital twins: the client has no need for accurate geolocation (in other words, absolute accuracy is not a concern), and from what I’ve seen, the relative accuracy of PIX4D is pretty good even without GCPs (again, I could be wrong - see next paragraph where I explain my reasoning). The client is really only concerned with being able to take measurements of structural members (I don’t really have a good idea of the exact tolerances they are working with, sorry! - I should be able to get a better idea once we have some test datasets processed). Furthermore, PIX4Dinspect does not seem to even accept GCPs as an input, so if the images alone are good enough for PIX4D, why is Acme claiming GCPs are necessary?

We don’t have any test tower datasets yet (soon), so for now I am wondering if I can gather some indicator of relative accuracy from my already-processed survey projects and their Quality Reports. I have been using the “mean RMS error” under the georeferencing section on the first page of the Quality Reports as an indicator of the relative accuracy. I’ve found that the mean RMS error on our previous survey projects is usually well below the GSD, which I assume would make the GSD the bottleneck in measurement accuracy. Is it a mistake to use this “mean RMS error number” as an indicator of relative accuracy? If so, what in the Quality Reports is a better indicator (if any)?

I have read these pages (1) (2) concerning GCP RMS error but I didn’t fully understand them. This post was a little bit easier for me to understand, but I’m still unsure if it confirms my assumption about the mean RMS error being an indicator of relative accuracy. I also encountered this post from Adam_Jordan_P.E in which he seems to assert that a relative accuracy near to 1x GSD can be achieved without any GCPs or scale constraint. This seems to agree with my findings, but I just want to make sure I’m reading these quality reports correctly! The correct scaling/relative accuracy of the digital twins are my chief concern, and if we can achieve a reasonably high degree of relative accuracy without GCPs, that means we don’t have to purchase additional equipment for our pilots!

Thanks in advance!

EDIT: here is a screenshot of the spreadsheet where I have compiled the GSDs and mean RMS errors of all of our survey projects so far. This is where I am drawing my conclusions from. With the exception of one project where we had to fly at 400ft AGL (hence the 0.10ft/pixel GSD) and had significant GPS issues while setting GCPs (the most probable cause for the 0.157 mean RMS error, I think?), most of the projects had a mean RMS error of 1/10th the GSD.

Hi cmccurdy,

The RMS error under the quality check displays the difference of GCP positions between what is measured and what is computed. It is an indicator of how well PIX4Dmapper fitted the model to the known GCPs. So a low RMS is an indication that you generally have a high quality dataset (high image overlap, high visual content, etc) and a reliable reconstruction relative to your GCPs. That is, it is an indication of the relative error as a function of the GCPs. It also assumes that the GCPs have a high absolute accuracy and were acquired with a high precision GNSS device.

But keep in mind that these relative errors are not evenly distributed. To independently assess the accuracy of a given point or the overall model we recommend using checkpoints. This will independently verify a known point against what is computed. I would suggest adding a couple checkpoints in your upcoming projects to understand how well PIX4Dmapper is truly performing.

Your assumption that the GSD is the bottleneck in measurement accuracy is correct assuming you have a quality dataset. More specifically, accuracy of the outputs should be close to 1-2 pixels (GSD) horizontally (XY) and 2-3 pixels (GSD) vertically (Z). If you want to keep the accuracy (relative and absolute) as high as possible then you will want to keep your GSD as low as possible. Using a higher resolution camera or flying lower will help.

After all this, I do agree with Adam_Jordan’s assessment that high relative accuracy can be achieved without GCPs. While adding GCPs can help correct potential problems, it isn’t necessary if the dataset is good. Many of PIX4D’s products do not even have the option to add GCPs because of this. Notably, PIX4Dfields (agriculture), PIX4Dreact (public safety), and PIX4Dinspect (telecom).

2 Likes

Thank you for the detailed response, Mike! Much appreciated!