I am currently processing a large project (~1,200 acres with 11,510 42mp images) in Pix4DMapper. I obtained a trial license of Pix4DMatic for testing purposes. I am disappointed in the results produced by Matic compared to Mapper.
First, Mapper produced a more accurate model than Matic. I use a WingtraOne PPK UAS, which typically produces ~0.1ft accuracy without any GCPs. I processed the calibration step using both products before marking any GCPs. Initially it appeared both products produced very accurate results. Then I marked 20 of the 40 GCPs and re-optimized both projects. The GCP residual errors were excellent for both projects. Then I visually inspected the 20 GCPs that I didn’t mark to compare to the targets. All 20 points in Mapper were extremely accurate (within 1-2 pixels). In Matic, some (3-5) of the points were much more inaccurate (>10 pixel error).
I decided to proceed to the DSM generation step with each product. There are a few clusters of trees in the project area, which could produce areas of no data in the point cloud. Mapper produced a DSM with no holes, while Matic had several large holes of no data. This will result in holes in the ortho.
Then I checked the vertical accuracy of the DSMs against the 20 GCPs that were not used in the model (check points). The accuracy of the Mapper DSM was less than 0.1ft for all check points. However several of the Matic checkpoints were off by over 1ft vertically.
Overall I think Matic has promise, but I greatly prefer Mapper, even for large projects. Matic seems too much of a black box compared to Mapper. There are very few processing options and virtually no feedback on processing progress. For very large projects, I like to see the progress of each sub-step. I also like the ability to tweak processing options for optimal results. Also having to geotag 11.500 images adds several hours to my workflow, as Matic doesn’t support import of geotags from a text file.
I was also very impressed with Mapper’s ability to handle a project of this size in a single project. I do have plenty of processing resources, so I’m sure that helps. (Dual 16 core 3.0ghz AMD EPYC, 384GB of memory, 2TB NVME, and nVidia Tesla T4).
Hopefully Matic will continue to develop so that it will be a viable replacement to Mapper. But I don’t think it’s there yet.
2 Likes
Hi @Andrew_Milanes thanks for taking the time to share your feedback, it is appreciated and will help us improve Pix4Dmatic for you. Please find my comments and questions below:
- Accuracy
The difference in accuracy is not normal and we’d like to ask you a few more questions to understand where the issue comes from.
Then I marked 20 of the 40 GCPs and re-optimized both projects. The GCP residual errors were excellent for both projects. Then I visually inspected the 20 GCPs that I didn’t mark to compare to the targets. All 20 points in Mapper were extremely accurate (within 1-2 pixels). In Matic, some (3-5) of the points were much more inaccurate (>10 pixel error).
Then I checked the vertical accuracy of the DSMs against the 20 GCPs that were not used in the model (check points). The accuracy of the Mapper DSM was less than 0.1ft for all check points. However several of the Matic checkpoints were off by over 1ft vertically.
What coordinate system did you set for the GCPs? Also, did you edit the GCP coordinate system before or after the calibration was done? If it was after, please also mention the default coordinate system that was set at the beginning, which is based on the images only.
Maybe we did not correctly read the accuracy of the images. Could you please share 2-3 images of the project? That way we can check if they were correctly read.
Mapper produced a DSM with no holes, while Matic had several large holes of no data. This will result in holes in the ortho.
Was the “interpolation” setting enabled in the DSM processing options in Pix4Dmatic? If it was, there should not be any holes in the DSM. If it was and there were still holes, could you please share a screenshot of the point cloud, DSM and Ortho? Ideally, from both Pix4Dmapper and Pix4Dmatic. This will help us visualize the issue.
I’m going to reach out to you by email about this dataset and the possibility of sharing it with us for investigating the issue.
- Processing options
Which are the processing options you’re missing most when comparing to Pix4Dmapper? If you could please explain which ones and in which cases you use them. That way we add the ones that are most useful.
- Feedback on processing progress
This should be available in Q1 or Q2 this year.
- Import of geotags from a text file and checking accuracy of data
This should be available in Q1 this year.
What coordinate system did you set for the GCPs? Also, did you edit the GCP coordinate system before or after the calibration was done? If it was after, please also mention the default coordinate system that was set at the beginning, which is based on the images only.
State Plane Louisiana South NAD83. I imported the GCPs, but did not mark them until after the calibration step. At that point I marked 20 of the 40 GCPs. I visually inspected all 40 GCPs, and the accuracy is what I expected from my Wingtra. Very close. Then I re-optimized the calibration and visually inspected the remaining 20 GCPs that were not marked. Some of them were much less accurate than before marking any GCPs.
Was the “interpolation” setting enabled in the DSM processing options in Pix4Dmatic? If it was, there should not be any holes in the DSM. If it was and there were still holes, could you please share a screenshot of the point cloud, DSM and Ortho? Ideally, from both Pix4Dmapper and Pix4Dmatic. This will help us visualize the issue.
Yes, the interpolation setting was enabled. I will email screenshots to you.
Hello,
We managed to reproduce the issue related to accuracy and have published a fix in version 1.9.1 preview available here: https://www.pix4d.com/download/pix4dmatic
The issue was related to the image geotag accuracies, which were reset to default values whenever one saved, closed, re-opened and then processed something in the project. Note: if you had already processed a calibration and this was not processed again by running Calibration or Reoptimize, there should not be an issue.
The fix is to recreate the project from scratch in the new version 1.9.1 preview. Starting from scratch is important to ensure everything is correctly generated. If you had marked GCPs or Checkpoints in a previous version you can export the marks and import them again in the new version to save time.
As for the holes in the DSM, we have noted the need and will work on fixing this in a future version.
Thanks for reporting the issue and your collaboration on solving it. Please let us know if you still experience an issue with the new version.
Regards,
Pierangelo