Building ghosting/double projection

I am processing a project that ran two orbits around one building, two orbits around a second building, and three orbits around a third structure. One building and the structure are rendering just fine in the point cloud, however the last building is not. There is a double projection of this building at two elevations. I did use a total of 6 GCPs and marked each in at least 8 images but my marks never did trigger the auto-marking function. Other than spending time to add many MTPs is there a way to mitigate this ghosting in the future?

Hi jdannemiller,

Welcome to the community!

Does the building with three orbits also equate to the one with multiple surfaces? Initially, I would suspect there was an error in the image geolocation causing this extra surface. I would see which orbit/images contribute to this extra surface and remove them. Given the limited information, it appears from the screenshot that one orbit would be sufficient. Are you also flying a nadir path as well?

Regards,
Jon

Jon,

Thank you for response. To answer your first question no, the structure with three orbits is a football stadium. The building in the picture was flown using only two orbits. I am attempting to process this building with only two orbits and no nadir to see if I can resolve building corners better than in the model produced using my nadir flights. I have run three nadir missions with a Zenmuse P1 but both Matic and Mapper have modeled building corners and geometries insufficiently. I understand this to be an issue due to the algorithms attempting to assume a base plane (ground for nadir/mapping and unknown so I assume area/point density average for oblique) but the geometries are very important to me for this project. I am processing a model right now with all nadir imagery and all orbits but at the resolution I am seeking it will take some time. I have also tried Pix4DCatch, but I am afraid the results are poor over large building perimeter distances (greater than 200’). What I mean by poor is that the starting points and the ending points do not align. This leaves me with no way to resolve building geometries as I do not know which points I can trust, nor do I have faith in a model (Catch processed in Matic) that does not converge.

Moving forward, I am going to setup a few testing projects to see if I can resolve what is happening here but it is odd given an RTK base was used, the area in question is part of a 68 acre grab but the problem only exists in about 10 of those acres, and only for a single building in the whole environment. I do not think there is an issue with the geolocation as an RTK base was used but hey, stuff does happen.

Again, I appreciate your response but at this point I cannot wait and I am probably going to use the new model with 10 GCPs, 100 MTPs, 7600 images over 10 flights. Lastly, I have been using Mapper for 8 years and I do enjoy the enhanced processing speed of Matic. But, the fidelity and accuracy in convergence of GCPs and MTPs, which happens realtime in Mapper, but after reoptimization in Matic, leaves a lot to be desired.

With all that being said I would really appreciate any guidance you have that would help me resolve building geometries using Matic as it appears to be the future of Pix4D.

Hello,

Thank you for the additional information, however, to answer your last question having a dataset would be most helpful to gain a full understanding to provide guidance. Would you be able to share the images and .p4m file? I can provide a link to a shared drive in private if so.

Regardless, It’s good to hear you are moving towards GCPs as I believe these are your strongest ally. I hear you with regards to RTK as “stuff” can always happen since one needs to rely on reception that can come and go. Saying this, I wonder if you have looked into any PPK workflows? One of their strengths is that you do not require a connection the whole time.

I hear you with regards to PIX4Dcatch, but would also encourage you to think of it as another tool in the toolbox. It could come in handy when you are unable to capture something from the air.

This leaves me with no way to resolve building geometries as I do not know which points I can trust.

GCPs and key MTPs dispersed in the region would be the safest way.

In our tests merging aerial and terrestrial datasets, we have been including smaller targets among the GCPs in regions of interest(building facades, corners, etc.) to help tie the scene together. We have found these along with GCPs to be invaluable. I’m attaching some simple ones I made to share.

Regards.

-Jon

Thank you for the responses.

  1. To the data sharing, I do not think that will be necessary as I believe we are going to be able to get the project out the door using MTPs. The problem with this as a long term solution is it will be very laborious and taxing on overhead return. This solutions works for us for now but I hope aggregating multiple flights of differing geometries becomes more seamless down the road.

  2. To the PPK workflow, this is something we have considered but we use OPUS solutions to correct all of our GCPs in the horizontal and level loops to establish the vertical. The RTK is just used to maintain good position for the drone during flight. So, I do not think PPK would add anything to our workflow in terms of time (vs RTK). But, I may be wrong and if I am I would welcome feedback.

  3. To Pix4D Catch, yes I agree. Right now we are only looking at this product as something that may add to our workflow later, but right now provides unreliable to bad returns when measuring building corners. The results (out of Catch and Matic) give too much weight to the lidar data which is very negatively affected by the poor quality of orientation sensors on board the iPhone. This leads to a full walk around of a building where the face of the wall at the starting point for a scan, not matching up to the same face at the ending point of the scan. Even after adding dozens of MTPs we cannot resolve this lack of convergence issue. An additional problem seems to be that somewhere in Catch or Matic, a bias towards downward facing scanning/pictures occurs. We do not see the same reconstruction density on upward facing scans that we see for downward facing scans. This is a problem as we have tried using Catch to do interior scans of structure interiors for forensic reconstructions. If you are interested in data for the first issue (lack of convergence) I would be willing to share something if you are interested.

  4. Lastly, and this may make me sound like a novice but here goes. Is there a way to get Mapper’s or Matic’s computed Northing/Easting/Elevation or Lat/Long/Elevation for MTPs? If so, we could add an MTP at every corner, verify model reconstruction fidelity, and use the computed coordinates to produce final products on our end. I have been using Mapper for a bit but this is my first trip down the road of sites with many different buildings. Any advice here would also be very welcome.

You’re welcome and thank you for your feedback and data.

I will send you a link to a shared drive in private to upload the dataset you are referring to in point 3. We would like to dig into this to see what’s going on and if we can offer any solutions.

Regarding point 4, I believe I might have what you are looking for. This can be done in PX4Dmapper, and the steps are assuming that the project was already processed with the MTPs (accurately is important) marked.

  • Open the GCP/MTP Manager.
  • Select the desired MTP/s you want the coordinates of.
  • Right-click on any selected Manual Tie Point in the Type column.
  • Select Edit Types in Selected Rows .
  • Select 3D GCP in the dropdown menu.
  • Click the Export GCPs… button.
  • Choose an appropriate name and location for the file, and click OK .

This will export the positions in a .csv file.

I hope this is what you are looking for. I look forward to receiving the dataset to investigate further.

Regards,
-Jon

Thank you this is a perfect solution for coordinates.

Hi @jdannemiller, thanks for sharing your feedback here, it is really appreciated. This is what helps us get better.

Can you please elaborate on this? Are we talking about the fact that in PIX4Dmapper we can see the initial and the triangulated GCP positions but not yet in PIX4Dmatic?

We’re working on adding this in PIX4Dmatic as well. In both cases though, for the new marks to be taken into account, one needs to run the Reoptimize. Your explanations will help to understand if there is some additional work we should do.

Have you considered combining PIX4Dcatch with the viDoc RTK rover? This will give you RTK accuracy for the positions and should solve your worries. More here: https://www.pix4d.com/product/vidoc-rtk-rover

We are working on an “Align & merge” workflow, that should enable you to process different acquisitions in separate groups of images, which can be calibrated with the processing settings that best fit these groups. Then, they can be aligned and merged at the calibration level to have a common calibration for the entire area. Do you think this would solve the issue?

EDIT: we would be interested to look into improving the calibration so that it works from the start. Would you agree to share the project? The one with (and without) MTPs, the one with MTPs will serve as reference for improving the calibration. If you’re OK, let me know and I’ll reach out to you personally to organize the transfer of data. Thanks!

You are most welcome for the feedback. We are more than happy to share what we can, engage as much as we can, to assist in every way we can. So a little about my background. I have recently moved from academia to industry and we are establishing a terrestrial modeling group for surveying, digital twins and inspections. My history with Pix4D products is 6 years using Mapper for modeling disaster regions where global theatre capture was more important than building geometries. Our issues, chronicled above, while muddy and beyond the scope of a Matic only Forum (so I really appreciate the whole Pix4D team addressing them) deal entirely with methods for reconciling corners on the ground and facades orthogonal to the ground plane.

To your first question, Mapper seems to just be better at real-time convergence of GCP marks. Matic, for me, adjusts after two new marks are added for MTPs/GCPs and if the capture attitudes are very dissimilar the points can “walk” in weird directions. This makes it difficult to ascertain which of the two marks/pictures was more problematic. In Mapper the feedback is instantaneous so the problematic capture attitudes are easier to pick out of the mix.

To your second inquiry yes we have considered the Vidoc. But, as this group is being developed, and as we are exploring many different paths, $6k is a little more than we are willing to spend given our current experience with geometric closures using Catch. I think, and please correct me if I am wrong, but our issue is most likely due to accuracy inconsistences with the iPhone IMU. This could lead to the captured pictures and/or the iPhone captured LAS (maybe LAS-like knowing Apple) being at odds with one another. The pitch, roll, yaw measured by the iPhone IMU may just lack the accuracy we need. If so, I am sure the Vidoc IMU could help provide better pitch/roll/yaw measurements. However, as I said above, we are just too far down too many rabbit holes to roll the dice on the Vidoc.

To you last inquiry, I have received an email from Jonathan Dennis and I will be uploading the data for your group to review. The building in the file is a reasonably perfect rectangle (as good as men can make) but the point cloud model in Matic bows on one of the long sides, and the point cloud fails to match up on the opposite long side. We have seen this issue using Catch on other projects, and we have had other issues all together. The biggest other issue is what appears to be a preference given to reconstructing the parts of a scene that can be observed when looking down, instead of the parts that would be seen when looking up. We recently used Catch to investigate if we could build a model of the interior of a residence where we were inspecting the post-installation of an LVL that has begun sagging. All we need to resolve is the ceiling and base of the second floor but it was sparsely created in Matic. The floor, chairs, art on the wall, and knick-knacks on the tables (all things you would observe by looking down) all show up, but the LVL (seen looking up) is not resolved in the tie points, any point cloud, or in the .obj mesh. I will be uploading the files for the rectangular building using the link provided in Jonathan’s email right after I finish this response. If you are interested in the raw captures and the Matic model for the LVL scene please let me know.

Again, thank you all for your assistance.

Thanks again for taking the time to explain and share feedback and sorry for the time it took to get back to you.

I have the impression that what’s missing is a real-time feedback of the triangulated marks, so that you see where the GCP really is while marking, so that you have hints to better mark the next images. This will be added soon, normally in the next preview release.

My experience so far is that there is a “drift” on the position that can create “closure” problems in longer corridors or projects. The viDoc RTK provides a centimeter accurate position for the images, so the drift on the position doesn’t happen anymore. The orientation should be corrected by the photogrammetric processing. Something else that may help, is to have both the LiDAR+RGB images from the iPhones or iPads. That additional data is then also taken into account in the Calibrate step.

Jonathan got in touch with me. I will see if I can get someone from the R&D teams look into that. We may get back to you asking for more data.

Thanks again!