Matic is processing 4500 image job slow

Trying to process a 4500 image job that is about 120 acres or so.
My PC specs
Processor: Intel(R) Core™ i9-10900X CPU @ 3.70GHz 3.70 GHz
RAM: 128 GB
Graphics: NVIDIA GeForce RTX 3080 Ti (Total memory 77678 mb)

No problem uploading pictures or GCPS. No problem with the initial calibrate so I can tie points.
Once I try to process densify, mesh, ortho, and dsm with all exports included it takes a handful of hours to reach 60% or so. Never saw the full amount of time because my PC crashes before it finishes.
Trying to process again as I type this and it is barely at 5% in about a 60 min or so time span.

My processing settings after I tied my GCPS
Calibrate: Large scale and corridor; Scalable Standard; Image Scale 1/1; Keypoints Auto at 10000; Internals Confidence Low
Densify: Hardware Accelerated; Image Scale 1/2; Multiscale On; Density Optimal at 3; Noise filter Off; Sky filter Off
Mesh: Aerial; Texture size 8192x8192; deghosting Weak; Sky Mask Off
DSM: Surface smoothing at 2, Interpolation On
Ortho: Deghosting and oblique Off
With all exports active, Quality report, Dense point cloud, mesh, DSM as compression, Ortho as GeoTIFF

I have a 35mm Zenmuse P1
I was also originally saving the project folder to an external device via USB but changed location to my PC to see if it was a USB lag. Still having the same problem regardless of saving it to my PC
Anything I can do to reduce my processing time? Most of our projects are about of this size

Thank you
Richard

Hi @xzamora

May I ask if you had enough disk space?

The minimum requirement of disk space for a project with (2,000-5,000 images at 20 MP) is 100 GB - 200 GB free space.

Best regards,
Rosana (she/her)

Yes I am using a 2TB external to save this data. I have 690GB free space on my (C:) drive and initially tried it through my (C:) drive but it crashed. So I am currently trying it on an external

External usb drive is definitely the bottleneck. Just add another 2tb nvme drive to your workstation. Only cost about $250.

Also, 5% completion in 1 hour isn’t bad. At that rate it will only take 20 hours to process.

How long would you say it should take to just calibrate 4500 images? I feel 3-4 hours is a lot longer than what I see it takes other peoples’ projects to process.

Hi @xzamora

May I ask if one of the 2TB external or the (C:) drive happen to be a SSD?

SSD would speed up the processing much better than HDD.

Also, is the deghosting option enabled? If this is enabled, it increases the processing time.

Best regards,
Rosana (she/her)

For reference, here is the quality report for a project I just finished yesterday.
StTammany_Lakeshore_20220930-quality_report.pdf (22.6 KB)

Yes my external is a sandisk solid state drive 2TB

My deghosting is disabled. My computer crashes on the densify step. I have the noise and sky filter disabled. 1/2 image scale hardware accelerated.

Thanks for the help

Hi xzamora,

I see that most of the time has been spent in densification. Can you please share the log file? Are you also able to comment on, or share the image content? My first suspition is that the image content is dictating the processing time.

Are you running windows 11? I see there is an issue with Windows 11 updates slowing down GeForce GPUs.

Using an external drive is not optimal, but understand why employed. Agree with Andrew with recommendation to install a larger/faster C drive.

Regards,
-Jon

Hey Jonathan,
My log files are too large to share on here. I’m processing 4500 images. My camera is a Zenmuse P1 45mp
I am currently running windows 10pro
Would you be able to recommend a good nvme drive? Currently have a 1TB nvme KIOXIA but only have 600GB free storage left
Thanks for all the help

Thanks for the support Andrew. Currently asking our IT company to order us another nvme drive.
Currently have a 1TB nvme KIOXIA with only 600GB free storage

I would request a 2tb Samsung 980 Pro or 4tb Seagate Firecuda 530 nvme. The Samsung is about $250 and the Seagate is around $700.