We have a Dell T3610 Xeon based system built up with 128GB of RAM (ECC DDR3, all slots filled), but Pix4D does not seem to be using much of the memory during processing - only 18GB or so is the max I have seen used so far. The baseline job I am processing has 300 36MP images (Sony A7R), and the job was flown for ~80% overlap, and set up for medium point cloud density.
We are getting 100% utilization of CPU (8 cores, 16 threads) and GPU as well (Nvidia GTX-1080), but not the memory.
Running the same job on Trimble Business Center’s Photogrammetry module, ~50GB of 128GB was used. Also, I have run some tests with burn-in software (Prime95) and gotten full utilization of all RAM, so I know that it is working.
Question is, should Pix4D be using more memory than this, if it is available?
Pix4Dmapper will use all the resources available for processing, unless the resources selected in the Resources and Notifications tab of the Processing Options are not the maximum ones. For more information: https://support.pix4d.com/hc/en-us/articles/202557909.
The hardware components are not used in the same way in all processing steps. For example, the RAM is the most important component for step 2, but less important for steps 1 and 2. You can find more information here: https://support.pix4d.com/hc/en-us/articles/202559519.
If the usage of the RAM is not maximum during processing, this means that the project is not demanding enough to use all the RAM available.
We have subsequently confirmed the comments above by Pix4D tech support. In processing a larger job (900+ 36mp images) at higher resolution settings, nearly all of the 128GB of memory was utilized on our system. For smaller jobs (400-500 36mp images) less than half of the memory was used. In even smaller jobs (due to smaller image size, i.e. 300 12mp images) still less memory was used, less than 20GB.
We would add to our discussion that when all the RAM is used, an amount of memory will be reserved on your hard disk (swap) to satisfy the memory needs. Swapping is slower and that is why it should be avoided. Therefore, we would recommend you to have enough RAM for processing, depending on the size of the project you are processing. The software will also try to adjust processing to the RAM available so that swapping does not occur.
I am having a similar issue with processing large datasets in PIX4D. We’ve recently upgraded our Dual-Xenon CPU tower from 32 GB to 256 GB RAM, and don’t see usage higher than 32GB during any processing step. The PIX4D project has nearly 2,000 20 MP pictures. In the Pix4D notifications tab all CPU threads, GPU device and all RAM have been allotted for processing.
Could you upload your .log file and the Quality Report of your project to our OneDrive? I would like to investigate the issue you’re experiencing.
As a note: Different steps use the resources differently, but at the same time, within a step, in some moments some small instructions need to be completed, which cannot be paralleled and therefore it will not use the full resources. More information in Hardware components usage when processing with Pix4D.
I have uploaded the log file, as you have requested. After more testing, I’ve noticed RAM usage sometimes approaches 40 GB. So the utilization is higher than when we had 32GB, but still well short of the total RAM available (256 GB installed).
I can’t speak about the performance of your machine when it had 32GB of RAM since I didn’t see the .log file, but even that amount of memory should be sufficient for projects with ~2000 images. Read more in Computer requirements. RAM is mainly used during Step 2 and Step 3, whereas in Step 2, the CPU is equally essential. In the .log file, I see that during Step 2, the CPU reaches 100% and RAM stays at the stable level. That’s actually expected. From my observations, when RAM reaches high levels of usage, the project very soon ends up crashing.
If you would like to speed up your processing, I would advise you to make a small change in your processing option, which will have a significant impact on the performance. Please give it a try an unselect the Multiscale option in Step 2. Afterwards, we can compare the results and processing time.
Another advice is to remember about updating the NVIDIA drivers to the latest versions. You can update yours using this link. It won’t improve the speed much prevents the software from crashing.
These cookies are necessary for the website to function and cannot be switched off in our systems.
They are usually only set in response to actions made by you which amount to a request for services, such as setting your privacy preferences,
logging in, or filling in forms. These cookies do not store any personally identifiable information.
These cookies allow us to count visits and traffic sources so we can measure and improve the performance of our site.
They help us to know which pages are the most and least popular and see how visitors move around the site.
All information these cookies collect is aggregated and therefore anonymous.
If you do not allow these cookies we will not know when you have visited our site, and will not be able to monitor its performance.
These cookies may be set through our site by our advertising partner (Google).
They may be used by Google to build a profile of your interests and show you relevant adverts on other sites.
They do not directly store personal information but are based on uniquely identifying your browser and internet device.
If you do not allow these cookies, you will experience less targeted advertising.
