I have a rather large project, 25,000 images, and I’m wondering if there is any way possible to use a second computer to assist with processing the sub-projects of the project that was created and stored on the first computer.
What is the resolution of your images?
If you split your project into subprojects, you can process them on different computers for step 1. Initial processing and merge them afterwards. However, assuming the resolution of your images is at least 12MP this makes a huge project. In this case, I would rather recommend to process the subprojects separately and not merge afterwards. Making sure that they align by using Ground Control Points (GCPs). For three main reasons: processing will be faster, there is a high risk that the project does not finish due to lack of processing ressources and last but not least, even if you manage to finish the processing the output files will be too large for most 3rd party software, which makes it unusable in most cases. I would keep the size of the subprojects around 2000 to 5000 images max and this with a good hardware.
There is some more information in this article.
12MP
thanks for the advice. can i just copy/paste the sub-project files onto the other computer and then copy/paste back (I wouldn’t have thought so), or should i try to access them and process from the directory where they were created through the network?
I plan on fully producing the individual sub-projects first, then merging so the client gets the single copy it wants regardless of its usability.
To be sure that the files are correctly stored I would access the project through the network or use Project > Save Project As… to save it on the second computer, e.g. if you open it first through the network to save it locally. Then, you can do the same to go back. I’ve seen some issues in the past when using copy/paste.
I don’t know what kind of hardware you have, but I would have a look at this article. For your project I would even recommend a more powerful computer than in the article. Note that if you fully process the subprojects, steps 2 and 3 will be erased once you merge and recomputed. The merging happens only with the outputs from the first step. I can recommend this article about merging. You can process the subprojects fully, but then I would advise to make a copy (Project > Save Project As…) for the merging, so that you do not lose your results.
great, thanks. yeah i figured copy/paste wasn’t going to work smoothly. I had originally copied the images to the C: of the computer i started in, i figured that would be quicker. will be interesting to see if its significantly slower working from a network directory. only one way to find out i guess.
hardware should be alright, no super computer but win 10, 64bit, SSD, quad i7-6700HQ @ 2.60GHz and 32GB RAM. i had planned on saving the sub projects and resaving them under another name before embarking on the individual process completion, then later merging with the original sub project names.
Damien,
You can simply copy/paste, I do it on every project to and from my NAS…just be sure to get everything in the Pix4D folder. But don’t run projects from a network, always copy them local as it will be much faster.
Honestly 32GB of RAM will not be very good in point cloud creation with such a large data set. I am going to start a new thread to address RAM and other hardware differences.
Hi Adam,
thanks for your input. I’m limited to 32GB unfortunately, mapping isn’t my day in day out work, a few times a year only, so I can’t justify a big desktop and thus using my laptops (2x Dell XPS 15 9550) which i’ve upgraded to 32GB.
That’s nice to hear copy/paste works. I had just started saving from the 15 preprocessed subprojects from C: to my NAS (save as in pix4D), but I’ll just copy from NAS back to the second C: as you suggest to save time. i was going to run one subproject from the NAS just to see the difference, as i wasn’t sure if Pix4D temporarily imports the file to local cache, but I won’t bother now.
I’m going to take the safe route and first fully process each of the 15 subprojects individually to get a working product and afterwards try and merge them all and see what happens. I didn’t really like how Pix4D automatically subdivided the whole project, so if the merge doesn’t work well in the end, I might try and manually create individually slightly overlapping projects (with relatively straight borders and single blocks), such that the deliverable is orderly.
Makes complete sense…not all of us need dedicated Pix4D boxes 
Personally I would do 15 projects myself and never think about “sub-projects”. I don’t even think my 128GB of RAM system would properly run them all merged together so you will need to combine the individual projects outside of Pix4D.
Good luck with the weeks of processing…I love to hear other people pushing Pix4D up to and beyond the limits
Well no success so far loading a project on my second laptop.
on C:(1), I used Save As to resave a project (pre-processed on C:(1) previously) on to the NAS
then on C:(2), I tried to load the project from the NAS, but got this message. i selected a directory on the C:(2) and then it asked the same for every single image in the dataset. it may have potentially worked but i’m not going to go through that process for each of the many thousands of images.
I then tried to copy the project (project file + project folder) saved on the NAS to C:(2) and load from there. Got exact same message.
Then i tried to copy the original project from C:(1), paste to NAS, then paste to C:(2), load. Got exact same message.
Then i tried to copy the original project from C:(1), paste to USB, then paste to C:(2), load. Got exact same message.
I’m wondering if this is because i first created one project, then split the project automatically into 15 sub-projects. maybe there are files in the original project file that the sub-projects need, and maybe i have to copy absolutely everything.
Well as long as you copy the p4d file and the entire directory, each method should work by specifying the correct picture folder just once. I have done this at least 100 times without issue. Your guess of the sub-project autogeneration is probably correct…I have never done that as I do my sub-projects manually.
I’ll now try copying all the projects at once… if that doesn’t work I may have wasted some days using the split project function
I was not able to reproduce your issue with the image paths. I did
- split a project into subprojects
- move the subprojects to another disk
- move the original images to another folder
When I open a subproject from the new location I also get the pop-up message, but when selecting the image folder it directly applies it to all images. Is there something I missed in the steps to reproduce the issue?