Basically, I have too many computers that I don’t want to buy another one, I would rather pay to use Amazon.
I ran a set of 400 images through their c4.8xlarge through step 1 and 2 and to me for a computer with those stats it was not nearly fast enough. Only marginally faster than my MacBook 2015.
I was watching the CPU and memory graph and most of the time the CPU was around 30% except for a brief time at the start of step 2 it went to 100%. The RAM stayed only about 25gb used out of a total of 296.
I am wondering what am I missing, why isn’t the program running balls out on this server? I’m going to give it one more crack at the p2.16xlarge instance and see if I cant get processing down.
If anyone has a better workflow or some tips that would be great. I would process on the macbook but the osx version is behind and it runs this computer pretty hard for 6 hours straight.
Thanks
|
| |
This article can give you a sense on the speed up you could expect with better hardware. The experiment compares the gain in processing time for a same dataset on different processing machines: https://support.pix4d.com/hc/en-us/articles/115003928846
Another advantage of a large amount of RAM is to simply finish the processing for projects with many images (mostly for step 2).
Some steps cannot be processed in parallel, so that can also explain why the CPU and RAM are not at 100%.
Note that for processing on servers you would need an enterprise license of the software.
Shame I didn’t see this thread earlier…using Amazon EC2 is a total waste of time, at least until you have the new Enterprise Pix4Dengine (I haven’t tested that). The current (before this new Pix4D announcement) flavors of Pix4D simply don’t work worth a darn on server setups.
If you need processing help then just give me a shout as my Intel Core i9 machine with 128GB of RAM and 11GB of VRAM is not being fully utilized right now…and I could use some more paying work to cover this $6,000 beast 
Adam.Jordan@nhiae.com
Todd,
I suspect you won’t see a big improvement with a p2 instance. I think the bottleneck is read writes to block storage. I saw a big difference when I went to provisioned storage. My last test I had IOPs set to 3000. I’m guessing my processing time would improve if I increase this number. The downside is you pay for provisioned iops. I don’t mind paying when I’m actually processing but don’t like paying between jobs. I am far from an aws expert and suspect there are ways to eliminate this cost between jobs.
Let me know if you find the path to blazing processing on aws. Right now my two year old intel i7 6600 with a GeForce 950 is beating the cloud by 30%
Dave