Bob, I can confirm my set up is running on Windows 7, 10 and server 2012 R2 with no special setting . I would reinstall OS and pix4d to make sure everything is rested and fresh.
Thanks for the info.
I have built several computers for Pix4d applications. I went all out with one and it has not paid out yet. Built a workstation with the following specs.
Asus Z10PE-D8WS motherboard, Dual xeon E5-2687W V3 processors, 8 x 8gb DDR4 2133 1.2V memory, Intel Solid State 1.2 tb 750 series SSD, Quadro K4200 running displays, Geforces GTX970 for processing.
For some reason the system hangs up on step 2 and 3. I have a baseline project that I run to test systems, 321 images. On the dual xeon system it takes 24hrs. On anther system with similar specs, other than it has a Intel i7-5930k and ASUS x99-A/USB 3.1 motherboard, will process the 321 images in 2 1/2 hours.
Like I said the dual Xeon system seams to almost hang at steps 2 and 3.
I have tried many different things with no change. Removed CPU and processed as single CPU, no help. Turned off Hyperthreading, no help. Turned off ECC on memory, no help.
I have contacted Pix4d support but they have been little help.
Any suggestions would be greatly appreciated.
Austin,
What OS are your running and is there a way you can send me that baseline project to compare on my system?
Windows 10 pro 64 bit
I could set up a dropbox folder
Send it on over.
I am running the same OS on my main workstation without any problem.
Whats your email and I will send dropbox link
I’ve got two pet theories yet unproven.
- After CPU, I/O might be much more important than we think. Out of frustration with speed I just built a new lightweight computer with a single AMD eight-core processor (cheap) with two 256gb Samsung 950 drives mounted to the PCI, and a third Samsung 850 through regular SATA. Only 16gb RAM. Cheap build and surprisingly fast.
I’m admitting my own ignorance, but I’m not sure how much we’re learning when one of the biggest variables may be I/O across different, otherwise similar machines. Maybe some of you guys ahead of me in computer tech could think about that and comment.
- Is everyone using jpg? Full level processing, half level processing? Without these being quantified reports of time spent on processing don’t mean very much.
I’ve forgotten more than I remember, but a number of legacy photogrammetric suites performed better with uncompressed tiff than jpg. I still use tiff a lot.
Steven,
I am currently running Austin’s baseline project. I will report with results and specs.
-
I/O is important but with your ssd running on PCI vs your regular Sata 3 port, the ssd is the bottleneck due to the fact the ssd max read and write is around 500MB while Sata 3 is 6Gb/s and PCI is way more then that depending on the port you use(5Gb-80Gb). I have notice a significant speed increase with using a Ram Disk. It is volatile meaning if your computer for some reason turns off or restarts you loose the data on it. I use it to process a project then move it over to a SSD that holds all my projects.
-
I use Full level processing with jpg ( I do not have a system to shoot in RAW) and mostly stay with stock settings unless ticking extra outputs. Most of the time I have found a discrepancy between the time in the quality report and the actual time it took to complete a project. I have since then began recording my projects via bandicam to get an accurate completion time.
Arturo I didn’t explain it well. I am running a cheap computer I built using two Samsung 950 Pro 256bg M.2 SSD into PCIe slots. One for the operating system and one for the project. A third Samsung 850 Pro is conventional SATA for standard files and folders and archiving.
The eight core AMD FX-8370 processor was also cheap. Since I put this one together as an experiment and its not a company machine, I’m overclocking at 4.5Ghz and combined temps look fine so far. There are faster processors out there for sure, but for the money this build was to see if combining inexpensive modern components with modern architecture could get significantly better results.
Steven that sounds great! I wounder if Austin can share his baseline file that I ran to compare times with the different system . Those AMD are cheap for the raw power they give you.
Now we just need network processing to build a AMD render farm.
Please share the results you get and your full system specs.
Steven,
or maybe you can share with me a job you have to use it on both of our systems. I can give you access to my FTP server to move large files as Dropbox becomes a pain when downloading large file sets.
Just an update
I updated motherboard BIOS and chipset driver and things run much faster now. Processes my 320 image baseline project in 1 hour 45 min.
Austin ,
I ran it at around 2 hrs and 20 min. If allowed , I would like to use your project as a baseline to test other systems.
Go ahead I don’t have a problem with that. Please keep me informed with your results.
I wanted to chime in with my first post on here… We have been fighting with how to get data processed as fast as possible. I have been working on a fairly extensive report with data on what systems seem to process things the fastest. The systems I am testing currently are:
Gaming type rig with 4GHz it 6700k Overclocked to 4.5 GHz and 32GB RAM with 2 m.2 SSDs in RAID 0 straight into the motherboard and a GTX 970 card
Server HP GL380 G7 with 2x 2.53GHz XEON 6 core CPUs. K2200 GPU and 88GB RAM with 4x 300GB SAS 10K RAID 0 for performance testing
Dell Precision with a single XEON 3.4GHz 4 core CPU, 32GB RAM, 256GB Samsung 850SSD and Quadro K620 GPU.
So far I am finding the gaming rig is 30-40% faster on all fronts. I will publish the report in it’s own post either this week or next. Our baseline dataset is a project with 18 GCPs and 461 images taken with a Canon S110 from a Sensefly eBee.
This project runs all 3 steps with .LAS outputs and GeoTiff DSM and Orthomosaic with a medium density point cloud in about 2 hours and 15 minutes on the gaming rig compared to 6+ hours on the workstation with a xeon and a similar time on the server with 12 core +hyperthreading.
So far it really feels like the XEON chips aren’t worth it. I have also been testing with a VM with an nvidia GRID GPU to help and am not seeing anything better…
If there is any chance we can all use that same dataset from this forum that would be great! Or I can provide my data for testing purposes. I am dedicatinga decent amout of resources from my team at this to get a good report that the folks at Pix4D can use or publish so everyone fighting with this can have a single place to look.
Thank you very much Jerry.
It seem your result are consistent with Pix4D recommendation for processing nodes.
Do you have an opinions on the new GTX1080 or 1070?
Jerry,
Is there a way to can send me those files so I can process them on my system to compare?
One of the questions of the OP remains unanswered, and Im interested in that as well: does hyperthreading help with pix4d? I would test it myself, but I only have an i5. If anyone with xeon or core i7 could run a test with hyperthreading enabled and disabled, I would love to hear the result.
I see most people have lots of ram. Ram is cheap, so it cant hurt, but I wonder, is pixmapper using that much on large projects? I have 16GB in my system, but I rarely see pixmapper use more than 2GB or so. That is using ~100-200 photo’s from my bebop (14MP jpeg). Maybe if you use higher quality photo’s with many more tie points, it goes up?
Another observations of my testing, is the lack of activity on my disks according to recourse manager; so I have to doubt if IO makes a big difference. Throughout the process, disk utilization is pretty much 0% with read and write bursts that a floppy might manage. Of course, the above caveat applies, as Im using pretty mediocre quality photo’s as my source material, so that may make a difference.
I suspect RAM speed may be a bigger factor. My new skylake system seems to perform noticeably better than my previous rig, and the biggest difference is fast DDR4. I’ll see if I can find time to test with underclocking the memory and seeing how much impact it has.
speaking of which, I think we’d all benefit from some standard benchmark. A project anyone can process and report their times. This should be a smallish project, so most computers could complete it in a few hours at most. Id upload one of my own, but it would be better if it was based on higher quality footage, so if anyone has a project to contribute for this, please share.