Bob,
I am processing a job right now , once done i will post results with and with out hyper threading with a job Austin and I have benchmark our systems with.
As far as Ram goes , I have had mix results with a project of around 5000 images using around 22GB of Ram.
I notice high disk usage only with Step 1.
Is there a way we can share email adresses on here so we can send links to the data? I would rather not just post a link on this forum to the data until we are done with some testing but will share with anyone who asks for it.
I have not tested a 1070 or 1080 but do intend on grabbing a 1070 as soon as they are easily available.
I will also test with out hyperthreading and post my results here with it on and off.
Jerry,
I have posted my email above.
Arturo,
I shot you an email with a Google Drive link. I have also included some of the project files there. Let me know what you see in your system. I am currently processing my data set without HT enabled but I am currently overclocked so it won’t be a perfect test but should help determine if HT makes a difference or not. So far my initial processing time is almost identical to the same settings, no overclock and with HT on.
Hey Guys - I am having a serious issue with my HPZ840 with dual Xeon E5 2620 E3 at 2.40 and 128GB RAM. Nvidia K5200 GPU.
I am currently processing 2500 images.
The Loading Point Cloud step has now taken 2 days and 07 hours to load 59%. Obviously something is not right here…
Any guidance please!
Can you give us some details as to what settings you are processing, size of images, UAV used, GPS tagged or not.
What is your end goal with the project? 2500 images is a pretty large project. Maybe send us your settings and what steps you are processing (initial, point cloud, Ortho…) And the detailed options you selected. Maybe just post a screen shot of the processing options window on each tab.
Earlier this week I did a 1400 image project in under 6 hours but didn’t require an Ortho or DSM, just a low density point cloud.
Its a standard eBee survey, nothing out of the ordinary.
Actually the total job is 20,000 images but I have divided it into 10 sections of roughly 1500 - 2500 images each.
I am very concerned that the dual Xeon has some wrong settings. I can see that all of the cores are being used at 100% and yet the Load Point Cloud step has taken 2 days to get to 59%.
As a comparison, I am also processing a project of only 1000 images on my 2012 i7 Mac in Bootcamp (Win 10) and the project is almost finished!
Any Xeon tips would be highly appreciated. Thanks!
What my research finally found is that a single i7 and a gaming graphics card (gtx 970) perform significantly better than any of my Xeon machines. Some projects were 70% faster on the i7. I don’t have any suggestions for setting on a XEON other than try running a single CPU and see what you get maybe. The gaming cards help quite a bit too. If you have a source to upload the pics to I can try processing it this weekend for you and get you the data and the details on how long it took.
Does that mean that the Pix4D code is not written to take advantage of Xeon CPU’s? It doesn’t make sense !
Breaking the project up a little further (1000 shots per section) might be a good plan too…
It utilizes the cores but two CPUs seems to not really speed things up. I actually had a dual xeon with 8 core CPUs at 2.8GHz I think and it wasn’t as fast as my i7 gaming rig.
Feel free to email me at jbalch@mackaysposito.com and I can send you my cell number if you want to call for some help.
My hunch is that it has more to do with the chip set and memory configuration than the CPU as well as the Quadro card opposed to a GeForce card.
Do you think its a software code issue? Dual Xeons with 24 cores/128GB RAM has more data throughput than a single i7.
It doesn’t make sense!
I have written to Pix4D asking for an explanation - Xeon machines are designed for ‘big data’ and hours of number crunching.
All of the cloud based servers that do any form of behind the scenes work for any industry are all Xeon based.
So why this application seems to slow down on Xeon based machines is beyond me!
Whats more annoying is the investment we make into getting faster results actually works slower…
So heres a summary:
Intel Xeon E5 processors feature:
Intel Xeon E5 processors with higher core counts also have increased shared L3 cache. The quad-core Xeon offers 10MB of L3 cache; the 6-core offers 12MB of L3 cache; the 8-core offers 25MB of L3 cache; and the 12-core offers 30MB of L3 cache.
Which processor is right for you? If you run apps that are not highly multithreaded, you will benefit the most from processors with a higher clock speed. Highly multithreaded applications will perform best on processors with a higher core count, even with a slightly lower clock speed.
Well, some of the main benefits of a XEON are in the RAM. It’s ECC so you will get less crashes and less errors. It is great for heavily multithreaded apps but some apps don’t do so well with multiple CPUs.
You are correct that the majority of cloud servers are running on a XEON platform but they have likely built their apps from the ground up to support certain setups for their specific work.
It is odd the speed of rendering on Pix4D with different CPUs. The biggest things you can do are use the fastest CPU you can afford (I’m talking GHz fast not necessarily cores or L3 cache), a good GPU (You have a pretty high end card there!) And make sure to do all of your processing on FAST SSDs. My fast rig has 2x Samsung 950 m.2 SSDs directly attached to my motherboard.
Can you post the settings you are processing on and what you really need to output for? Like do you need the detail of a medium or high density LAS file? How about the 3D mesh or orthomosaic?
I use the standard 3D maps with all file types selected under each tab. Nothing unusual.
Even the Mac Pro now only comes with Xeon options, as an example.
I think as you say the code was written aimed at the prosumer market which tends to use off the shelf machines based on the core range of CPU. Now that Xeon is making its way into the mainstream, i would hazard a guess that a complete rewrite would be required in order to fully utilise all of the Xeon technologies.
Philip,
I had the same problem with a workstation spec’d as follows that I built myself.
Asus Z10PE-D8WS motherboard, Dual xeon E5-2687W V3 processors, 8 x 8gb DDR4 2133 1.2V memory, Intel Solid State 1.2 tb 750 series SSD, Quadro K4200 running displays, Geforces GTX970 for processing.
The system would hang for days at steps 2 and 3 using 100% of the CPU’s. see previous post in this thread.
I fixed it by updating my BIOS and motherboard chip set drivers. Go into your Control Panel/administrative tools/computer management/device manager under system devices do you have any “?” or “!” next to any items? see below pic
I can tell you from experience that a dual xeon system is faster than an I7 system. I do a project every quarter that is 2300-2500 pictures and it will take about 13.5 hours to run thru the 3 steps the first time using the standard 3d map settings. I have a test project 300 photos that I bench systems with and on my xeon workstation it runs about 1.75 hr on a new i7 2.25hr.
One thing I will note is it looks like you are using a quadro graphics card. I have see notes in the past on pix4d website that they were having problems with the drivers on the quadro cards. I have a quadro and geforce on my system. 2 displays are plugged into the quadro, I typically disable it in pix4d and only process with the geforce. I posted a new thread last week on this forum under the hardware topic (post topic Graphics Cards) asking if Pix4d had fixed the driver issue with the quadro cards but have not gotten any replies.
Thanks for the reply. I had to stop the processing today because theres no way after 2 days and 64% I can wait another 3 days!!! Thats ridiculous!
The HP support app notified me to update the BIOS which I have done. One of the notes in the release doc mentioned support for Xeon E5 series.
I went through all of the BIOS settings to make sure there wasn’t some disabled setting, however all looks fine.
Restarted again the project after step 1. Lets see how long it hangs after point cloud generation and before step 3, Load Point Cloud. This is where the hang is. It needs to load the PC in order to start generating the DSM, Ortho etc.
My SSD is a Samsung 850 1TB so that should be fast enough.
The Quadro K5200 GPU is super fast, you can see that in the log whenever it sends a task to it. Blazing speed so I don’t think there are any issues with it. The HP Z workstations don’t support the 8 pin power for consumer cards, only the 6 pin for the pro range or vice versa.
Did you check your system device to see if you had any “!”
I wish I would have updated BIOS and chipset at different times so I would know for sure what fixed my problem. But it sounds exactly the same as yours.
We use an ebee also but have never had a project with 20,000 photos, What kind of project are you doing that required 20,000 photos?