New Xeon Computer

PIX4Dmapper PIX4Dmapper Questions/Troubleshooting
1

Hello all,

After a lot of search and even email/call to support(with no answer) , I am asking the community.

Currently my company is in a position to purchase a pix4d workhorse. The main question I have is:

Does pix4d take advantage of cores(28,36,or 44 cores) or high clock speeds? or even a combination of both? 

I know the Xeon 2697 v3 is recommended but is pix4d able to take advantage of let’s say running 2x those in a dual socket configuration ?

We are processing data sets of 2000-6000 pictures and need them done in a fast turn around time.

Thank You!

 

 

 

 

 

2

Arturo, I am using an older gen HP Z800 with dual Xeon 5690 CPU. I have used both single and dual CPU Z800.

Pix4D does indeed take advantage of the 24 core available of the dual CPU on my Z800, as well as the GPU processing of the graphics cards. I have two Quadro. I did have to enable multi-threading in the machine setup. I’ve also enabled the thermal/clock speed function.

System files are on a solid state drive. I have never come close to consuming most of the RAM available, but I have been able to use almost all of the single/dual CPU, along with the GPU during processing.

Short answer is multi-core, multi-processor yes. I think you found the sample build sheets? And there are explanations as to which processes consume what resource. I hit a threshold with RAM useage I don’t understand. And there are so many variables and potential bottlenecks. So I do not want to speak as an authority. But I’ve never thought I had too much CPU.

Steve

 

3

Thank You Steven for the reply.

Exatcly what I was thinking. We are going with dual E5-2687W with 256GB ram and a pcie ssd for extra speed.

4

Arturo, I wanted to follow up with one more comment. I described the system that I have, and what my experience has been, but I’m not at all sure I have the system optimized for the software, or the software is optimized for the system. I had wanted to follow through with the support team on system settings that can be tweaked within the HP Xeon workstations.

I know enough to know that there’s lots I don’t know. I see some other folks have worked on optimizing systems and I might open a thread to see if someone has answered a question I’m just now asking about system architecture.

At least with the Xeon, I suspect one setting in the setup not made correctly can have a tremendous impact on a program’s performance. And much of that depends on the program’s design.

So with my HP workstation and Pix4D I still have questions about such things as:

* Hyper-threading. I’m using multi-core processing and hyper-threading, but I’m not yet sure about hyper-threading.

* Is there a benefit to remaining only on physical cores?

* Interleaving versus non-interleaving memory.

* Numa versus Non-Numa memory.

* Memory ID

* Memory Usage. With two Xeon processors I have yet to utilize more than a quarter of my 96gb of ram. I haven’t determined why.

* GPU processing. I use it, but my knowledge about it is very primitive ‘GPU good’. There is a lot about the way the GPU impacts a system I don’t yet understand. Are there disadvantages given other available computer resources?

 

There’s a lot I don’t know. 

I like the HP system very much and there is a lot of published material from HP on optimizing the station for particular applications. And I’ve seen many vendors publish their own papers on bios and system settings. This isn’t a criticism of the Pix4D guys. I think some of us started using a little program intended for creating small clouds from consumer cameras in a much bigger way than what was originally intended. I might have been one of them:)

So Arturo, please don’t take my observations as coming from an authority.

Steve

 

5

I’d like to heard more on your experiences in the processing pipe.

It’s been a bumpy ride for me…my setup is a Dell m6800 purchased in January last year along with my first Pro rental. Sense then, changes to my capture pipe have not changed (same Phantom 2V+ and Android phone). Over the passed year Pix4D has been working very hard developing their software, releasing frequent updates. When an update is released I install and run my test dataset to see what’s-what.

Sometimes most things are fine, but sometimes there are problems in one or more parts of the processing pipe that can not be worked around and so I fall-back to previous versions and watch this forum and/or wait for the next update.

I’m not smart enough to understand the interaction between hardware, OS and Applications. That along with unpredictable App behaviour from update-to-update makes for a tough time maximizing system hardware and software.

Thanks for sharing you observations and happy processing!

Gary

6

Just a quick one but does anyone think P4D would run on a VM machine. Here at work we have access to Microsoft Azure services and their VM built machines which we could take up and down as we needed on a project by project basis

7

Paul, what specs do the VM have? In theory they should work just depends on the resources each VM is given.

8

Arturo, you can specify the resources and the price per minute that the machine is active goes up. You’re looking at 

20 core, 140GB Ram 1TB SSD and an Intel Xeon® E5-2673 v3 (Haswell) processor, and with Intel Turbo Boost Technology 2.0 can go to 3.2 GHz

€2.35/hr. €1754/month. For comparison Basic VM machines start at 1.5 cent per hour or about €11/month

 

Paul

9

Paul,

 

That should work fine, but be careful as Pix4d User License Agreement has an issue with people using VM unless you have a cloud license from them.

 

With large expense in “cloud” computing we invested 1 time into our set up and never looked back. Bye bye to hourly charges for processing.

I would be willing to charge you less per hour and all you have to do is send me the data.

 

10

Gentlemen,

I just set up my dual Xeon server and installed Pix4d. The software only seems to recognize one of the sockets (6 cores). It is using all of them and wont use the other cores. We are running Windows server 2012 and it sees all the cores. Any suggestions?

 

Matt

11

Matt,

 

Dumb question , is Windows recognizing both cores?

12

Yes Windows OS sees both sockets and all 24 cores (hyper threading is enabled).  The system is an HP DL380 G9, latest BIOS and firmware on all components.  When I go into processing options it sees the cores in one socket (6 with HT disabled and 12 with HT enabled).  My guess is I missed a setting somewhere in the software setup to get it to see both sockets.  My problem is I am not the user and don’t know anything about the software.  Thanks for any help.

 

Bob

13

Bob,

There really isn’t any setting in the installation that addresses the cores. Any cores Windows see , pix4d should see it as well. I am running a 32 core (hyperthreaded) system on Windows 10 with pix4d seeing and using all cores.

14

I’ve used a dual Xeon machine extensively with Pix4D and During the time I used I never saw a mismatch between what cores Windows saw, and what cores Pix4D saw. I will from time to time (ok, obsessively!) watch the Resource Monitor and see how what I’m doing is acting on the machine and see what I can do without. I think the younger guys in my last position thought it was funny the older guy bringing a state of the art machine to its knees.

Anyway, while by no means an ‘expert’ I did learn a few things. At least on the machines I’ve worked with (which must be a generation or more behind yours) in the setup of the computer there were a lot of settings for CPU and memory and how the two interact. There was a distinction between multi-core, and hyper or multi-threading. And then there were virtual cores.

I’ll go out on a limb (make a guess for non-English speakers), that it might have something to do with that. I could be wrong but you could put the machine under load and open up the Resource Manager, under ‘Performance’ or some such in Task Manager, and see each core Windows sees, computing it’s little heart out for you.

Steve

15

P.S. Matt you didn’t miss anything unless the software changed from last week. To be clear, There is no place to specify in the software individual sockets. The installation is as clean and as simple as can be and so is their resource management. Just the simple slider and GPU Yes/No.

16

Matt,

 

What Steven has said about open the Resource Manger and see if all cores are under load is a great idea.

 

Between multi core , multi threading , and virtual core enable, (all bios settings by the way) Pix4d only utilizes what Windows is able to see.

 

We could go a step further with bios settings. What motherboard do you have Matt/Bob?

17

The OS sees all the cores on both sockets.  I have been through the BIOS and all the processor settings are enabled.  When PIX4D is processing, it is hammering all 12 cores on CPU1 in resource manager but everything on CPU0 is just sitting there.  One thing that is different is that I am running Server 2012R2 on the box instead of a desktop OS. I’ll do some more digging and post my findings when I have them.  Thanks for the input so far.

18

I think I might put the system under load with a different program or benchmarking app, then take a look at the cores. That could point you at the system if you see the same thing.

19

I ran Passmark software on the box and watched performance monitor while it ran the CPU test.  It maxed out all 24 cores during the test.  This leads me to believe that BIOS settings are correct for the OS installed.  I have another, older, box that I am going to try Windows 7 or 10 on and see what happens.  Thanks.

 

Bob

20

Bob, Windows 7 64-bit I can confirm ran Pix4D flawlessly on an HPZ800 with dual Xeon 5690.