Dustin, unless the camera is dead center over an object, that object has to be heighted correctly in a surface model to have an accurate location. The farther off from vertical (nadir) the worse the effect. (You can really see this in tall objects like poles and towers with their apparent leaning over. The top of the object is displaced.)
The fact that the point cloud fits like a glove is testament to what is probably a pretty good field survey, and proof too that all this crazy stuff can work really well.
But it looks like your control points are on the tops of elevated manholes, themselves on an elevated vault, on the side of a slope?
Orthophotos are ‘draped’ over a surface model created from the point cloud after being cleaned of objects to approximate bare earth. If the statement above characterizing the field conditions is accurate, the surface model would have been smoothed of irregularities and objects.
You might be able to go into the orthomosaic seam editor and pick a more nadir photo to use. Check out the planar and ortho toggle.
You might draw (at their correct elevation) a little surface around objects so they make it through the process at their correct elevation.