The images that are saved on the drone’s SD card are geotagged by DJI. Regarding the Lat./Log. coordinates, the drone’s GPS saves reliable information in the image EXIF. There is no problem from that side. However regarding the altitude there might be some inaccuracies depending on the location when you are mapping.
DJI measures the absolute elevation above sea level using EGM 96 as the reference. However the vertical coordinate is still not fully reliable. Indeed, we made testing in our office here and we found that the vertical coordinate is off by several meters that can reach an error of 100 meters. Some users noticed the same. Note that this is just an offset meaning that the within the model, the accuracy is not affected, only the absolute location.
DJI released more recent drone firmware, but as far as we know, the issue remains. Therefore our developers suspect something wrong in the DJI EXIF regarding the vertical coordinates. As a consequence, we always recommend to process with ground control points (GCPs) in order to fix these uncertainties.
In other words, your GCPs should not affect the reconstruction of the model but should adjust the elevation of the model to correct altitude. If some parts of the model are misaligned (what you call leverage effect I guess), it is usually solved by manually and locally create matches between the images.
In order to improve the calibration, you could add manual tie points (MTPs) to better align the images. The MTPs should be added at the problematic areas where there is some instance little overlap in the model, misaligned groups of points, etc. To know where to add a MTP, you should verify the re-projection of a point. Click different locations in the point model and check on the images (thumbnails in the right sidebar) if the point is re-projected at the correct location on all images.
How to add and mark MTPs in the rayCloud: https://support.pix4d.com/hc/en-us/articles/202560349