Hi Micahel,
I would say there is a scale problem as the area covered with the initial geolocation is larger than the area covered after the adjustment:
Since you are using “All prior” as camera optimization, I would use “All” to let the software change the internal parameters.
I would also covert the PPK coordinates from WGS84 into your NAD83 system. Please have a look at this post: [Desktop] Horizontal grid corrections and transformations
Another thing that you could try is using “Geometrically verified matching”:
Please let me know how it goes after these recommendations. If it still fails, would it be possible to share the data?
Thank you very much.