Could you post screenshots of the image, GCP and output coordinate system windows with checking the “Advanced Coordinate Options” checkbox in the bottom left of the window as below?
Click Project > Image Properties Editor under Coordinate System click Edit…:
Click Project > GCP/MTP Manager under GCP Coordinate System click Edit…:
Click on Project > Select Output Coordinate System:
This will enable us to compare how the coordinate systems were defined.
If they are the same, another possibility could be how the GCP coordinates were imported. Maybe they were imported as X, Y, Z instead of Y, X, Z. This will depend on your coordinate system.
During which step of processing or after which action do you get the error message?