When you run software as another user you're in fact starting a new minimal isolated environment that doesn't carry on some "excessive" variables (among others variables responsible for injecting libraries or setting privileges). Replace your sudo nautilus call with the following - it will carry on user-specific X server settings from the current session:

pkexec env DISPLAY=$DISPLAY XAUTHORITY=$XAUTHORITY nautilus

This is a one time low level solution but it will work on a misconfigured machine. If you want to permanently "fix your sudo" you need to find the issue with your environment configuration and correct it as described in other answers.