I recently changed my sudo password timeout (the amount of time that passes before sudo asks you for your password again). It defaults to 15 minutes; I raised that to an hour with sudo visudo and changing Defaults env_reset,timestamp_timeout=60. You can make sudo never ask for a password again, however, by setting it to -1.

While I have a feeling that every admin under the sun will tell me this is a bad idea, I'm wondering what the specific security risks are. If someone is logged in as me, don't they already have my password? What specific scenario will having a non-infinite password timeout protect me from?

My Ubuntu box runs a web server exposed to the public.