Disabling the popups in the configuration seems not to be possible, as told by coteyr in the comments on this page (https://askubuntu.com/a/218780/19753: "It's important to note that other then removing update-manager-core you can't really stop the popup. You can just delay it till something else runs apt-get update" – coteyr Nov 18 '12 at 9:14).

I also wanted to get rid of the popups and of the underlying program that would consume the resources of my Ubuntu 12.04 system on a weak Toshiba AC100 (ARM) -- https://answers.launchpad.net/ac100/+question/214505/.

So the simplest working solution must be removing the "update-manager" (as I've mentioned in the comments at https://unix.stackexchange.com/a/46315/4319 ):

apt-get remove update-manager

I have had no problems after that because I could still use apt-get or synaptic to do package updates.

Useful comments:

antivirtel: on 16.10 it couldn't be removed, since it will remove other packages, like ubuntu-desktop. A possible solution: remove the binary file (/usr/bin/update-manager) or chmod -x /usr/bin/update-manager, and apt hold package to prevent updates (echo update-manager hold | sudo dpkg --set-selections).

mchid: ubuntu-desktop is a metapackage so you can remove it without removing the actual desktop so that shouldn't be a problem.

Dylan Parry: It's probably better to just remove update-notifier rather than the entire update manager. Remember to run pkill update-notifier after removing it (or reboot), otherwise Ubuntu will show an error in the indicator panel.