Keyboard layouts in Ubuntu are stored in /usr/share/X11/xkb/symbols/. The easiest way to get your desired result is to create a new layout by copying an existing one. Choose the one you are using at moment (you can have a look inside the files to find see the user friendly name of a layout). So for English (US) type

sudo cp /usr/share/X11/xkb/symbols/us /usr/share/X11/xkb/symbols/us_math

into a terminal window. Now you have to edit the key specifications of the number keys. Open the file for editing, for example:

gksudo gedit /usr/share/X11/xkb/symbols/us_math

Search for the layout variant you are using, in the case of English (US) you will find it at the beginning of the file:

name[Group1]= "English (US)";

Change this text to:

name[Group1]= "English (math)";

Below you see the definition of the particular keys. Change (or insert, if you are using a variant that doesn't explicitly define these keys) the entires for keys AE01 to AE10. In our example we would replace

key <AE01> {    [     1,    exclam      ]   };

key <AE02> {    [     2,    at      ]   };

key <AE03> {    [     3,    numbersign  ]   };

key <AE04> {    [     4,    dollar      ]   };

key <AE05> {    [     5,    percent     ]   };

key <AE06> {    [     6,    asciicircum ]   };

key <AE07> {    [     7,    ampersand   ]   };

key <AE08> {    [     8,    asterisk    ]   };

key <AE09> {    [     9,    parenleft   ]   };

key <AE10> {    [     0,    parenright  ]   };

by

key <AE01> {    [     exclam,   1   ]   };

key <AE02> {    [     at,       2   ]   };

key <AE03> {    [     numbersign,   3   ]   };

key <AE04> {    [     dollar,   4   ]   };

key <AE05> {    [     percent,  5   ]   };

key <AE06> {    [     asciicircum,  6   ]   };

key <AE07> {    [     ampersand,    7   ]   };

key <AE08> {    [     asterisk, 8   ]   };

key <AE09> {    [     parenleft,    9   ]   };

key <AE10> {    [     parenright,   0   ]   };

This will give you access to the special keys without using Shift and by Shift-ing you will be able to write numbers if you like to. Now edit anything else you like and save and close the file.
In the last step we will register the new keyboard layout with Gnome so you will be able to select it from the System settings.

Therefore open /usr/share/X11/xkb/rules/endev.xml in the text editor of your choice, f.e.:

gksudo gedit /usr/share/X11/xkb/rules/evdev.xml

Find the text <layoutList> and insert the following description directly after it:

<layout>

  <configItem>

    <name>us_math</name>



    <shortDescription>en_math</shortDescription>

    <description>English (math)</description>

    <languageList>

      <iso639Id>eng</iso639Id>

    </languageList>

  </configItem>

</layout>

After saving the file you have to restart the X server. As Ctrl+Alt+Backspace didn't work on my system I restarted the system and was now able to select the new layout in System Settings... -> Keyboard Layout.