Here is the complete answer (based off Alex Falappa and Lekensteyn's answers):

It is indeed possible to get nVidia Optimus GPUs to run CUDA on Ubuntu.

Bumblebee is not necessary for CUDA. (more about Bumblebee: Is a NVIDIA GeForce with Optimus Technology supported by Ubuntu?)

However, when you need to show graphical examples using OpenGL, you do need something like Bumblebee for Optimus systems, otherwise you either don't see anything on your display or get the error:

ERROR: Support for necessary OpenGL extensions missing. 

If you need to use a graphics-intensive program such as Blender, Bumblebee is currently a good way to go.

Skip to Installing CUDA if you don't want to install Bumblebee (i.e. if you don't need OpenGL).

Otherwise, read on.

Installing Bumblebee

Follow the instructions at How well do laptops with Nvidia Optimus work?

These are currently kept up-to-date by a Bumblebee developer.

Once you've installed Bumblebee and rebooted, you will be able to choose the graphics card to use when launching programs by using the optirun program.

For example:

optirun blender

To quickly test if all works correctly, use:

optirun glxspheres

You'll see a program with colorful spinning spheres if all's working correctly.

Verify that the GL vendor string in the terminal contains the word nvidia.

If you run only glxspheres, you'll see the vendor string contains the Intel card instead.

When running a CUDA program, you need to install the CUDA toolkit and a nvidia driver. If you intend to compile programs, you also need the SDK. The installers can be found on http://developer.nvidia.com/cuda-downloads, please read the below instructions before borking your Optimus laptop.

Installing CUDA

Driver

I recommend to install the nvidia driver from Ubuntu's package manager. If you install Bumblebee, you don't need to worry about the driver. Otherwise, after installation, disable the nvidia libraries as described on https://askubuntu.com/a/107746/6969. If you don't, you'll lose 3D acceleration and possibly get stuck on a low resolution.

Toolkit

Basically you have to download the installer, make it executable and run it.
- Download the installer. As of Jan 9th, 2013, 5.0.35 is the most recent driver. As I've a 64-bit OS, I use the 64-bit 11.10 Ubuntu package (though I'm running 12.10)
- Make it executable and allow installation to /usr/local/cuda:

chmod +x cudatoolkit_5.0.35_linux_64_ubuntu11.10-1.run

sudo ./cudatoolkit_5.0.35_linux_64_ubuntu11.10-1.run

When the installation message occurs which asks you where to install CUDA, just press Enter to accept the default /usr/local/cuda:

......................................

Enter install path (default /usr/local/cuda, '/cuda' will be appended):

After installation, it'll print some messages that suggest to put the cuda library directory to your library search path:

========================================



* Please make sure your PATH includes /tmp/cuda/cuda/bin

* Please make sure your LD_LIBRARY_PATH

*   for 32-bit Linux distributions includes /tmp/cuda/cuda/lib

*   for 64-bit Linux distributions includes /tmp/cuda/cuda/lib64:/tmp/cuda/cuda/lib

* OR

*   for 32-bit Linux distributions add /tmp/cuda/cuda/lib

*   for 64-bit Linux distributions add /tmp/cuda/cuda/lib64 and /tmp/cuda/cuda/lib

* to /etc/ld.so.conf and run ldconfig as root



* Please read the release notes in /tmp/cuda/cuda/doc/



* To uninstall CUDA, remove the CUDA files in /tmp/cuda/cuda

* Installation Complete

You may skip this step if you want, but then you have to set LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH later when running a program.

SDK

If you want to be able to compile CUDA applications, you can install the SDK in a similar way as described above. Download, make it executable and run it (not as root, e.g. without sudo!).

Using CUDA

CUDA does not need a nvidia-driven X server to work. In that case you can run your random test program like:

LD_LIBRARY_PATH=/usr/lib/nvidia-current:/usr/lib32/nvidia-current:$LD_LIBRARY_PATH someComputallyIntensiveProgram

If you have not added CUDA to your library path, you will need:

LD_LIBRARY_PATH=/usr/lib/nvidia-current:/usr/lib32/nvidia-current:/usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH someComputallyIntensiveProgram

(you can strip the 32-bit paths from it if your program is 64-bit).

If the CUDA program does have something to display using OpenGL, you have to use optirun:

 optirun blender

Or, if you did not have CUDA added to your default path:

LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda/lib:$LD_LIBRARY_PATH optirun blender

Installing a Blender Build

(With Pre-compiled CUDA Kernels)

When you run optirun blender, you may get a message from Blender saying that CUDA kernel compilation failed, and a message in the terminal similar to the following:

Compiling CUDA kernel ...

nvcc warning : Option '--opencc-options (-Xopencc)' is obsolete and ignored, when

targeting compute_20, sm_20, or higher

gcc: error trying to exec 'cc1plus': execvp: No such file or directory

CUDA kernel compilation failed, see console for details.

If you want to use Blender's GPU rendering feature, you may need a Blender build with pre-compiled CUDA kernels. Builds from Blender.org all have pre-compiled CUDA kernels; the ppa:cheleb/blender-svn builds (more information at this question) do not.

To install an official Blender build, simply follow the instructions laid out in this answer.

If you've installed Blender to /usr/lib/blender, you should then be able to run Blender from the terminal and use GPU rendering with:

 optirun '/usr/lib/blender/blender'