I finally found a nice tutorial geared at doing just this. I will outline the steps I took as I already had my LAMP stack installed but the full tutorial can be found here.

Note for the new:

In the tutorial, it begins by switching to the root user with:

sudo su

In my case I simply prefixed those commands sudo instead of switching users, so I will be documenting my steps that way.

Begin

Step one: Install the Apache Worker MPM (Multi-Procesing Modules)

sudo apt-get install apache2-mpm-worker

This replaces the prefork I had installed which is the default when installing Apache.

Step 2: Install PHP5 and necessary modules

sudo apt-get install libapache2-mod-fastcgi php5-fpm php5

At this point you may get an error installing 'libapache2-mod-fastcgi':

Reading package lists... Done

Building dependency tree       

Reading state information... Done

Package libapache2-mod-fastcgi is not available, but is referred to by another package.

This may mean that the package is missing, has been obsoleted, or is only available from another source



E: Package 'libapache2-mod-fastcgi' has no installation candidate

This part is not in the above tutorial

To reconcile this, the multiverse repository must be added to the apt sources.

To do this:

sudo nano /etc/apt/sources.list

To which I appended the following lines:

deb http://archive.ubuntu.com/ubuntu precise multiverse

deb http://archive.ubuntu.com/ubuntu precise-updates multiverse

deb http://security.ubuntu.com/ubuntu precise-security multiverse

precise in this case refers to my version of Ubuntu "Precise Pangolin".

So now, save those changes and back to terminal:

sudo apt-get update

and again:

sudo apt-get install libapache2-mod-fastcgi php5-fpm php5

which will (should) now work.

Now enable these Apache modules:

sudo a2enmod actions fastcgi alias

restart apache

sudo service apache2 restart

Step 3: Apache configuration

To make Apache work with PHP-FPM, we need the following configuration:

<IfModule mod_fastcgi.c>

    AddHandler php5-fcgi .php

    Action php5-fcgi /php5-fcgi

    Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi

    FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -host 127.0.0.1:9000 -pass-header Authorization

    <Directory /usr/lib/cgi-bin>

        Require all granted 

    </Directory>

    #directory statement mult be on multiple lines

</IfModule>

Note:

If you're using an older version of Apache (2.3.x or older), leave out the line

   <Directory /usr/lib/cgi-bin> Require all granted </Directory>

You can check your installed version with the command

apache2 -version

You can put it in the global Apache configuration (so it's enabled for all vhosts), for example in /etc/apache2/conf.d/php5-fpm.conf (this file does not exist, so you must create it), or you can place it in each vhost that should use PHP-FPM.

I choose to go the global route, so:

sudo nano /etc/apache2/conf.d/php5-fpm.conf

paste in the code block above, and save, exit.
This new file will be automatically loaded by Apache's default configuration which loads all files in the /etc/apache2/conf.d/ directory.

restart Apache:

sudo service apache2 restart

Now create the following PHP file in the document root /var/www:

sudo nano /var/www/info.php

Add:

<?php phpinfo();

save & exit.

Now we call that file in a browser (e.g. http://your-server-ip/info.php)

Under Server API at the top you should see FPM/FastCGI.

Success!

For more information like how to change PHP-FPM to use a unix socket instead of the default TCP port or how to configure this for individual virtual hosts instead of all of them, see the source tutorial linked at the top.