Typing:

sudo apt-get install apache2 php5 libapache2-mod-php5

will install everything you need and will start the apache server with support for PHP.

To verify that the php module is loaded, type:

a2query -m php5

if not enabled, then load with:

sudo a2enmod php5

and restart apache:

sudo service apache2 restart

Update for php7.x (tested on Ubuntu 16.04, 16.10, 18.04, 18.10)

Thanks to the comments, I update the answer for php7.x.

Install:

sudo apt-get install apache2 php7.x libapache2-mod-php7.x 

Verify:

a2query -m php7.x

Load:

sudo a2enmod php7.x

Restart apache:

sudo service apache2 restart