First you need to remove all php5 files :

sudo apt-get remove --purge `dpkg -l | grep php | grep -w 5.5 | awk '{print $2}' | xargs`

After that try to search for php 5.4. available resources :

apt-cache policy php5

My result :

php5:

  Installed: (none)

  Candidate: 5.5.9+dfsg-1ubuntu4.4

  Version table:

     5.5.9+dfsg-1ubuntu4.4 0

        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages

        500 http://archive.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages

     5.5.9+dfsg-1ubuntu4 0

        500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

If there is no resources available than we need to add some by following command :

echo "deb http://php53.dotdeb.org stable all" | sudo tee -a /etc/apt/sources.list

and

apt-get update

If there is a problem with signatures like following :

Reading package lists... Done

W: GPG error: http://php53.dotdeb.org stable Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY xxxxxCODExxxx

we need to sign our Deb repository by commands :

gpg --keyserver keys.gnupg.net --recv-key xxxxxCODExxxx

gpg -a --export xxxxxCODExxxx | sudo apt-key add - apt-get update

After that execute :

sudo apt-get update

apt-cache policy php5

you should see last result :

php5:

  Installed: (none)

  Candidate: 5.5.9+dfsg-1ubuntu4.4

  Version table:

     5.5.9+dfsg-1ubuntu4.4 0

        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages

        500 http://archive.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages

     5.5.9+dfsg-1ubuntu4 0

        500 http://archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

     5.4.33-1~dotdeb.1 0

        500 http://php53.dotdeb.org/ stable/all amd64 Packages

and finally install your version by hitting :

apt-get install php5=5.4.33-1~dotdeb.1

Enjoy ;)