You don't need anything except the binaries. Those should already be copied to another location by make install. The rest can be removed. If you're unsure make a backup before removal. Remove, test if still working, if still working remove the backup.

Note:
As ændrük pointed out: Using checkinstall over make install has the virtue of enabling Ubuntu to track the installation. Later you can revert any changes made using the regular Ubuntu package management.