It is not so simple... the application and system configuration files are easy to eliminate:

sudo apt-get purge <package>

(of course you have to know which is the package that contains your application).

Notice however that this does not delete any configuration file or setting that the application could have created in your home folder or in other places.

For the files that the application has created in the user home directories, this is quite application dependent and more complex to do (and often a good headache for doing upgrades). In general, it is almost impossible to find a generic way of doing it; every application can store data where it wants.

Nevertheless, most applications stores settings in directories of files "hidden" in your home directory, of the style ~/.config/application-name/ or ~/.application-name.

You can start by searching where they can be with

find ~ -iname "*application-name*"

and researching from here. (Basically, the above command list all files of directories under your home directory that contains the string "application-name", in a case-insensitive way). It will be a trial and error thing.

After you have found all the directories/files containing options for the application, you can delete them (if you are sure that they are for the application alone --- a backup before deleting things is a good thing to do).

Even so, application can leave traces and configuration files in a lot of places. You have to see it app per app to completely clean every trace of it.