On each user you want to have access to the server, run in a terminal:

ssh-keygen

ssh-copy-id <username>@<serverIPorDNSname>

It will ask you where to save it, the default is just fine. It will ask you to enter a passphrase. This is to encrypt the key just in case the computer is stolen or compromised. If you do not want a passphrase, you leave both passphrase prompts blank.

The first command creates a ssh key for the current the user it is run on. The second command ssh's into the server and adds the the current user's key to the server's list of who is allowed to login. Since ssh-copy-id is using ssh, you will need to do this before you disable password logins.

SSH keys are two matching halves. One half is told to everyone, the other is kept secret. Using a mathematical algorithm, the public half can be used to decrypt data encrypted with the private key. If data decrypts successfully with the public key, you know beyond a doubt that it was the private half of the key that encrypted it. So when you login with a ssh key, the client sends a message (encrypted using the private key) that basically says "let me in". The server checks using the public key and if the public key is in the list of allowed computers the login is successful.

ssh-copy-id is (using ssh) copying the public half to the servers list of what keys are allowed to login (this list is stored at /home/<username>/.ssh/authorized_keys2, or in the case of root at /root/.ssh/authorized_keys2). If you already disabled password authentication, you can still add a computer. You just have to copy the public key (at /home/username/.ssh/id_rsa.pub on a client computer) to something like a USB stick and then add the contents of id_rsa.pub to the end of the /home/<username>/.ssh/authorized_keys2 file on the server. (You could do this by running command like cat /media/USB/stick/path/to/id_rsa.pub >>/home/<username>/.ssh/authorized_keys2 on the server.)