Securing Openssh logins with SSH Key pairs

Here is my attempt to assemble a non-bullshit guide since the interwebs are so flooded with unnecessary obscurantism concerning this subject. I favor Debian but the general traits are applicable on most Linux distributions.

Let’s begin:

What you have realized a long time ago

  • Password only logins sucks.

Mission Objectives

  • Implement Public key based authentication.
  • Generate 4096 Bit private key.
  • Optionally, Convert your private key to PKCS8 format.
  • Profit!


Public key = Resides in your home directory on the server you want to SSH to.
Private Key = Your Super secret key that you access your server with.
You unlock the key locally with the password
you supplied at the time of generation. It should be hard to steal!
But we can at least take some precautions to make it harder to crack. Still, if it get stolen and you are aware of it, generate new keys anyway.

Create the secret holy divine key file to use when you SSH to the server

Note: Due to wordpress inability to wrap your code or display vertical scrollbar when it’s needed, please make sure you see the whole commands, otherwise manually scroll vertically.

    1. Create a .ssh directory in your home dir (probably exists already):
      mkdir -p $HOME/.ssh

      Make sure you allow no one else access here than yourself:

      chmod 0700 $HOME/.ssh
    2. Create the Key files (preferably avoid running this ON the actual server, since your private key shouldn’t touch your server). Use a strong password.
      ssh-keygen -t rsa -b 4096 -f my_new_keys
    3. Ship the to the server:
       ssh-copy-id -i "user@server"
    4. Now, you can see if your key works (if you use putty, google for puttygen and openssh keys to convert your private key to putty format):
      ssh user@server -i my_new_keys


Optionally, change the format of the private key you just generated to PKCS8 for stronger security.

Make sure your SSH-client can handle PKCS8. The server needs no modification since this is client side.
This will let you input your password to decrypt the key with the password you decided earlier, and encrypt it again using the pkcs8 key format.

openssl pkcs8 -topk8 -v2 des3 -in my_new_keys -passin OLDPASS -out NEWPASS

Confgure SSHD

Now it is time to change some things in your sshd config on your server.
Make sure you have the following set in /etc/sshd/sshd_conf

PasswordAuthentication no
RSAAuthentication yes
PubkeyAuthentication yes

This will force you to only use your private key when you login to your server.

Or specify a user that can only login via ssh using key:

Match User SuperBOFH
PasswordAuthentication no

Or match a whole group:

Match Group sudoers
PasswordAuthentication no

Re-load openssh after changed has been done to the sshd_conf file:

 service ssh reload 

Now keep your private key safe! Preferably on an encrypted usb thumb drive or similar.