Complete Guide: Secure SSH Setup on Your VPS

Setting up a Virtual Private Server (VPS) correctly from the start is crucial for security. This guide walks you through the entire process of connecting to a fresh VPS and creating a secure, non-root user account.

Part 1: Initial SSH Connection to Your VPS

Step 1: Generate SSH Keys on Your Local Machine

If you don’t already have SSH keys, open your terminal and run:

ssh-keygen -t rsa -b 4096

Press Enter to accept the default location (~/.ssh/id_rsa), and optionally set a passphrase for added security.

Step 2: Add Your Public Key to Your VPS Provider

Log into your VPS provider’s control panel
Navigate to the SSH Keys section (usually under Security settings)
Click to add a new SSH key
Copy the content of your public key: cat ~/.ssh/id_rsa.pub
Paste it into the control panel and give it a memorable name

Step 3: Deploy Your VPS

When creating or rebuilding your VPS:

Select your preferred operating system (Ubuntu, Debian, etc.)
Attach your SSH key during setup
Complete the deployment

Step 4: Make Your First Connection

Connect to your server as root:

ssh root@<your-server-ip>

On first login, you may be prompted to:

Enter a password (sent via email by your provider)
Change the root password immediately

Step 5: Fix Host Key Mismatch (If Rebuilding)

If you’ve rebuilt a VPS and see a “REMOTE HOST IDENTIFICATION HAS CHANGED” error, remove the old key:

ssh-keygen -R <your-server-ip>

Then reconnect:

ssh root@<your-server-ip>

Part 2: Creating a Secure Non-Root User

Now that you’re connected as root, it’s time to create a regular user account. You should never perform daily operations as root – it’s a major security risk.

Step 6: Check if Username is Available

Before creating a user, verify the username doesn’t already exist:

id username

Replace username with your desired username. If you see id: 'username': no such user, you’re good to proceed.

Step 7: Create Your New User

Use the adduser command to create a new user with a home directory:

sudo adduser username

You’ll be prompted to:

Set a password
Enter optional information (Full Name, Room Number, etc.) – you can skip these by pressing Enter
Confirm the information

Step 8: Grant Sudo Privileges

Add your new user to the sudo group so they can run administrative commands:

sudo usermod -aG sudo username

Step 9: Set Up SSH Key Authentication

Now configure SSH key access for your new user:

# Switch to the new user
su - username

# Create the .ssh directory
mkdir -p ~/.ssh
chmod 700 ~/.ssh

# Create the authorized_keys file
nano ~/.ssh/authorized_keys

In the nano editor:

Paste your public key (the same content from ~/.ssh/id_rsa.pub on your local machine)
Press Ctrl + X, then Y, then Enter to save

Set the correct permissions:

chmod 600 ~/.ssh/authorized_keys

Exit back to root:

exit

Step 10: Test Your New User Connection

Before disabling root login, test that your new user works. Open a new terminal window on your local machine and connect:

ssh username@<your-server-ip>

If this works successfully, you’re ready for the final security step.

Part 3: Disable Root SSH Login (Critical Security Step)

Why Disabling Root Login is Essential

Allowing root SSH access is one of the most common security vulnerabilities. Here’s why:

Brute Force Target: Attackers always know the root username exists, so they only need to guess the password. With a regular username, they need to guess both the username AND password.
No Accountability: When multiple people have root access, you can’t track who did what on the system.
No Safety Net: Root can execute any command without restriction. A typo or mistake can destroy your entire system. Regular users with sudo require explicit permission for each administrative action.
Attack Surface Reduction: Limiting root access to local console or sudo commands means even if SSH is compromised, attackers don’t immediately have full system control.

Step 11: Disable Root SSH Access

Once you’ve confirmed your new user account works:

sudo nano /etc/ssh/sshd_config

Find the line:

#PermitRootLogin yes

Change it to:

PermitRootLogin no

Save and exit (Ctrl + X, then Y, then Enter).

Step 12: Restart SSH Service

Apply the changes:

sudo systemctl restart sshd

Step 13: Verify the Configuration

From your local machine, try to connect as root:

ssh root@<your-server-ip>

You should see: Permission denied (publickey).

This is exactly what you want! Your new user account should still work perfectly.

Additional Security Recommendations

Change the Default SSH Port

Edit SSH config:

sudo nano /etc/ssh/sshd_config

Change:

#Port 22

To:

Port 2222

Restart SSH:

sudo systemctl restart sshd

Connect using:

ssh -p 2222 username@<your-server-ip>

Enable a Firewall

Install and configure UFW (Uncomplicated Firewall):

sudo apt update
sudo apt install ufw

# Allow SSH (important: do this BEFORE enabling!)
sudo ufw allow 22/tcp
# Or if you changed the port: sudo ufw allow 2222/tcp

# Enable firewall
sudo ufw enable

# Check status
sudo ufw status

Disable Password Authentication

For maximum security, disable password-based SSH login entirely (only allow key-based authentication):

sudo nano /etc/ssh/sshd_config

Set:

PasswordAuthentication no

Restart SSH:

sudo systemctl restart sshd

Summary

You’ve now successfully:

✅ Connected to your VPS using SSH keys
✅ Created a secure non-root user account
✅ Configured sudo privileges
✅ Set up SSH key authentication for the new user
✅ Disabled root SSH login
✅ Significantly improved your server’s security posture

Your VPS is now configured following security best practices. Always use your regular user account for daily operations, and only use sudo when you need administrative privileges.

Remember: Security is an ongoing process. Keep your system updated with sudo apt update && sudo apt upgrade regularly, and monitor your server logs for suspicious activity.