Connecting to Your Server over SSH: Passwords, Keys, and Common Errors

Log in to your VPS from your local machine with a password or an SSH key, and fix the usual "can't connect" problems

Once you have a Linux server, the first thing to do is log in remotely over SSH. This guide covers the two ways to sign in and how to troubleshoot when it won't connect.

1. Logging in with a password

Open a terminal on your local machine (PowerShell or Windows Terminal on Windows, the built-in Terminal on macOS/Linux) and run:

ssh root@your-server-ip

The first time, you'll be asked to confirm the host fingerprint; type yes and press Enter, then enter your password. The default port is 22. If you or your provider changed it, pass the port with -p:

ssh -p 2222 root@your-server-ip

Password login is convenient but an easy target for brute-force attacks, so switch to key-based login as soon as you can.

Key-based login replaces your password with a public/private key pair. It's both safer and more convenient.

Generate a key pair locally

ssh-keygen -t ed25519 -C "your comment"

Press Enter through the prompts. This creates /.ssh/ided25519 (your private key) and /.ssh/ided25519.pub (your public key). Guard the private key carefully and never share it.

Copy the public key to the server

The easiest way:

ssh-copy-id root@your-server-ip

If ssh-copy-id isn't available (common on Windows), just append the contents of ided25519.pub to the server's /.ssh/authorizedkeys file by hand.

Log in without a password

ssh root@your-server-ip

Once key login works, edit /etc/ssh/sshdconfig on the server, set PasswordAuthentication no, and restart SSH (systemctl restart ssh; the service is named sshd on CentOS/Rocky) to turn password login off entirely.

3. Fixing common errors

  • Connection timed out — usually a network or firewall issue. Double-check the IP, make sure the port is open (cloud security group, ufw, or firewalld), and confirm the server is powered on.
  • Connection refused — the host is reachable but nothing is listening on that port. Verify the port number and that the SSH service is running.
  • Permission denied (publickey,password) — wrong account/password or a misconfigured key. Confirm the username and check permissions in /.ssh on the server (700 for the directory, 600 for authorizedkeys).
  • REMOTE HOST IDENTIFICATION HAS CHANGED — typically the fingerprint changed after a reinstall. Remove the stale local entry and reconnect:
ssh-keygen -R your-server-ip

Summary

Get in with a password first, then set up key login and disable password authentication. When you can't connect, work through it in order: timeout means firewall, refused means port, denied means account or key.