SSH Bastion

The SSH bastion provides a persistent SSH tunnel into the cluster, allowing external database clients to reach internal services without exposing those services directly to the internet.

Overview

Namespace

bastion

Image

linuxserver/openssh-server (Alpine-based)

SSH port

2222 (internal), 30022 (NodePort)

Authentication

Ed25519 public key only

Username

tunnel

Managed by

ArgoCD (ssh-bastion application)

Architecture

                   ┌──────────────────┐
                   │   SQLyog / SSH   │
                   │   Client (local) │
                   └────────┬─────────┘
                            │ SSH (port 30022)
                   ┌────────▼─────────┐
                   │   Cluster Node   │
                   │   (NodePort)     │
                   └────────┬─────────┘
                            │ port 2222
                   ┌────────▼─────────┐
                   │   SSH Bastion    │
                   │   Pod (bastion   │
                   │   namespace)     │
                   └────────┬─────────┘
                            │ TCP forwarding
              ┌─────────────┴──────────────┐
              │                            │
     ┌────────▼─────────┐       ┌──────────▼────────┐
     │  MySQL Router    │       │  Other internal   │
     │  (port 6446 R/W) │       │  services         │
     │  (port 6447 RO)  │       │                   │
     └──────────────────┘       └───────────────────┘

  • SSH client - Connects to any cluster node on NodePort 30022

  • Bastion pod - Lightweight OpenSSH server that accepts key-authenticated tunnel connections

  • Internal services - Accessible via Kubernetes DNS from within the bastion pod

Security design

The bastion minimises attack surface through several layers of hardening:

  • Key-only authentication - Password and keyboard-interactive authentication are disabled

  • Ed25519 keys - Only strong elliptic-curve keys are accepted

  • Tunnel-only access - TCP forwarding is enabled; X11 forwarding, agent forwarding, and SFTP are disabled

  • Single user - Only the tunnel user is permitted to connect

  • Session limits - Maximum 3 authentication attempts, 5 concurrent sessions, and a 30-second login grace period

  • Non-standard port - NodePort 30022 reduces exposure to automated scanning

  • No exposed database ports - Only the SSH port is reachable externally; database ports remain cluster-internal

Authorized keys management

The public key is stored in a Kubernetes Secret (ssh-bastion-authorized-keys) in the bastion namespace. This secret is created manually and annotated with argocd.argoproj.io/compare-options=IgnoreExtraneous so that ArgoCD does not prune it.

To update the authorized key:

kubectl delete secret ssh-bastion-authorized-keys -n bastion
kubectl create secret generic ssh-bastion-authorized-keys \
  --namespace bastion \
  --from-file=authorized_keys=<path-to-public-key>
kubectl annotate secret ssh-bastion-authorized-keys \
  --namespace bastion \
  argocd.argoproj.io/compare-options=IgnoreExtraneous

After updating the secret, restart the bastion pod to pick up the new key:

kubectl rollout restart deployment/ssh-bastion -n bastion

Connecting with SQLyog

SQLyog supports SSH tunneling natively. Configure a new connection with the following settings:

SSH tab

Setting Value

SSH Host

Any cluster node external IP

SSH Port

30022

Username

tunnel

Authentication

Private key (.ppk format)

SQLyog requires keys in PuTTY .ppk format. Convert an OpenSSH key using puttygen:

puttygen <private-key-file> -o <private-key-file>.ppk

MySQL tab

Setting Value

MySQL Host

idealogic-prod.mysql.svc.cluster.local

Port

6446 (read/write) or 6447 (read-only)

Username

Your MySQL username

Password

Your MySQL password

SQLyog establishes the SSH tunnel first, then connects to MySQL through it. All traffic between the client and the cluster is encrypted via SSH.

Connecting from the command line

Establish a tunnel from your local machine:

ssh -N -L 3307:idealogic-prod.mysql.svc.cluster.local:6446 \
  -p 30022 tunnel@<node-ip>

Then connect a local MySQL client to the forwarded port:

mysql -h 127.0.0.1 -P 3307 -u <username> -p

Deployment components

The bastion is deployed via ArgoCD from the ssh-bastion/ directory:

File Purpose

ssh-bastion/deployment.yml

Deployment with init container for key ownership and OpenSSH server container

ssh-bastion/service.yml

NodePort service exposing port 30022

ssh-bastion/sshd-config.yml

ConfigMap with hardened sshd_config

argocd/ssh-bastion.yml

ArgoCD Application definition

Init container

The deployment uses a busybox init container to copy the authorized key from the Secret volume to an emptyDir volume with correct ownership (UID 1000). This is necessary because Kubernetes Secret mounts are owned by root and the OpenSSH server requires the authorized_keys file to be owned by the connecting user.

Troubleshooting

Check pod status

kubectl get pods -n bastion -l app=ssh-bastion

View logs

kubectl logs -n bastion -l app=ssh-bastion

Verify the authorized key is mounted correctly

kubectl exec -n bastion deploy/ssh-bastion -c openssh-server -- ls -la /config/.ssh/authorized_keys

The file must be owned by tunnel (UID 1000) with permissions 600.

Host key changed warning

The bastion pod generates new SSH host keys on each restart. If your SSH client rejects the connection with a host key mismatch warning, remove the old entry:

ssh-keygen -R "[<node-ip>]:30022"

Related resources