How I Set Up a Production-Grade SSH Tunnel Between My App Server and Data Server

This morning, I had to solve a real infrastructure problem.

My application server needed to connect to PostgreSQL and RabbitMQ running on another server. The easy option was to expose those service ports publicly, but that is not a good production practice.

So I used SSH tunneling.

At first, the tunnel worked from the host server, but my Docker containers could not connect. The main lesson was simple but important: inside Docker, 127.0.0.1 is not the host machine. I had to bind the tunnel to the Docker bridge gateway IP and then update the application connection string.

After fixing the issue and making the tunnel run with systemd, auto-restart, restricted SSH keys, and proper firewall rules, I documented the full process.

This post is the practical checklist I wish I had before starting.

1. What Problem This Solves

Sometimes your application server needs to connect to a database, RabbitMQ, Redis, or another private service running on a different server.

Instead of exposing that database/service publicly to the internet, you can create a secure SSH tunnel.

Example:

App server: runs Docker containers for API/backend/frontend.
Data server: runs PostgreSQL, RabbitMQ, Redis, etc.
Goal: containers on the app server can connect securely to services on the data server.
Best practice: services remain private; only SSH is exposed.

2. Core Concept

SSH local forwarding syntax:

ssh -N -L LOCAL_BIND_IP:LOCAL_PORT:REMOTE_HOST:REMOTE_PORT user@remote-server

Meaning:

Connect to remote-server by SSH.
Open LOCAL_BIND_IP:LOCAL_PORT on the app server.
Forward anything received there to REMOTE_HOST:REMOTE_PORT from the remote server's perspective.

Example:

ssh -N -L 127.0.0.1:5433:127.0.0.1:5432 datauser@10.0.0.5

Then local apps can connect to:

127.0.0.1:5433

and they will reach PostgreSQL on the data server:

127.0.0.1:5432

3. Best-Practice Architecture

Recommended production setup:

[Docker container on App Server]
        |
        | connects to Docker bridge gateway IP, for example 172.18.0.1:5433
        v
[SSH tunnel listening on App Server bridge IP]
        |
        | encrypted SSH connection
        v
[Data Server SSH]
        |
        v
[Private PostgreSQL/RabbitMQ/Redis on 127.0.0.1]

Important idea:

Host apps can use 127.0.0.1.
Docker containers cannot use host 127.0.0.1 to reach the host tunnel.
Containers usually need the Docker bridge gateway IP, for example:
- 172.17.0.1
- 172.18.0.1
- 172.19.0.1

4. Rules You Should Always Follow

Do not expose database ports publicly.
Do not bind tunnels to 0.0.0.0 unless absolutely required.
Prefer binding to:
- 127.0.0.1 for host-only access
- Docker bridge gateway IP for container access
Use a dedicated SSH key only for the tunnel.
Restrict that SSH key on the target server.
Run the tunnel using systemd, not manually in terminal.
Use automatic restart.
Use SSH keepalive options.
Verify from both host and Docker container.
Add firewall rules only for required interfaces and ports.

5. Discovery Checklist Before Creating a Tunnel

Run these on the application/source server:

whoami
hostname
ip -4 addr show docker0 2>/dev/null
ip -4 addr | grep -E 'docker0|br-'
docker network ls
systemctl list-units --type=service --all | grep -i tunnel || true
ss -tlnp | grep -E ':(5432|5433|5672|6379|3306)' || true

Find Docker network gateway:

docker network inspect bridge --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}'

For a custom Docker network:

docker network inspect YOUR_NETWORK_NAME --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}'

Run these on the data/target server:

whoami
hostname
docker ps --format '{{.Names}} {{.Networks}} {{.Ports}}'
ss -tlnp | grep -E ':(5432|5672|6379|3306)' || true

Check if target service is reachable locally on data server:

timeout 3 bash -c '</dev/tcp/127.0.0.1/5432' && echo postgres_ok || echo postgres_fail

6. Create a Dedicated SSH Key on the App Server

Replace:

APPUSER with the Linux user that will run the tunnel.
data_tunnel_key with a meaningful name.

sudo -u APPUSER mkdir -p /home/APPUSER/.ssh
sudo chmod 700 /home/APPUSER/.ssh

sudo -u APPUSER ssh-keygen -t ed25519 \
  -f /home/APPUSER/.ssh/data_tunnel_key \
  -C "tunnel@app-to-data" \
  -N ""

sudo chmod 600 /home/APPUSER/.ssh/data_tunnel_key
sudo chmod 644 /home/APPUSER/.ssh/data_tunnel_key.pub

cat /home/APPUSER/.ssh/data_tunnel_key.pub

Copy the public key output. You will place it on the data server.

7. Restrict the Key on the Data Server

On the data server, edit:

nano ~/.ssh/authorized_keys

Add the public key with restrictions.

Example for PostgreSQL 5432 and RabbitMQ 5672:

command="/bin/false",restrict,port-forwarding,permitopen="127.0.0.1:5432",permitopen="127.0.0.1:5672" ssh-ed25519 AAAA... tunnel@app-to-data

What this does:

command="/bin/false" prevents shell login.
restrict disables most SSH features.
port-forwarding allows only port forwarding.
permitopen allows forwarding only to specific destination host:port.

This is very important. Without this, if the key is stolen, it may allow normal SSH access.

8. Test the Tunnel Manually First

From the app server:

ssh \
  -o ExitOnForwardFailure=yes \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  -i /home/APPUSER/.ssh/data_tunnel_key \
  -N \
  -L 127.0.0.1:5433:127.0.0.1:5432 \
  DATAUSER@DATA_SERVER_IP

Open another terminal and test:

ss -tlnp | grep 5433
timeout 3 bash -c '</dev/tcp/127.0.0.1/5433' && echo tunnel_ok || echo tunnel_fail

If it works, stop the manual SSH command with Ctrl+C.

9. Docker Container Access

If your app runs inside Docker, do not use 127.0.0.1 inside the container.

Inside a container:

127.0.0.1 means the container itself, not the host server.

So bind the tunnel to the Docker bridge gateway IP.

Example:

-L 172.18.0.1:5433:127.0.0.1:5432

Then container connection string should use:

Host=172.18.0.1;Port=5433

or:

postgres://user:password@172.18.0.1:5433/dbname

10. Create a systemd Service

Create:

sudo nano /etc/systemd/system/app-data-tunnel.service

Example service:

[Unit]
Description=Secure SSH tunnel from app server to data server
After=network-online.target docker.service
Wants=network-online.target

[Service]
User=APPUSER
ExecStart=/usr/bin/ssh \
  -o ServerAliveInterval=30 \
  -o ServerAliveCountMax=3 \
  -o ExitOnForwardFailure=yes \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  -i /home/APPUSER/.ssh/data_tunnel_key \
  -NT \
  -L 127.0.0.1:5433:127.0.0.1:5432 \
  -L 172.18.0.1:5433:127.0.0.1:5432 \
  DATAUSER@DATA_SERVER_IP
Restart=always
RestartSec=10
StartLimitIntervalSec=60
StartLimitBurst=10

[Install]
WantedBy=multi-user.target

Replace:

APPUSER
DATAUSER
DATA_SERVER_IP
172.18.0.1 with your Docker bridge gateway
ports as needed

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now app-data-tunnel.service
sudo systemctl status app-data-tunnel.service --no-pager --full

Check logs:

journalctl -u app-data-tunnel.service -n 100 --no-pager

11. Firewall Rules

If UFW is enabled and Docker containers cannot reach the tunnel, allow only the Docker bridge interface and only the required port.

Find interface names:

ip link | grep -E 'docker0|br-'

Allow traffic from Docker bridge to the tunnel port:

sudo ufw allow in on docker0 to any port 5433 proto tcp comment "Docker to PostgreSQL SSH tunnel"

For a custom bridge interface:

sudo ufw allow in on br-xxxxxxxxxxxx to any port 5433 proto tcp comment "Docker network to PostgreSQL SSH tunnel"

Check:

sudo ufw status numbered

Avoid:

sudo ufw allow 5433/tcp

That can expose the port more broadly than needed.

12. Verify from Host

systemctl is-active app-data-tunnel.service
ss -tlnp | grep 5433
timeout 3 bash -c '</dev/tcp/127.0.0.1/5433' && echo host_ok || echo host_fail

If PostgreSQL client exists:

psql -h 127.0.0.1 -p 5433 -U DBUSER -d DBNAME -c 'select now();'

13. Verify from Docker Container

Use the same Docker network as your app.

docker run --rm --network YOUR_APP_NETWORK busybox:1.36 sh -c '
  nc -zvw3 172.18.0.1 5433 && echo container_ok || echo container_fail
'

If this fails but host test works, common causes are:

Tunnel is not bound to Docker bridge IP.
Wrong Docker gateway IP.
UFW/firewall blocks bridge traffic.
Container is on a different Docker network.

14. Update Application Connection Strings

For host app:

Host=127.0.0.1;Port=5433

For Docker app:

Host=172.18.0.1;Port=5433

Example Docker Compose environment:

services:
  api:
    environment:
      ConnectionStrings__Default: "Host=172.18.0.1;Port=5433;Database=mydb;Username=myuser;Password=mypassword"

After updating:

docker compose up -d
 docker compose logs api --tail=100

15. Security Verification

Confirm the tunnel key cannot open a shell:

ssh \
  -i /home/APPUSER/.ssh/data_tunnel_key \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  DATAUSER@DATA_SERVER_IP 'whoami' || echo "shell_blocked_ok"

Try forwarding a forbidden port; it should fail if permitopen works:

ssh \
  -i /home/APPUSER/.ssh/data_tunnel_key \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  -o ExitOnForwardFailure=yes \
  -NT \
  -L 127.0.0.1:29999:127.0.0.1:22 \
  DATAUSER@DATA_SERVER_IP

If this succeeds, your authorized_keys restriction is wrong.

16. Troubleshooting

Problem: systemd service fails immediately

Check:

journalctl -u app-data-tunnel.service -n 100 --no-pager

Common causes:

Wrong SSH key path.
Wrong username or server IP.
Target server does not accept the key.
Known_hosts prompt blocks connection.
Local port already in use.
permitopen does not match destination port.

Problem: local port already in use

Check:

ss -tlnp | grep 5433

Use a different local port or stop the conflicting service.

Problem: host can connect but container cannot

Check:

docker network inspect YOUR_APP_NETWORK --format '{{range .IPAM.Config}}{{.Gateway}}{{end}}'
ss -tlnp | grep 5433
sudo ufw status numbered

Make sure the tunnel listens on the Docker gateway IP, not only 127.0.0.1.

Problem: tunnel works, app still fails

Check:

App connection string.
Docker environment variables actually applied.
Container DNS/network.
Database credentials.
Database allows local connection on data server.

17. Production Checklist

Before saying the job is complete, verify all of these:

[ ] Dedicated SSH key created.
[ ] Public key added to target authorized_keys.
[ ] Public key is restricted with command="/bin/false", restrict, port-forwarding, and permitopen.
[ ] Tunnel service created under systemd.
[ ] Tunnel starts after network and Docker.
[ ] Restart=always is configured.
[ ] ExitOnForwardFailure=yes is configured.
[ ] Host test passes.
[ ] Docker container test passes.
[ ] Firewall only allows required interface/port.
[ ] Application connection string uses correct host and port.
[ ] Application logs confirm successful connection.
[ ] No tunnel is bound to 0.0.0.0 unless intentionally approved.

18. Recommended Template

Use this as your base systemd template:

[Unit]
Description=Secure SSH tunnel: APP to DATA
After=network-online.target docker.service
Wants=network-online.target

[Service]
User=APPUSER
ExecStart=/usr/bin/ssh \
  -o ServerAliveInterval=30 \
  -o ServerAliveCountMax=3 \
  -o ExitOnForwardFailure=yes \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  -i /home/APPUSER/.ssh/data_tunnel_key \
  -NT \
  -L 127.0.0.1:LOCAL_PORT:127.0.0.1:REMOTE_PORT \
  -L DOCKER_GATEWAY_IP:LOCAL_PORT:127.0.0.1:REMOTE_PORT \
  DATAUSER@DATA_SERVER_IP
Restart=always
RestartSec=10
StartLimitIntervalSec=60
StartLimitBurst=10

[Install]
WantedBy=multi-user.target

19. Simple Example

Scenario:

App server user: appuser
Data server user: datauser
Data server IP: 10.10.10.20
PostgreSQL on data server: 127.0.0.1:5432
Local tunnel port: 5433
Docker gateway: 172.18.0.1

Service:

[Unit]
Description=PostgreSQL SSH tunnel for Docker app
After=network-online.target docker.service
Wants=network-online.target

[Service]
User=appuser
ExecStart=/usr/bin/ssh \
  -o ServerAliveInterval=30 \
  -o ServerAliveCountMax=3 \
  -o ExitOnForwardFailure=yes \
  -o BatchMode=yes \
  -o IdentitiesOnly=yes \
  -i /home/appuser/.ssh/data_tunnel_key \
  -NT \
  -L 127.0.0.1:5433:127.0.0.1:5432 \
  -L 172.18.0.1:5433:127.0.0.1:5432 \
  datauser@10.10.10.20
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Application inside Docker connects to:

172.18.0.1:5433

20. Final Mental Model

Remember this:

Tunnel local bind side = where your app connects.
Tunnel remote side = where SSH server connects from the target server.
Docker container cannot use host 127.0.0.1.
Never expose private services directly if SSH tunneling is enough.

If you follow these processes, you can safely set up SSH tunnels for databases, RabbitMQ, Redis, and similar internal services with production-grade practices.

SSH Tunnel for Docker Apps: PostgreSQL & RabbitMQ

Summary

Tags