🖥️SYSADMIN
2024-11-158 min

Reverse SSH Tunnels: Accessing Machines Behind NAT

How I built a reliable remote access system for 200+ fuel stations with no public IPs using reverse SSH tunnels and systemd.

#SSH#Networking#Linux#Remote Access#Systemd

The Problem

When you're managing hundreds of industrial machines deployed at fuel stations across the country, you quickly run into a common infrastructure challenge: most of these machines have no public IP address.

They're behind GSM modems, corporate firewalls, or simply on private LANs with no port forwarding possible. Yet, you still need to:

  • Deploy updates remotely
  • Debug issues in real-time
  • Run automation scripts
  • Monitor system health

The Solution: Reverse SSH Tunnels

Instead of trying to reach the machine from outside, we flip the connection:

bash
# On the remote machine (fuel station)
ssh -R 2222:localhost:22 -N -o ServerAliveInterval=60 user@central-server

This creates a tunnel where:

  • The remote machine connects outbound to our central server
  • Port 2222 on the central server forwards to port 22 on the remote machine
  • We can now SSH into the remote machine through the central server

Making It Production-Ready

A simple SSH command isn't enough for production. Here's what we added:

1. Systemd Service for Persistence

ini
[Unit]
Description=Reverse SSH Tunnel
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/ssh -R 2222:localhost:22 -N -o ServerAliveInterval=60 -o ExitOnForwardFailure=yes user@central-server
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

2. Unique Port Assignment

Each machine gets a unique port (calculated from machine ID):

bash
PORT=$((2000 + MACHINE_ID))

3. Autossh for Extra Reliability

bash
autossh -M 0 -R $PORT:localhost:22 -N user@central-server

Results

  • 200+ machines accessible remotely
  • 99.9% uptime on tunnel connections
  • < 5 minute recovery time on network issues
  • Zero manual intervention needed

Key Takeaways

  • Reverse tunnels solve the NAT/firewall problem elegantly
  • Systemd + autossh = production-grade reliability
  • Unique port mapping is essential at scale
  • Always use key-based auth, never passwords

$ echo "Thanks for reading!"

Share:
AZ

Written by

Ayoub Zakaria

DevOps / Cloud / MLOps Engineer

Building reliable infrastructure for 400+ machines. Sharing real-world DevOps challenges, solutions, and lessons learned.

Get in touch →

Want more DevOps insights?

Explore all articles