Remote session management

When running long jobs remotely (training, builds, data processing), you need to protect against network disconnections. Without session management, a dropped connection kills your running process.

tmux basics

tmux keeps your session alive on the server, even if your connection drops.

# Start a new session
tmux

# Detach from session (keeps it running)
Ctrl+b d

# List existing sessions
tmux ls

# Reattach to session
tmux attach

# Reattach to a specific session by name
tmux attach -t session_name

For a full list of keybindings, see the tmux cheat sheet.

Eternal Terminal (optional)

Eternal Terminal (ET) automatically reconnects when your network drops, without losing your session state. It’s an alternative to SSH that handles reconnection gracefully.

Install the client on your local machine:

sudo add-apt-repository -y ppa:jgmath2000/et
sudo apt-get update
sudo apt-get install -y et

Connect with agent forwarding:

et -f hostname

ET uses your SSH config for hostname/user resolution. The remote server must have ET installed and port 2022 open.

SSH agent forwarding with tmux

When using SSH agent forwarding (ssh -A or et -f) with tmux, there’s a common problem: after reconnecting, the SSH_AUTH_SOCK environment variable in your tmux session points to a stale socket path. This breaks git and other SSH operations.

The solution is to maintain a stable symlink to the current socket. Run this on the remote server to add it to your bashrc:

cat >> ~/.bashrc << 'EOF'

# Stable SSH agent socket path for tmux sessions
if [ -n "$SSH_AUTH_SOCK" ] && [ "$SSH_AUTH_SOCK" != "$HOME/.ssh/ssh_auth_sock" ]; then
    mkdir -p ~/.ssh
    ln -sf "$SSH_AUTH_SOCK" "$HOME/.ssh/ssh_auth_sock"
    export SSH_AUTH_SOCK="$HOME/.ssh/ssh_auth_sock"
fi
EOF

After adding this, new login shells will update the symlink and point to it. Your tmux sessions will use the stable path that always resolves to the current socket.