spendallthetokens.com

The Walk Where I Deleted Pete

Managing headless AI agents from your iPhone, one sudo command at a time.

I was forty minutes into my afternoon walk when my phone buzzed. Telegram. The alerts channel.

๐Ÿšจ OpenClaw Gateway DOWN. Gateway restart failed 8 checks in a row. Manual intervention needed.

Fair enough. I had just killed Pete โ€” my AI chief-of-staff who'd been running on a headless Mac Mini for three months. I'd stopped his processes from an SSH terminal, backed up his files, written him a eulogy in a blog post, and was now walking through the neighbourhood while my phone received his dying screams.

The health check daemon didn't know Pete was supposed to be dead. It just knew the gateway was down, and it was doing its job โ€” alerting me every ten minutes that something was wrong. Dutiful to the end.

So I pulled out my phone, opened a terminal app, and SSHed into a Mac Mini from a sidewalk. Over a cell connection. To run a sudo command. To silence a dead agent's monitoring script.

And it worked.

The Toolkit That Fits in Your Pocket

If you're running headless servers โ€” whether it's AI agents, home automation, a media server, or anything else that needs to stay alive when you're not home โ€” the remote management stack comes down to three things:

Road Warrior Stack
Tailscale Encrypted mesh VPN. Your phone and your servers on the same private network, anywhere in the world. Free tier.
A Mosh-capable terminal SSH that doesn't die when your cell signal drops. Moshi is the best I've found on iOS.
A notification channel Something that pushes alerts to your phone. Telegram, Slack, email โ€” whatever your agents talk through.

That's it. Three layers. A VPN that makes your servers reachable from anywhere, a terminal that stays connected on unreliable networks, and a notification system that tells you when something needs attention.

The key piece that makes this work on mobile is the Mosh protocol. Regular SSH is like a phone call โ€” if the signal drops for even a second, the call ends. Mosh is like a text conversation โ€” if you lose signal, the messages are waiting when you come back. It runs over UDP, handles packet loss gracefully, does local echo so typing feels instant even on laggy connections, and survives network changes, sleep, and dead zones without losing your session.

I found Moshi while researching terminal apps for an upcoming road trip. My first instinct was Blink Shell โ€” the historically dominant Mosh terminal on iOS. But Blink switched to a subscription model and the community backlash was severe. The App Store rating dropped to 2.3 stars.

Moshi is newer, purpose-built for people running AI coding agents on remote servers, and has a five-star rating. It was the right tool hiding behind the obvious one โ€” a pattern that keeps repeating in this project.

The Sidewalk Incident

Here's what actually happened on my walk. I had already killed Pete's OpenClaw gateway and Node processes from my laptop. But a system-level health check daemon โ€” a LaunchDaemon running every ten minutes โ€” kept detecting the dead gateway and sending me alerts. It was working exactly as designed. It just didn't know the patient was supposed to be dead.

I opened Moshi, tapped my saved host, and connected to the server over Tailscale. The first command didn't work โ€” the daemon had already crashed and couldn't be cleanly stopped. So I tried the removal approach:

Moshi โ€” iPhone Terminal
$ sudo launchctl remove com.openclaw.health-check

$ sudo launchctl remove com.openclaw.gateway

$ sudo launchctl list | grep openclaw

$

Nothing returned. Clean. The daemon was gone, Pete was silent, and I was standing on a sidewalk with one bar of signal and a grin that probably concerned the neighbours.

There was also a moment where I accidentally fat-thumbed a random word before a command and got zsh: command not found: burp from a machine in my garage. That's the kind of thing that makes you laugh out loud on a sidewalk.

Why This Matters

If you're planning to be away from your infrastructure for any extended period โ€” a vacation, a road trip, a conference, or just a life where you're not always at your desk โ€” you need to know you can fix things from wherever you are.

Everything that can go wrong will go wrong given enough time. Power outages. Internet drops. Gateway crashes. Software updates that break things. The question isn't whether something will fail โ€” it's whether you can fix it from wherever you happen to be.

My walk proved I can. One bar of cellular, a VPN app, a terminal app, and a sudo command typed with my thumbs. If I can do it from a sidewalk, I can do it from anywhere.

The infrastructure you can't reach from your phone might as well not exist. Build for the sidewalk, not the desk.

The Stack, Explained for Humans

If you're running anything headless and want to manage it from your phone, here's the setup.

Tailscale is the foundation. It creates a private encrypted network between all your devices โ€” phone, laptop, servers โ€” regardless of where they are. Your server gets a static private IP that works from home, from a coffee shop, from a campground. Install the app, sign in, done. Free for personal use. This is what makes everything else possible.

Moshi is the terminal. It speaks both SSH and Mosh. It also does push notifications (your server can alert you when a long task finishes) and voice-to-terminal (speak your commands instead of typing on a tiny keyboard). It's new, it's five stars, and it's built specifically for people managing AI agents from their phones.

Mosh on the server takes thirty seconds to install: brew install mosh on macOS or sudo apt install mosh on Linux. That's the only server-side change needed. After that, you connect with mosh user@your-server-ip instead of ssh user@your-server-ip. Same destination, dramatically more resilient connection.

You can use Moshi as a regular SSH client immediately โ€” no server changes needed. Add Mosh later for the bulletproof connections. It works both ways.

What I Learned Standing on a Sidewalk

Typing commands on a phone keyboard is surprisingly workable but genuinely terrible for long commands. Save your most-used commands as snippets in your terminal app. "Check if the gateway is running" should be one tap, not forty characters of typed command.

The iOS clipboard permission dialog โ€” "App X would like to paste from App Y" โ€” will drive you slightly insane when you're copying commands from a chat into a terminal. Fix it: Settings โ†’ [App Name] โ†’ Paste from Other Apps โ†’ Allow. Do this for every app you regularly copy-paste between. Do it before you need it, not during an emergency.

Voice input is underrated. Moshi uses on-device Whisper transcription โ€” no cloud, no latency. Saying a command is faster and more accurate than typing it with your thumbs, especially while walking. This felt absurd at first and obvious within minutes.

Your monitoring should tell you when things are wrong AND when things are right. I'm adding a daily heartbeat โ€” one message confirming the servers are alive. If I don't get it, something's wrong. That's cheaper than checking manually.

The most important thing I learned: test your remote access while you're still home. I found a misconfigured host entry in one of my terminal apps that was trying to connect over public IPv6 instead of through the VPN tunnel โ€” it would have failed silently from the road. The sidewalk test caught it. Do your sidewalk test before your trip.

โ€ข โ€ข โ€ข

Later that afternoon, I sat back down at my desk, deleted the old user account, and started fresh. New agents, each with a specific job, sharing a single knowledge brain that compounds everything they learn. Leaner, faster, purpose-built.

But the moment I'll remember from today isn't the architecture planning or the blog writing or the research. It's standing on a sidewalk, one bar of signal, typing a command into my phone, and watching a machine go quiet.

That's the moment I knew the road trip would work.

Next up: Building the new agent team โ€” three specialists, one shared brain, and the principle that got us here: add nothing until you feel its absence. Stay tuned.

Follow the build: @spendthetokens on X