ngrok for the wicked#

I've started using ngrok a lot lately (I know, I know, late to the party). But then last week, Homebrew has updated it to a version where it wants some 15$ to supply custom domain names. I mean, I could pay that, but I'm paying Hetzner like 8 or 9$ for a server, and then I'm paying for my domain and… it's still cheaper?

I understand that hosting is more commoditized compared to tunnels — I guess the market is wider — but still, it felt like I could spend a few hours and get something similar working. Why do I need custom domains? Because lost logins make me unhappy, and cookies want the same domain. Plus, testing OAuth is really painful since everybody wants to bind redirect URL.

There are many open source alternatives, but the one I really liked is called SirTunnel. It's a small script which uses Caddy's JSON API to add and remove domains. I did not use it, though, since it got me thinking: why do I have to add and remove domains if what I basically need to serve some local processes on the internet? So if I started a process of that particular site, it should just work, no other movements necessary.

The Plan

It's simple! I create a wildcard domain (something like *.xxx.solovyov.net, d for development), and then reverse-proxy everything through an SSH tunnel locally.

I'm still going to use Caddy because it's ability to generate certificates automatically and laconic config appeal to me. 👍

Caddy on Server -> SSH Tunnel -> Caddy on Laptop -> Local Processes

Why do I need a local Caddy? Because SSH can proxy only single port and local processes occur on different ports. Why Caddy and not nginx? You'll see.

Execution

So, I've got a local Caddy working with many domains mapped to various ports. My plan is to run every project on a separate port, so when I start a process, it's immediately available to the world. Feels a bit exhibitionist, but very convenient. ☺️

Next was permanent SSH tunnel. I could've done that myself, but I just found a recipe.

The first problem was that I tried to configure wildcard-handling domain in Server-Caddy — this causes it to ask for a wildcard certificate, which requires a DNS plugin for my provider. I moved my domain to Cloudflare (which is one of the supported providers), added plugin, created new API token and… it did not work! Argh. Okay, really, I'm not starting new projects every week, so I'll just list them all in config.

Next problem was that it all worked. I could not believe my eyes! But the story does not end here.

You know what is irritating about ngrok? It's that latency. Especially when you're on a bad connection. I mean my site is right here on my laptop but those roundtrips just to test OAuth and what not… Argh. I did not invent anything better than just writing that development domain in /etc/hosts. And. It. Worked! Too bad /etc/hosts does not support wildcards and I dislike additional processes like DNS servers on my laptop.

Tutorial

You'll need a domain name you own and control (really, control matters more than ownership here 😜) and a VPS somewhere in the world. Please do not copy and paste stuff blindly, you'll have to change at least the domain name for this to work. :)

Domain

Add *.xxx entry of type A to your domain pointing at your VPS.

Local Caddy

brew install caddy — correct for your package manager — and start with this Caddyfile:

{
	auto_https disable_redirects # so remote caddy is happy
	email [email protected] # so you can debug problems with certs
}

(local) {
	{args.0}.xxx.solovyov.net {args.0}.xxx.solovyov.net:80 {
		encode zstd gzip
		reverse_proxy localhost:{args.1}

		handle_errors {
			respond "Local: {http.error.status_code} {http.error.status_text}"
		}

		log {
			level DEBUG
			output file /opt/homebrew/var/log/caddy/{args.0}.log
		}
	}
}

import local test 5000

This (local) thingie is called a snippet. Now I can just copy this import line as many times as I want, not having to repeat those lines.

We instruct Caddy to listen to port 80 so that basic HTTP works. This is convenient for debugging plus our SSH tunnel is targeting this port. But HTTPS is still useful to have since then external and internal setup is more similar.

brew services start caddy or equivalent to make Caddy start after run. Or run after start?

Persistent SSH Tunnel

Just follow a tutorial from Tyler, should be simple enough. Your /.ssh/config entry should look like this:

Host sshtun
	HostName your.remote.server
	RemoteForward 6800 127.0.0.1:80

What Tyler doesn't tell is that you have launchctl load -w Library/LaunchAgents/your.plist.name.plist, this little -w marks it enabled so launchctl will start it after restart (sounds like a common theme ain't it?).

Remote Caddy

No need to disable automatic redirects, so minimal version will look like this:

{
    email [email protected]
}

(sshtun) {
    {args.0}.xxx.solovyov.net {
        encode zstd gzip
        reverse_proxy localhost:6800

		handle_errors {
			respond "Remote: {http.error.status_code} {http.error.status_text}"
		}

        log {
            output file /var/log/caddy/sshtun.log
        }
    }
}

import sshtun test

You can see I'm using snippets here as well, but the port is the same every time, since this is our SSH tunnel. I'd be glad to switch to wildcard domain, maybe I'll try again later to remove a need to edit this file.

/etc/hosts

If you want the same glorious setup, add this to /etc/hosts:

127.0.0.1 test.xxx.solovyov.net

but do not forget to use your actual DNS address. :)

Adding a new domain

There are three edit points:

• Local Caddyfile — to map name to port (again, I give different projects different ports so that I can start a few of them simultaneously)
• Remote Caddyfile — since I couldn't get wildcard certificate
• /etc/hosts — to short-circuit browser

The End

And you know what? No need to start many ngrok processes occupying your terminal when you want a few of sites running! Epic.