Expressing my laziness in concrete ways

Expressing my laziness in concrete ways

I'm a lazy programmer sometimes. Let me tell you a story about something I wrote earlier this year that's not exactly the finest set of programs ever produced. It's about the whole feed rate limiting thing on my web server.

I've been getting reports of people who run into the block even when they didn't do anything wrong. They didn't start up something that polled every two seconds and pulled the full ~500K feed every single time, for example (and yes, this has happened at least once).

No, the problem goes like this - someone sees the little orange feed icon up there (on the web view, that is) and clicks on it and gets a screenful of XML. My server also goes "okay, you just got the feed". Then they take the URL, hand it over to their feed reader, and it reaches out and tries to make the same request. My server says "hey wait a minute you clown, you JUST GOT IT", and rejects it with a 429.

See the problem? It can't tell the difference between a pairing of a one-time human request + their feed reader's startup sequence and someone who's actively hammering the thing. It's because the thing is relatively stupid. It knows about IP addresses, request types (conditional or not), and elapsed times. That's it.

In order to support some kind of "you're going to make a handful of closely-spaced unconditional requests at startup but will be good thereafter" leniency, it would have to actually have some thought put into it. Now you're talking about more of a "token bucket" system, or something else of that sort where it does some time-based accounting and allows for "bursty" behavior at first. That means tracking a lot more than just "the last time you got a full copy of the feed".

But you know what? That's work. It's not fun, it's not interesting, and it doesn't do me any favors besides avoiding receiving feedback messages from confused users of feed readers. So, I've been lazy, and I haven't done it. I've instead done a bunch of other things which also had to be done and had slightly better contexts.

I'll admit something else: I don't have a ready solution to this. I've never written a burst-handling inflow system before. It would be different if I could just reach back into my head and go "oh yeah, this is just one of those things from XYZ project". But nope, this time there's nothing in the past to "borrow" in the present.

Also, this feels more like a "moving average" type of problem, which then means *actual math*, and that's just not my bag, normally. So, I find reasons to do something else. Repeat as necessary.

Again, most feed readers and their users are doing just fine. This is something I have to do in order to deal with the pathological cases who are small in number but large in impact. I suspect that just a handful of them take up way more resources than all of the normal, happy, good people put together.

As with so many technologies, they would all be unnecessary if not for the people who are causing the problems. It's why some of us get wistful for the "old days" when the net was far smaller and the amount of bad behavior was accordingly tiny - eternal September and all that.

Do I want to write rate-limiters? Hell no. I'd rather do anything else.