Learning from past tickets to better solve new tickets

Learning from past tickets to better solve new tickets

Here's a story about introducing search in a vacuum. In a previous job, I worked support tickets and took phone calls. On my first day, I was handed a pile of parts resembling a whitebox PC and got a manager to burn a Slackware CD for me. I figured the rest out by myself. The technical aspects were no big deal: get on the network, start a web browser, and log into the ticketing system. What I had to learn then was policy: what will the company do and what will it ignore? What are the best practices for handling customer requests?

I started reading tickets out of the queue. Every time I saw something new (which was quite often at first), I'd bookmark it. That way I could come back to it later and find out how some other tech had resolved it. I quickly found out how the "spheres of support" really were defined in practice, and what other techs would stand behind, and where they'd punt and say "you did it, you support it".

The problem with all of this is that I started getting too many bookmarks. Also, a linear list of bookmarks with the "#xxxxxx-yyyy Some subject the customer said" is slow to scan, slower to search, really hard to remember that it's really about topic X or Y. I switched to using a flat file, putting just the ticket number and a few choice words, one per line. It might say "040506-1234 reset urchin password" or "040608-2356 blow away plesk lockout". grep did the rest.

This went on for a bit, and then one night, I overheard a tech chatting about some customer's problem in the break room. I mentioned that I had noticed something just like that on some other machine not too long in the past, and said I'd dig up the ticket for him. Sure enough, back at my desk, I was able to pull it from my file and shot it over in an IM message. He was astounded: how did I find that, and so quickly?

Some background to explain his astonishment: our ticketing system did many awesome things, but full-text search was not one of them. When I came up with that ticket as a citation in a few seconds, it seemed like magic. I came clean and told him about my list. He thought it was the best thing ever and requested access.

I quickly ported my flat file over to a simple MySQL database and whipped up some CGI gunk to allow searching it, then put a simple form page in front of it and stood it up. People really started getting into it, and soon they wanted to add new stuff, so I added something which would do that too. Pretty soon we had a nice little community going.

It got to the point where I wrote a digest generator. Once a week, all of the entries added in the past week would be mailed out to anyone who wanted to see. A bunch of people signed up for this since it was a way to keep tabs on the latest awesome save or "hoser ticket" (as we called them) even if you weren't on shift to see it as it happened.

For some reason, this service never graduated to a production sense. I guess something which was done in a grassroots fashion by "mere support people" (ha!) just isn't that important. No matter, it probably worked better being separate from the official ticketing system, anyway.

As of last summer, I heard my poor little workstation box is still alive in some closet, serving up requests for those who still know about this service. Useful things are hard to kill.