Watch me try to braindump a TCP server into existence

Watch me try to braindump a TCP server into existence

Many years ago, someone asked me to rattle off everything you'd have to do to have a barebones TCP server in C. It would be something which sat there and accepted new connections, in other words. Luckily for him, I had actually written some code like this in the recent past, so it was still fresh in my head. I went through and braindumped it on him, and I guess it was good, since the next thing I knew, I was on a plane for an in-person interview.

This memory came back to me in the past couple of days while pondering the general concept of interviewing. It made me wonder if I could still rattle it off in the "unfolding" approach that I did back on that phone call. By unfolding, I mean I don't write the entire thing top to bottom in one pass. Instead, I write just a little thing that does one piece, then insert another, then another, and so on, until it's "done" (as much as software can be done, that is).

I started talking my way through it again just to see how I'd explain it now. It occured to me this might be an interesting topic for a post, and here we are. Let's see if I can do this again right now, having just run through it in a recording (more about that later), but without looking at the code. This will probably have some errors in it, so don't just use it without checking things over first!

So, first, we want a socket. It's AF_INET and SOCK_STREAM and I think there's a third arg that we set to zero. Call this fd, and store it in an int. Okay. Now we want to bind that to a particular address family and port number, so we'll call bind with a that fd, a sockaddr_in, and a socklen_t. The struct sockaddr_in goes above this, obviously, and we need to initialize that to { 0 } to clear out any wackiness. The sin_family is AF_INET, and the sin_port is htons(whatever), because we might have to account for byte ordering differences. "whatever" might as well be 12345 for testing purposes.

Obviously we want to check the retvals from both of these, and if it's less than 0, whine about it on stderr and include the result of strerror(errno) before calling exit(1) to die. If those aren't working, there's nothing much we can do about it.

Now we need to set non-blocking mode on this, so that means fcntl on that fd with some magic value I'd have to look up which means "get". That value gets checked for a negative error condition as usual. Then we call fcntl with the fd, the "set" magic value, and the previous return value | O_NDELAY. This should also have its retval checked for an error.

Next it's time to say that we intend to use this as a listening socket and not an outgoing connection, so we call listen with that fd and some depth for backlogs. This is rubber-chicken waving at its finest, since I'm not sure I've ever run into a situation where I wished I had a higher value for the queue depth. I use 16 for no particular reason as a result. Again, this is checked for an error.

Now we're listening, so it's time to wait for stuff to happen. I choose to use select for the sake of this example even though many alternatives exist. We start a loop which runs forever, and in that loop define a fd_set called rfds. Then we FD_SET the listening fd into rfds and create an int called max_fd and set it to that same value. It'll be useful later.

Then it's time to call select, and the args are max_fd + 1, rfds and a bunch of NULLs for write fds, exception fds and the timeout. If select returns 0, there's nothing to do, so just "continue" back around to run the loop again.

Oh, and before I forget, we want to FD_ZERO rfds before using it.

If select returns -1, then we look at errno. If errno is EAGAIN (or maybe EINTR?), then we probably were bonked by a signal and were interrupted. Either way, it's nonfatal (because "worse is better"), so just continue. Otherwise, it's a problem and we probably need to cut and run since bad things are afoot.

If we're still here in the bottom of the loop, then odds are good that we have a new connection arriving. We use that FD_ISSET macro for the listener fd on rfds, and if true, jump out to a function which will accept the connection, passing the listener fd along. That function just calls accept and hands it the listener fd, a sockaddr_in, and yet another socklen_t. It might fail, so check the return value for negative values and deal with it, but don't die, since it's no problem for us. Just return.

Otherwise, we have a new client on the non-negative fd number which was returned, and we need to start tracking it.

That right there is enough to let multiple clients connect to a server port 12345, but it isn't enough to let any useful work happen. For one thing, we never check for activity on those new file descriptors, so if they talk to us, we'll never know about it. We'll have to add them to the fd_set of rfds before that can happen.

So, okay, back into the code. Let's have a STL map called "clients" from an int to a struct Client, and that struct doesn't have anything in it just yet. It'll be more interesting later on. We create this map right above starting the endless loop where select is called.

Now, we change the accept_new_client function to return a pass/fail bool, so it's false on those error conditions and true if it succeeds. We also make it take an "int*" which will be set with the return value from accept() which is the new client's file descriptor. Back where accept_new_client is called, a new int is created and is handed to that function. The return value of that function is also checked, and if it's true, then we drop a Client into the map using the int as the location. Basically, clients[client_fd] = Client(), more or less.

This means we now know about our clients, and we have to look for activity from them. Up above the select line but below where max_fd is set up, it's time to add a new loop. Create a const_iterator for clients and flip through all of the entries. For each entry, take the first value (which is just the client fd) and FD_SET it into rfds. Also, if it's greater than max_fd, set max_fd to it.

Now select will be looking for activity from our clients, so it's time to deal with that possibility. Down where we test for FD_ISSET on the listener fd, set up another loop to flip through all of the entries in clients just like before. This time, we use FD_ISSET on the client fd, and if true, pass it and the Client struct to a new function called read_from_client.

read_from_client just creates a char[] of some length and calls read with the client fd, that char pointer, and the sizeof that buffer. If it returns a negative value, the client is gone, so return false. If it returns zero, they're also gone, so return false too. What's left is a positive count of bytes, and that's how many bytes in buf are valid. This is where we'd hand the buf off to some kind of parser and say "here you go, you have 'ret' bytes to chew on, oh, and here's the Client struct". For now, however, we don't do much other than saying "read X bytes" and return true.

Back at the call location, we need to check the return value from read_from_client. If it's false, the client needs to be deleted. So let's create a vector of ints called to_delete just before this scan-the-clients loop, and if read_from_client returns false, just "push_back" that fd onto the vector.

Just beyond the client scan loop, now we iterate through to_delete, and for everything in there, call shutdown on that fd with a 'type' value (SHUT_RDWR, I think) and call close, too. Then we 'erase' that entry from the clients map and move on.

Why didn't I just shutdown + close + erase in the actual clients loop? Well, remember that we're iterating on the clients map in that loop, and removing something from that container might invalidate the iterator. Rather than digging around and worrying about whether this is one of those situations where that is unsafe, I just punt the deletions to a second pass and avoid it completely.

I actually recorded myself doing all of this a few minutes ago, but managed to hit ^D one too many times without realizing it during the recording. As a result, it ends after about 8 minutes right after my first test run. At that point, the program only accepts the connection and doesn't deal with tracking clients or anything which follows from that.

Still, it might be an amusing thing to watch. Thrill at the pauses while I jump to a man page in another window! Gaze in wonderment as I somehow manage to go to the exact line which has an error on it... because I'm running the compiler in yet another window!

This one is probably the most "raw" recording I've done yet. Normally I do a bit of prep work and try to figure out what's going to go where before I start. This time I decided to just go for it and let any false starts or dead-ends stay in there. If nothing else, it should give a good feel for the way things jump around at times.

It's a long one, so you might want to watch it at 200% or even 500%. There's also a new feature in the viewer: space will pause and unpause, and all four of your arrow keys will seek through the recording.

Look for "tcpserv" at the bottom of the list. This one is free, so it's already "unlocked". Enjoy!