Don't setenv in multi-threaded code on glibc

Don't setenv in multi-threaded code on glibc

Remember in 2014, when I wrote about getting stuck if you called basically anything between fork and exec if you have threads active? The general idea was that thread #1 calls [un]setenv, grabs a lock, and then thread #2 forks, (lazily) copying the process, and copying the lock in the process. Then the child process calls [un]setenv, tries to grab the lock, finds it already set, and hangs forever.

If you read that post, you might have started thinking "oh, well, at least it's thread-safe". Eh, well, no, not so much. While those functions won't stomp on themselves, there's plenty of badness which can and will happen if you call them.

The best one is that you can make getenv crash. Yes, that's right, just trying to read the environment will blow up on you if you race with a setenv call.

Don't believe me? Try it yourself.

$ cat mtenv.c
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
 
static void* worker(void* arg) {
  for (;;) {
    int i;
    char var[256], *p = var;
 
    for (i = 0; i < 8; ++i) {
      *p++ = 65 + (random() % 26);
    }
 
    *p++ = '\0';
 
    setenv(var, "test", 1);
  }
 
  return NULL;
}
 
int main() {
  pthread_t t;
 
  printf("start\n");
  setenv("foo", "bar", 0);
  pthread_create(&t, NULL, worker, 0);
 
  for (;;) {
    getenv("foo");
  }
 
  return 0;
}
$ gcc -g -o mtenv mtenv.c -lpthread && ./mtenv
start
Segmentation fault (core dumped)
$ 

No, really, it is dying inside glibc's getenv, because you are not allowed to setenv in a multi-threaded program! Crash it in a debugger and you can see exactly what's going on:

Program received signal SIGSEGV, Segmentation fault.
0x00000033a8a35036 in getenv (name=0x400849 "o") at getenv.c:81
81	      for (ep = __environ; *ep != NULL; ++ep)

If you dig through the remains, you'll find that one of several things may have happened. If it crashed right here, ep might be pointing at a former location of "environ" which is no longer part of your segment. You just read off the end of your memory. Segfault.

It might crash another way, too: this is a char**, so it's supposed to have a bunch of pointers to char* buffers, one for each var=val pairing. What if that region has been reused and now contains a bogus pointer into the weeds? Yep, segfault.

There might be a couple more ways, but you get the idea. You're dead in the water.

One more time, from the horse's mouth:

Modifications of environment variables are not allowed in multi-threaded programs.

-- the glibc manual

Do you run multi-threaded programs on glibc? Do they call setenv or similar? Do they crash randomly in getenv? This is probably why.

"But wait", you say. "I don't use getenv". Not so fast. I bet you actually do... indirectly.

Do you ever take time elements and turn them into a time_t with mktime()? Guess what? In glibc, mktime() does a tzset(), which then calls getenv("TZ").

Maybe you don't call mktime() yourself. Cool. But what about your libraries? Do you use libzip? libzip needs to map DOS time onto Unix time and calls mktime() at some point. Oops.

Really, you can't be sure who's going to be poking around in there. So all you can do is not call setenv. If for some reason you DO need to set up something in your environment for an eventually-multi-threaded program, you'd better get it out of the way before you kick off any threads, and then leave it like that forever. Don't try to change it while the program is running.

Incidentally, this is the same technique some programs use to fork with threads: they fork early before any threads are up. The parent continues on to start its threads, while that initial child then spawns everything else as needed (while being careful to not create threads itself).

None of this is new, but we do re-discover it roughly every five years.

See you in 2022.