Timer is one of Go’s many ingenious designs. It only exposes a limited number of APIs when implementing single or multiple timing, encapsulating the relatively complex underlying data structures and concurrent operations.

When reading the source code of the Kubernetes scheduler, I learned more about the Go timer’s application and its encapsulation in kubernetes/utils for additional functions. And here in this article, I would like to share what I have absorbed, and try to cover all its implementations and expand more with the Kubernetes source code.

How to use timer

Go-timer means the Timerin the sleep.go.

NewTimer and AfterFunc

The comments explain its usage. It can only be created with NewTimer or AfterFunc. An event will be sent to C when the timer ends, and we can then consume this event via select. See the following paradigm.

What is printed here is default, because the select has been executed before the timer is triggered. There are two usual ways to correct this.

• Block the program directly and wait for the timer to be triggered.
• Put the select in a for loop to ensure that the logic of the timer can be triggered.

Another point to be noted is that break can only jump out of select, causing the for loop to be executed non-stop. And to break the entire for loop, a label is required. However, many developers do not like this method for its low readability, and they will turn to the AfterFunc, which is more decoupled, and executes the relevant logic via Callback function.

Reset timer

Both NewTimer and AfterFunc are for one-time execution, and you can use Reset method to reset the timer if you want to use it multiple times. Let’s achieve it with a slight modification to the above example, adding the following lines after the sleep call.

c.Reset(1 * time.Second)
time.Sleep(2 * time.Second)

Keep in mind that Reset must be performed on the timer been executed or the expired, a point marked in the official documentation.

For a Timer created with NewTimer, Reset should be invoked only on stopped or expired timers with drained channels.

The best way is the Stop method unless you are definitely sure that the timer has already been triggered when you call Reset, as in the example above.

Officially recommended

if !t.Stop() {
  <-t.C   // drain the timer if it is not stopped
}
t.Reset(d)

But “block” will occur

• When the timer (created with NewTimer) is timeout, Stop will also return false, then <- t.C will be blocked for there is no value in the channel.
• When you create the timer with AfterFunc, there is no chan field within and no operation to empty the channel, code will be blocked as well.

And select is the best to avoid bugs in either case.

if !t.Stop() {
   select {
     case <-t.C: // try to drain the channel
     default:
   }
}
t.Reset(d)

After

Another way to trigger the timer in addition to NewTimer and AfterFunc is time.After(d Duration), which directly returns the channel object in the timer instead of returning the timer object, and is usually used directly in select as a failsafe means to ensure that the select statement won’t be blocked all the time. The common usage is

Compared to the additional creation of a timer, this method is simpler. But problems may arise when you use two After in oneselect clause. The one that waits longer may never get the chance to execute. For example, the code below will only print str without stacking str.

The reason is both Afters will be reset after the first After is executed, so the timer that waits longer will never be triggered unless you define the timer outside the loop.

Ticker

Ticker is a timer that can be executed repeatedly without a manual reset. If you consider a timer as a Job object in Kubernetes, then a Ticker is the CronJob.

Its usage is basically the same as Timer, but its definition initializes a period to mark the interval between the two executions. And remember to initialize it separately instead of putting it into select, otherwise, it will be initialized repeatedly, causing memory leaks. See a negative example.

for {
   select {
   case <-time.Tick(1 * time.Second): // memory leak
      fmt.Println("repeat logic")
   }
}

What is a timer

Upon mastering the 5 methods listed above, we can now use timer gracefully in most scenarios. Then what about the timer’s implementation? It is not simple because Golang has encapsulated it so well.

The NewTimer and AfterFunc code shows us that it is the runtimeTimer object and its related methods that actually implement the timer logic.

What really functions in sleep.timer

• when, the time when the timer is triggered.
• f, the callback execution method. NewTimer adopts the sendTime method to simply send a current event to the channel, while, AfterFunc executes a goroutine to call the method in arg.
• arg parameter, the parameter used inf. A channel is passed into NewTimer, while a func into AfterFunc.

As is seen, the underlying runtimeTimer itself does not need a channel, so the AfterFunc implementation is not via channel.

Back to runtimeTimer, it uses the puintptr type internally, which is essentially a pointer towards the current g stack. Learn more about Go GMP scheduling if interested.

In the complex runtimeTimer, there are mainly three logic related to time.Timer.

• Add timer, whose main logic is in doaddtimer(pp *p, t *timer).

A btree-like structure maintains the order of the timer in the bottom layer, thus all operations of inserting, executing, and resetting the timer involve the “tree”. A parent node contains 4 child nodes, and when performing insert, it is necessary to determine whether the new timer is executed earlier. If yes, the current parent node needs to be replaced. This is how the siftupTimer method works.

• Execute timer. Its main logic happens in runtimer(pp *p, now int64).

By obtaining the most recent timer, p determines whether to execute the timer and it will only execute the timer in the Waiting state.

Two main logic lie in timer execution. First, determine whether a repeat is required. If so, calculate the next execution time and add the timer into the queue. Otherwise, delete the timer.

Then get the current P, unlock and execute it.

• Modify the timer state. The Stop and Reset will call the stopTimer(t *timer) and resettimer(t *timer, when int64) methods respectively. The former executes the deletion logic, involving the rearrangement of tree nodes. The latter makes sure only when the timer is in the timerNoStatus, timerRemoved state can it be re-queued, otherwise it will return errors. This is also the reason why the Stop method must be executed before using the timer.

Expand Timer

Let’s see some interesting applications of timer in Kubernetes.

clock.Timer

The clock.Timer tool class is used a lot in the scheduler, and it is a package of Go Timer and Ticker in Kubernetes, and adds the time-related parameters, integrating all functions in one. Seen from the definition of the Clock interface, PassiveClock contains the two common time methods of Now and Since.

By combining Clock with AfterFunc and NewTricker into a new interface, it can support different application scenarios. And also, provide the RealClock to implement all related methods.

TimedWorker

In TimedWorker, clock.Timer is used directly, and a worker containing the execution method is initialized with AfterFunc.

TimedWorker is saved via a TimedWorkerQueue, which is used in taintEvictionQueue in TaintManager eventually.

Scheduler also uses native time.Timer to build the PluginMap in waitingPodsMap. This PluginMap contains plugins that implement CSI, CNI, and CRI interfaces, and enables the scheduler to Allow or Reject pod based on the state.

Backoff

You may have used Jitter, a small tool in the utils/wait package. Backoff, also a tool in the utils/wait package, is not so prevalent. But we shouldn’t skip it here since it uses clock.

When implementing theBackoff method, the timer needs to be created or reset.

The Poll method is also very practical. It can execute a certain conditionFunc periodically (interval) within a certain period of time until the condition is true or timeout. Here, the timed execution and timeout are implemented by the native Timer.

This method almost contains the standard timer and ticker usage we mentioned in the beginning.

• Define a Ticker that can trigger events to tick.C periodically, and then send events to another channel.
• Define a timer, including timeout, to receive its channel with select in the for-loop. If it is triggered, then return.
• Monitor the context from the contionFunc. Once a Done event is received, end directly.

Both timer and ticker have added the defer Stop calls to avoid memory leaks caused by failing to be triggered.

Queue

Now, come to an example of client-go implementation of workqueue with clock. The details are in this delayingQueue, executing a waitingLooop method to determine whether the job in the queue needs to be popped and executed.

Define a new clock.Timer by continuously reducing the waiting time. Thankhen listen to the timer’s channel, and eventually add the data to the queue.

Timer is used widely in Kubernetes, but not so unified since the old code using native timer or ticker is not completely refactored to clock, though clock can already cover all the timer application scenarios.

The end

Timer is only a small function of Go, but how to use it reasonably to avoid various bugs is no easy task. In more cases, writing logic with timer and action combined in situations involving timeout, timed execution, or waiting is timer’s best practice.

Of course, there’s more to explore. But before that, use it, and master it.

Thanks for reading!