Metalama 2024.1: Performance Improvements Analysis

TL;DR

Using Metalama instead of the standard C# compiler will increase your dotnet build time by typically 50%. This overhead varies linearly with the number and complexity of aspects in your project. Even with one aspect on every single method, your build should not be more than twice as slow with Metalama than without it.

Our Methodology

So, how did we arrive at these results?

We forked the NopCommerce open-source project and introduced aspects to methods at random, regulated by two factors:

• the percentage of types that will have aspects (denoted Types in the equation below); and
• the percentage of methods in each target type with aspects (denoted Methods).

Using BenchmarkDotNet, we analyzed the execution time of dotnet build /t:rebuild for 12 different combinations of the factors. We used a dedicated computer with a minimal Alpine installation to ensure no background process could interfere with the measurements.

We then computed a linear regression based on two factors, Types (the percentage of types affected) and Types×Methods (the percentage of methods affected), where both Types and Methods are included as a percentage between 0 and 1. The output of this function, TimeRatio, is the ratio between the build time with Metalama and without Metalama.

TimeRatio=a+b×Types+c×Types×Methods

Here are the results of the regression:

Our linear model has an R2 of 0.98, suggesting that only 2% of the variance is unexplained. Given that benchmark results variance is higher than 2%, we can confidently say this model is highly reliable.

In simple terms:

• Building your code with Metalama will always be at least 45% slower than without Metalama, and
• Metalama’s build time is linearly dependent on two factors: the percentage of affected methods in your projects and the percentage of types containing these affected methods.

The dependency on the number of types containing affected methods is because each type is typically implemented in its own file, and part of Metalama’s cost depends on the number of files that need to be rewritten.

Our Improvements in 2024.1

Let’s compare the improvements we made in the 2024.1 version to the 2024.0 baseline:

As you can see, the efficiency of Metalama in processing aspects has nearly doubled!

On the other hand, the constant factor (the fixed cost of using Metalama) has slightly increased, resulting in a 2.5% slowdown when there is no aspect at all. We’re still investigating the reason for this.

Wrapping Up

Our efforts in enhancing Metalama’s performance have been successful, with aspect processing speed nearly twice as fast as in Metalama 2024.0. However, there’s still room for improvement as using Metalama will slow your build by at least 45% compared to not using it.

Rest assured, we’ll continue to prioritize performance in our future releases. Stay tuned!