C# is gradually incorporating additional features typical of dynamic languages – thanks to the increased support for them that’s integrated into its underlying platform.

One of these features is the Roslyn Compiler-as-a-service (CaaS) API, that allows any program to use compiler services to manipulate code at runtime.

In this article, I will discuss an approach to take advantage of Roslyn to increase the amount of dynamic language features we can use into our programs. This approach preserves type safety if required and enhances the performance of dynamically added code when compared with using ExpandoObject instances.

Dynamic Languages – Introduction

Dynamic languages like Python, Ruby or JavaScript are very popular in web development, rapid prototyping, game scripting, interactive programming, and other scenarios requiring dynamic adaptiveness [1]. They enable rapid development of database-backed web applications and implement the Convention over Configuration and DRY (Do not Repeat Yourself) principles.

A dynamic language provides high-level features at runtime that other languages only provide statically by modifying the source code before compilation[2]. They support runtime metaprogramming features, which allow programs to dynamically write and manipulate other programs (and themselves) as data. Examples of these features are:

• Fields and methods can be added and removed dynamically from classes and objects.
• New pieces of code can be generated and evaluated at runtime, without stopping the application execution, adding new classes, replacing method bodies, or even changing inheritance trees.
• Dynamic code containing expressions may be executed from strings, evaluating their results at runtime.

These metaprogramming features can be classified in different levels of reflection (the capability of a computational system to reason about and act upon itself, adjusting itself to changing conditions), and they are a key topic of our research work [1] [2].

Dynamic languages are frequently interpreted, and generally check type at runtime [5]. The lack of compile-time type information involves fewer opportunities for compiler optimizations.

Additionally, runtime type checking increases both execution time and memory consumption, due to the type checks performed during program execution and the additional data structures needed to be able to perform them[1][2][3]

Because of this, there are approaches to optimize dynamic languages [3] or to add their features to existing statically typed ones [4]. These are based on modifying the language runtime [8] or using platform features to instrument the code at load time. In either case, runtime modification of programs is normally supported via an external API using compiler services that can be accessed programmatically [4].

These approaches were successful, proving that dynamic language features can be incorporated to statically typed languages and take benefit from both static type checking and runtime flexibility.

However, they require the development of a supporting API or a custom virtual machine [3]. This requires a lot of effort and, as the new dynamic features will not be part of the platform, they must be distributed along with the programs that use them.

However, the modern .NET ecosystem enables us to obtain a lot of these features just with standard platform services or modules.

The main contribution of this article is to demonstrate how the Roslyn Compiler-as-a-Service (CaaS) API can be used to achieve runtime flexibility and type safety, while implementing most of the described dynamic language features.

We’ll demonstrate how a class can be fully created at runtime, incorporating custom methods and properties. The compiler services compile the dynamic code ensuring that no type errors exist [4].

You’ll also see how this dynamic class can comply with a known interface so type safety can be preserved in the program, at least with part of the dynamically created code. Finally, we will also check if this allows a substantial performance gain when compared to an older, non-statically typed approach that creates dynamic objects via the ExpandoObject[9] class.

All code shown in the following sections have been successfully tested on Visual Studio Enterprise 2019 using a .NET Core 3.1 solution.

Using ExpandoObject to create dynamic code

The ExpandoObject class was incorporated to the .Net Framework since version 4.0 as part of the System.Dynamic namespace. This class implements the IDictionary interface.

Thanks to the syntactic sugar embedded into the language and the dynamic type, it can be used to create new objects whose content can be fully defined at runtime.

The following code snippet shows how a programmer can add the ClassName property to an ExpandoObject instance by just writing a value to it. Instead of giving an error for writing to a property that does not exist, the property is created and initialized inside the ExpandoObject instance.

Also, ExpandoObject behave like dictionaries, and AssignProperties takes full advantage of it to dynamically add multiple properties.

The following code snippet shows how easy it is to create an object instance whose contents can be fully customized using this approach.

The C# compiler does not perform any static type checking on variables of dynamic type. This allows the code to compile, even if the compiler lacks information about the static structure of an instance.

As a tradeoff, every operation that reads or invokes members dynamically, loses the type safety provided by the compiler: every type error is detected and thrown at runtime.

This next example will be type checked at runtime and found valid, as all accessed members have been already defined, and the dynamic state of the object determines the operations that can be applied to it (duck typing[10]).

Methods can be also added to ExpandoObject instances, but normally they have to be bound to a particular one, which is the instance that holds the properties these methods need to perform calculations on, as shown in the following code snippet.

This way, to add this method to multiple ExpandoObject instances, we need to create one per instance, using each instance variable name. The ExpandoObject approach does not follow the traditional class-based language behavior, as individual instances can be modified and evolved independently. It is closer to the prototype-based object-oriented languages model [11].

 The Absolutely Awesome Book on C# and .NET. Covers C# 6, 7 and 8 & .NET Core

Using Roslyn to create dynamic code

What happens if we need a more “traditional” class-based approach?

Using ExpandoObject is not suitable, as ExpandoObject cannot work as runtime modifiable classes that contain the structure of all their instances. However, we can achieve this with the Roslyn module from Microsoft [12] [8], that can be added to any project via the Microsoft.CodeAnalysis.CSharp.Scripting NuGet package:

We can create source code and ask Roslyn to compile it at runtime. To do so, data structures to hold the necessary member information must be created.

We have created a simple demo program to demonstrate this, using the DynamicProperty class to hold just new property names and types. A more complete implementation could use FieldInfo and MethodInfo classes from System.Reflection to obtain data from existing class members.

The same can be done to dynamically build the source code of the methods to add. This can be achieved using Roslyn SyntaxTree analysis features. It allows us to read any source code file (.cs) into a syntax tree and do several operations, including obtaining the actual source code of any method we want. This may seem like a limitation, as source code might not be available because only the binaries were distributed, but source code can be obtained by an assembly decompiler (such as dotPeek [14]) from compiled files.

In our demo, we just read a method source code from a sample code file.

However, this is just a convenient feature to “reuse” existing source code. Nothing prevents us from fully providing source code as a string, the same way as dynamic code evaluation features from typical dynamic languages [7].

Once we have appropriate data structures to hold the dynamic class information, we can create it. This is when we can achieve a certain degree of type safety by forcing the newly created class to implement elements known by the compiler: inherit from an existing class or implement multiple interfaces. This way, access to part of the structure of the dynamically created class can be type checked statically, as shown in the following code snippet that forces the class to implement the IHasAge interface we created.

We do not know the full structure of the newly created class at this point, but we know that it implements the IHashAge interface. Therefore, we can access its members normally, and when using them, type errors will be detected by the compiler. We can still use a dynamic typing approach to access the rest of the class structure. The difference is that now both static and dynamic type checking can be used in the same class if we want.

How we achieve this using Roslyn?

We built the source code of the new class from the classes we created to hold member information. Later, we used our Exec dynamic code execution function to both compile the class and create a new instance of it (our demo only supports dynamic classes with a non-parameter constructor). Exec is the function that really uses the Roslyn API.

Once we create a new dynamic type like this, accessing it may be hard because we did not specify a namespace or class visibility. To facilitate this, we create a single instance of the new type and return it in CreatedObj, a dynamic property of a custom Globals class we pass to the Exec method. Finally, we return this object converted to the passed generic type:

The source code of the new dynamic class is created via the ToString method of the instance dt of the utility class I created for this demo, DynamicType.

Please note that this code is just a demonstration of how dynamic class creation can be achieved.

If this code is extended to support a dynamic class creation framework, additional features should be considered, such as allowing multiple instances of dynamically created classes. All this is possible instructing Roslyn to perform the compilation of source code via its CSharpScript.EvaluateAsync method, that accepts:

• The source code of the class to be created.
• Global variables that this code may use. In our implementation this is the Globals class.
• Assembly references potentially used by the code. We include System by default, and any other assembly containing the parent class of the one that is going to be created ( or the interface it implements).
• Finally, the import parameter is used to add any namespace that the source code needs (same as the using keyword).

As a result, this task may return a compiler error if the code does not compile. We use this information to compose the full compiler error and return it to the caller. This way, the reported errors have the same level of detail as the ones the compiler outputs before a program is run.

Therefore, this approach allows us to build a fully dynamic class, compile it, and return an instance whose static type may be partially known, providing type safety and a traditional class-based approach.

The same code with little variations can also be used to evaluate expressions dynamically, thus implementing another typical dynamic language feature.

The main disadvantage of this approach is that, once the new class is created and compiled, it cannot be modified again as compilation “closes” it.

Further modification requires creating a new type and re-creating all the instances of the old type into instances of the new one. For the same reason, classes compiled statically cannot also be modified. However, the .NET platform already has features to modify method bodies of already compiled classes at runtime [15], which partially covers this.

Performance Analysis – Dynamic Typing

As we said, using dynamic typing usually imposes a performance penalty. Our approach both combines flexibility and static typing advantages, but does it hold any performance advantage over the ExpandoObjectapproach?

Reading from a dictionary is very efficient, but an actual method call could be faster. To test this, we compared 2000000 calls to the GetAge method on both the ExpandoObject and our dynamic class instances, both methods sharing the same implementation.

Performance measurement has been done at steady state, using a C# port of a statistically rigorous measurement procedure [16][15] we used in past [3] with excellent results. This procedure runs a maximum of 30 iterations of 30 executions of the code to be measured, stopping prematurely if the coefficient of variation (CV, the ratio of the standard deviation to the mean of the collected data) falls below 2.

This ensures that the effect of external events into measured times are minimized. Execution was performed in Release mode, over an AMD Ryzen 1700 with 64Gb of 2400Mhz RAM. The results appear in the following table:

Our measurements show that the Roslyn approach holds an average performance advantage of 15% over the ExpandoObjectalternative, thus being significantly faster. This means that if full dynamic typing is not really needed to implement a feature, the Roslyn approach provides convenient type safety and performance advantages when using the dynamically created types through the program code.

There is also an initial performance cost when creating a new type via Roslyn (the cost of compilation). Programmers do not need to choose between static or dynamic typing, hybrid approaches obtain advantages from both.

Conclusion

Roslyn CaaS can be effectively used to build dynamic code enabling flexibility and type safety if the class to be build has at least a partially known structure.

Measurements show that this hybrid approach holds a significant performance advantage over the fully dynamic ExpandoObject. This approach also maintains a more traditional class-based behavior that may be easier to use by programmers that need to dynamically incorporate pieces of code.

Acknowledgments

This work has been partially funded by the Spanish Department of Science, Innovation and Universities: project RTI2018-099235-B-I00. This has also been partially funded by the project GR-2011-0040 from the University of Oviedo

Download the entire source code at github.com/dotnetcurry/Dynamic-Class-Creation-Roslyn.

This article was technically reviewed by Yacoub Massad.

References

[1]     L. D. Paulson, “Developers shift to dynamic programming languages,” IEEE Computer, vol. 40, no. 2, pp. 12-15, 2007.

[2]     O. Callau, R. Robbes, E. Tanter and D. Röthlisberger, “How (and why) developers use the dynamic features of programming languages: the case of Smalltalk,” Empirical Software Engineering, vol. 18, no. 6, pp. 1156-1194, 2013.

[3]     F. Ortin, J. Redondo and J. B. G. Perez-Schofield, “Efficient virtual machine support of runtime structural reflection,” Science of Computer Programming, vol. 74, no. 10, pp. 836-860, 2009.

[4]     F. Ortin, M. Labrador and J. Redondo, “A hybrid class- and prototype-based object model to support language-neutral structural intercession,” Information and Software Technology, vol. 44, no. 1, pp. 199-219, 2014.

[5]     L. Tratt, “Dynamically typed languages,” Advances in Computers, vol. 77, pp. 149-184, 2009.

[6]     J. Redondo and F. Ortin, “A comprehensive evaluation of common python implementations,” IEEE Software, vol. 32, no. 4, pp. 76-84, 2014.

[7]     I. Lagartos, J. Redondo and F. Ortin, “Efficient Runtime Metaprogramming Services for Java,” Journal of Systems and Software, vol. 153, pp. 220-237, 2019.

[8]     J. M. Redondo, F. Ortin and J. M. Cueva, “Optimizing reflective primitives of dynamic languages,” International Journal of Software Engineering and Knowledge Engineering, vol. 18, no. 6, pp. 759-783, 2008.

[9]     Microsoft, “ExpandoObject class,” Microsoft, 2020. [Online]. Available: https://docs.microsoft.com/en-us/dotnet/api/system.dynamic.expandoobject?view=netcore-3.1. [Accessed 30 4 2020].

[10]     D. Thomas, C. Fowler and A. Hunt, Programming Ruby, 2nd ed, Chicago (Illinois): Addison-Wesley, 2004.

[11]     J. M. Redondo and F. Ortin, “Efficient support of dynamic inheritance for class-and prototype-based languages,” Journal of Systems and Software, vol. 86, no. 2, pp. 278-301, 2013.

[12]     Microsoft, “Overview of source code analyzers,” MIcrosoft, 2020. [Online]. Available: https://docs.microsoft.com/en-us/visualstudio/code-quality/roslyn-analyzers-overview?view=vs-2019. [Accessed 30 4 2020].

[13]     J. Redondo, “New features of C# 8 and beyond,” 2019. [Online]. Available: http://www.researchgate.net/publication/330514620_New_Features _of_CSharp_8_and_beyond. [Accessed 25 10 2019].

[14]     JetBrains, “dotPeek: Free .Net Decompiler and Assembly Browser,” JetBrains, 2020. [Online]. Available: https://www.jetbrains.com/decompiler/. [Accessed 30 4 2020].

[15]     T. Solarin-Sodara, “POSE: Replace any .NET method,” GitHub, 8 1 2018. [Online]. Available: https://github.com/tonerdo/pose. [Accessed 30 4 2020].

[16]     A. Georges, D. Buytaert and L. Eeckhout, “Statistically rigorous java performance evaluation. In: Object-oriented Programming Systems and Applications,” in OOPSLA’ 07, New York, NY, USA, 2007.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!