While computers talk to each other using 1s and 0s, you often encounter APIs that communicate in a text-based format. How does an app convert human-readable text into machine-readable binary data?

The answer is data serialization and deserialization. The Android community has access to high-quality data serialization libraries such as Moshi and Gson. But, there’s a newcomer on the horizon that promises to be better than existing options: Kotlin Serialization.

In this tutorial, you’ll build an app that suggests a list of things to do when you’re bored. Through this process you’ll:

• Learn the differences between data serialization, deserialization, encoding and decoding.
• Work with the Kotlin Serialization library.
• Integrate the library with Retrofit to interact with an API.
• Write custom serializers for Kotlin classes.
• Learn the limitations of the library.

Getting Started

Download the starter project by clicking Download Materials at the top or bottom of the tutorial. Then, open the starter project, Bored No More!, in Android Studio.

Build and run. You’ll see the following screen:

The app consists of two screens. The first screen displays a list of activities to try when you’re bored, while the second screen displays the details of a specific activity.

The starter project uses static data preprogrammed into the app. Throughout this tutorial, you’ll refactor the code to communicate with the Bored API to fetch a list of activities.

Before you start coding, take a closer look at data encoding and serialization.

Understanding Data Encoding and Serialization

An object that exists in a program’s memory consists of binary data that a computer can use directly. Serialization is the process of reducing an object to its primitive contents, like numbers, strings and arrays, in a manner that preserves its structure. Encoding a serialized object is the process of converting its primitive contents to an output format by following a specific set of rules.

Consider an object in memory that consists of two primitive values:

val hugeParty = Party(pizzas = Int.MAX_VALUE, people = Int.MAX_VALUE)

Its serialized representation contains its primitive values in a structured format. It’s not important what this representation looks like: What matters is that it contains information about the object so it can be encoded into a specific format.

"01101110 01110101 01101101 01010000 01101001 01111010 01111010 01100001 01110011 ..." // Hypothetical serialized representation

The object’s encoded representation contains its data represented according to the rules of a specific format, such as JSON, XML, YAML or TOML. Here’s an example of the JSON representation:

{ "pizzas": 2147483647, "people": 2147483647 } 

Of course, before a computer can use encoded data it must decode the data.

Understanding Data Decoding and Deserialization

Decoding is the process of parsing encoded data to produce a deserialized representation of an object consisting of its primitive contents while maintaining its structure. Deserialization is the process of converting such a decoded stream of primitives into an object. Thus, decoding and deserialization are the opposites of encoding and serialization, respectively.

The Kotlin object is broken down into primitives via serialization and encoded into data. Then, the data can be decoded into primitives and deserialized into a Kotlin object. Here’s a diagram that summarizes the flow of converting a piece of data to and from an encoded representation:

Now that you understand the differences between serialization and deserialization, it’s time to get started with the Kotlin Serialization library.

Kotlin Serialization

The Kotlin Serialization library, or kotlinx.serialization, combines a compiler plugin and a runtime API.

Head over to Android Studio to add the following dependencies to your project.

First, open the project level build.gradle and add the Kotlin Serialization plugin to your classpath in the dependencies block:

dependencies {
  // Other classpath declarations
  classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version"
}

Then open the app module’s build.gradle and apply the plugin in the plugins block at the top of the file:

// Other plugins
apply plugin: "kotlinx-serialization"

Next, add a dependency on the JSON encoder module in the dependencies block:

dependencies {
  // Other dependencies
  implementation "org.jetbrains.kotlinx:kotlinx-serialization-json:1.2.2"
}

Finally, sync the project to download these dependencies.

But wait, why is the Kotlin Serialization library split into a compiler plugin and an encoder module? To understand why you need to learn more about how the library works.

The Compiler Plugin

Traditional data serialization libraries use one of two approaches:

1. Code Generation, such as Moshi.
2. Reflection, like Gson.

While the code generation approach yields much faster runtime performance, it also leads to increased build times. On the other hand, reflection-based libraries offer slow runtime performance but impose little to no penalty on build times.

Like Moshi, the Kotlin Serialization library relies on code generation. However, unlike Moshi, it uses a compiler plugin to generate code instead of an annotation processor. Using the compiler plugin, the Kotlin Serialization library can offer excellent runtime performance while maintaining fast build times.

The JSON Encoder Module

In the previous section, you learned about the differences between serialization/deserialization and encoding/decoding. While the compiler plugin provides the former functionality, the library delegates the responsibility of encoding/decoding data into specific formats to separate modules.

The JSON Encoder module lets you convert serialized Kotlin objects into their JSON representation and vice versa. The library also offers official modules for other formats, like CBOR and Protocol Buffer. You can even use third-party modules for other formats.

Comparing Kotlin Serialization Library to Moshi and Gson

The Kotlin Serialization library offers quite a few advantages over the existing data serialization libraries Moshi and Gson, such as:

• Faster runtime performance than Gson and Moshi in reflective mode through code generation.
• Faster build times than Moshi through the use of a compiler plugin rather than an annotation processor.
• Better support for Kotlin’s type system compared to Gson, including support for nullable and default values.
• Support for many encoding formats through the use of separate library modules.

Now that you understand the advantages of Kotlin Serialization, it’s time to learn how to use it!

Modeling Data

The Kotlin Serialization library generates serializers for classes annotated with @Serializable.

A serializer is a class that handles an object’s serialization and deserialization. For every class annotated with @Serializable, the compiler generates a serializer on its companion object. Use this method to get access to the serializer for that class.

Go to Android Studio, and open BoredActivity.kt in data. In this file, you’ll find the BoredActivity data class that represents an activity to try when you’re bored.

Now you need to make this class known to the Kotlin Serialization library. Import kotlinx.serialization.Serializable and annotate BoredActivity with @Serializable:

import kotlinx.serialization.Serializable

@Serializable
data class BoredActivity(
  // The rest of the data class...
)

Then build the project and see it compile.

Now, if you list the methods on the BoredActivity companion object as shown in the image below, you’ll notice a new serializer() on it that returns an instance of KSerializer. You do not need to modify the code in this step, so you can remove the init method after you’ve observed the method list.

For most use cases, that’s all you need to do. However, the library offers several customization options and utilities for more advanced use cases. The next sections describe these features.

Encoding Data Manually

You can use the auto-generated serializer() to gain access to a class’s serializer. Then you can use it with the JSON encoding module to manually serialize or deserialize data as shown in the code example below:

import kotlinx.serialization.json.Json
import kotlinx.serialization.Serializable

@Serializable
data class PlatformInfo(
  val platformName: String,
  val apiLevel: Int
)

fun main() {
  val lollipop = PlatformInfo("Lollipop", 21)
  val json = Json.encodeToString(PlatformInfo.serializer(), lollipop)
  println(json) // {"platformName":"Lollipop","apiLevel":21}
}

You could also access a class’s serializer by using the top-level generic serializer() as shown in the next example:

import kotlinx.serialization.serializer

val lollipop = PlatformInfo("Lollipop", 21)
val json = Json.encodeToString(serializer<PlatformInfo>(), lollipop)
println(json) // {"platformName":"Lollipop","apiLevel":21}

Serializing Composite Types

The library can serialize all primitive Kotlin values out of the box, including Boolean, Byte, Short, Int, Long, Float, Double, Char and String. It also supports composite types based on these primitives, such as List, Map, Set, Pair and Triple. enums work automatically, too!

You can access the serializer for composite types based on custom types by using the base serializer and passing to it the custom type’s serializer. For example, to serialize List, you can use ListSerializer along with PlatformInfo.serializer():

val platforms: List<PlatformInfo> = listOf(...)
val platformsSerializer = ListSerializer(PlatformInfo.serializer())

Similarly, you can use SetSerializer and MapSerializer when needed. If you’re unsure how to construct your serializer, you can always use the top level serializer function as illustrated earlier.

Customizing Property Names

Many encoding formats use snake case variable names to represent data. To model such data with a Kotlin class, you need to break the language’s camel-case style convention. For example, consider the following example of JSON data:

{
  "platform_name": "Android",
  "api_level": 30
}

To model such an object, you use the @SerialName annotation instead as shown below:

import kotlinx.serialization.SerialName

@Serializable
data class PlatformInfo(
  @SerialName("platform_name")
  val platformName: String,
  @SerialName("api_level")
  val apiLevel: Int
)

The @SerialName annotation lets you specify a custom name for the encoded property in a serializable class. It tells the library to map the value of an encoded object’s platform_name field into the PlatformInfo class’s platformName field, and vice versa.

Marking Transient Data

If your model class contains properties that you must not serialize, annotate them with @Transient. A transient property must have a default value. Consider the code example below:

import kotlinx.serialization.Transient

@Serializable
data class PlatformInfo(
  // ...
  @Transient
  val isCurrent: Boolean = apiLevel == Build.VERSION.SDK_INT
)

isCurrent is annoted as @Transient and assigned a default initial value.

Transient properties are neither serialized into encoded output nor read from decoded input. Consider the example below that utilizes PlatformInfo:

val lollipop = PlatformInfo("Lollipop", 21)
println(lollipop) // PlatformInfo(platformName=Lollipop, apiLevel=21, isCurrent=false)

val json = Json.encodeToString(PlatformInfo.serializer(), lollipop)
println(json) // {"platform_name":"Lollipop","api_level":21}

The above code creates a PlatformInfo object and prints it, the object includes all three properties. Then, the serializer() is used to encode the object into JSON. The transient property is not encoded or included in the JSON.

That’s a lot of information about building and customizing serializers! Now it’s time to put it to use. In the next section, you’ll learn how to add Retrofit to the mix.

Integrating With Retrofit

In its current state, the app uses static, preprogrammed data. In this section, you’ll use Retrofit with the Kotlin Serialization library to fetch BoredActivity objects from the Bored API, which uses JSON to communicate requests and responses.

The app uses a repository to supply BoredActivity objects to its viewmodels. The repository relies on BoredActivityDataSource to fetch those objects. The starter app ships with two implementations of this interface:

• FakeDataSource: Returns static data hard coded into the app. It doesn’t communicate with the Bored API.
• RealDataSource: Communicates with the Bored API using Retrofit.

In its current state, the app uses the fake data source by default. It’s unsafe to use the real data source right now, as Retrofit wouldn’t know how to parse the JSON responses returned by the API.

To fix this, you need to give Retrofit the ability to handle JSON responses.

Adding the Retrofit Converter for Kotlin Serialization

Retrofit uses a pluggable system for serializing API requests and responses. It delegates this responsibility to a set of Converter objects that transform data into whatever format applies to the API.

For your app, you’ll use the retrofit2-kotlinx-serialization-converter library by Jake Wharton. It lets Retrofit use the Kotlin Serialization library to convert API requests and responses.

First, open your app module’s build.gradle and add the following dependency:

implementation "com.jakewharton.retrofit:retrofit2-kotlinx-serialization-converter:0.8.0"

Sync the project to download the dependency. Then open Module.kt in data. It contains a Dagger module named ApiModule that provides various dependencies, including Retrofit. Replace retrofit with the following:

@Provides
@ExperimentalSerializationApi
fun retrofit(okHttpClient: OkHttpClient): Retrofit {
  val contentType = "application/json".toMediaType()
  val converterFactory = Json.asConverterFactory(contentType)
  return Retrofit.Builder()
    .client(okHttpClient)
    .addConverterFactory(converterFactory)
    .baseUrl("https://www.boredapi.com/api/")
    .build()
}

The above code creates converterFactory as a converter factory that uses JSON and adds it to the Retrofit instance. Now that Retrofit can communicate with the API through JSON objects, it’s safe to switch to the source of real data.

Switching to the Real Data Source

In the same Module.kt, you’ll find another Dagger module named DataModule. It binds an instance of FakeDataSource to BoredActivityDataSource.

Modify boredActivityDataSource to bind an instance of RealDataSource instead:

@Binds
fun boredActivityDataSource(realDataSource: RealDataSource): BoredActivityDataSource

Build and run. Now, you’ll see data from the real API. Pull down to refresh the list of activities to get new suggestions every time!

Writing Serializers Manually

While the auto-generated serializers work well in most cases, you can provide your own implementations if you wish to customize the serialization logic. In this section, you’ll learn how to write a serializer manually.

A serializer implements the KSerializer<T> interface. It’s generic type parameter specifies the type of object serialized by the serializer.

Open BoredActivity.kt in data. Below BoredActivity, add a new class:

import kotlinx.serialization.KSerializer
// ...
class BoredActivitySerializer: KSerializer<BoredActivity>

This new class implements KSerializer.

The compiler will complain about missing methods in the class. You need to implement two methods, serialize and deserialize, and one property, descriptor.

You’ll work on descriptor first.

Writing a Descriptor

As you might guess from its name, descriptor describes the structure of the object being serialized. It contains a description of the type and names of the properties to serialize.

Add this property to BoredActivitySerializer:

import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.descriptors.element
// ...
class BoredActivitySerializer: KSerializer<BoredActivity> {
  override val descriptor: SerialDescriptor = buildClassSerialDescriptor("BoredActivity") {
    element<String>("activity")
    element<String>("type")
    element<Int>("participants")
    element<Double>("price")
    element<String>("link")
    element<String>("key")
    element<Double>("accessibility")
  }
}

This descriptor describes the object as a collection of seven primitive properties. It specifies their types as well as their serialized names.

Next, you’ll work with serialize.

Writing the Serialize Method

Now, you’ll add an implementation of serialize below descriptor:

import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.encoding.encodeStructure
// ...
class BoredActivitySerializer: KSerializer<BoredActivity> {
  // ...
  
  override fun serialize(encoder: Encoder, value: BoredActivity) 
  {
     encoder.encodeStructure(descriptor) {
       encodeStringElement(descriptor, 0, value.activity)
       encodeStringElement(descriptor, 1, value.type)
       encodeIntElement(descriptor, 2, value.participants)
       encodeDoubleElement(descriptor, 3, value.price)
       encodeStringElement(descriptor, 4, value.link)
       encodeStringElement(descriptor, 5, value.key)
       encodeDoubleElement(descriptor, 6, value.accessibility)
    }
  }

It accepts an encoder and an instance of BoredActivity. Then it uses encodeXYZElement to write the object’s properties one by one into the encoder, where XYZ is a primitive type.

Note the integer values passed into each encodeXYZElement. These values describe the order of properties. You use them when deserializing an object.

Finally, you’ll add deserialize.

Writing the Deserialize Method

Add an implementation of the deserialize right below serialize:

import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.decodeStructure
import kotlinx.serialization.encoding.CompositeDecoder
// ...
class BoredActivitySerializer: KSerializer<BoredActivity> {  
  // ...
  
  override fun deserialize(decoder: Decoder): BoredActivity = 
      decoder.decodeStructure(descriptor) {
    var activity = ""
    var type = ""
    var participants = -1
    var price = 0.0
    var link = ""
    var key = ""
    var accessibility = 0.0

    while (true) {
      when (val index = decodeElementIndex(descriptor)) {
        0 -> activity = decodeStringElement(descriptor, 0)
        1 -> type = decodeStringElement(descriptor, 1)
        2 -> participants = decodeIntElement(descriptor, 2)
        3 -> price = decodeDoubleElement(descriptor, 3)
        4 -> link = decodeStringElement(descriptor, 4)
        5 -> key = decodeStringElement(descriptor, 5)
        6 -> accessibility = decodeDoubleElement(descriptor, 6)
        CompositeDecoder.DECODE_DONE -> break
        else -> error("Unexpected index: $index")
      }
    }
    BoredActivity(activity, type, participants, price, 
                  link, key, accessibility)
  }
}

It accepts a decoder, and returns an instance of BoredActivity.

Note the iteration over index returned by the decoder, which you use to determine which property to decode next. Also, notice that the loop terminates whenever the index equals a special token called CompositeDecoder.DECODE_DONE. This signals a decoder has no more properties to read.

Now that the serializer is complete, it’s time to wire it with the BoredActivity class.

Connecting the Serializer to the Class

To use BoredActivitySerializer, pass it as a parameter to @Serializable as follows:

@Serializable(with = BoredActivitySerializer::class)
data class BoredActivity(
  val activity: String,
  val type: String,
  val participants: Int,
  val price: Double,
  val link: String,
  val key: String,
  val accessibility: Double,
)

Build and run. You won’t notice any changes, which indicates your serializer works correctly! Add a log statement in your serializer to confirm that it’s being used. For example, add the following to the top of deserializer:

Log.d("BoredActivitySerializer","Using deserializer")

With this change, you complete Bored No More!. Don’t forget to try its suggestions the next time you’re feeling bored. :]

Bonus: Tests

The app ships with a few tests to ensure your serializers and viewmodels work correctly. Don’t forget to run them to ensure that everything is alright! To run the tests, in the Project pan in Android Studio, right click com.raywenderlich.android.borednomore (test). Then select Run Tests in com.raywenderlich…:

The results of the tests look like:

While the Kotlin Serialization library is great, every technology has its drawbacks. This tutorial would be incomplete if it didn’t highlight the library’s limitations. Keep reading to learn about them.

Limitations

The Kotlin Serialization library is opinionated about its approach to data serialization. As such, it imposes a few restrictions on how you write your code. Here’s a list of a few important limitations:

• Non-class properties aren’t allowed in the primary constructor of a serializable class.

  // Invalid code
  @Serializable 
  class Project(
    path: String // Forbidden non-class property
  ) {
    val owner: String = path.substringBefore('/')    
    val name: String = path.substringAfter('/')    
  }
  

• Only class properties with a backing field are serialized while the others are ignored.

  @Serializable 
  class Project(
    var name: String // Property with a backing field; allowed
  ) {
    var stars: Int = 0 // property with a backing field; allowed
    
    val path: String // no backing field; ignored by the serializer
      get() = "kotlin/$name"                                         
    
    var id by ::name // delegated property; ignored by the serializer
  }
  

• Optional properties must have a default value if they’re missing in the encoded data.

  @Serializable 
  data class Project(val name: String, val language: String)
  

  Deserializing the following JSON into a Project object…

  { "name" : "kotlinx.serialization" }
  

  …produces a MissingFieldException:

  Exception in thread "main" kotlinx.serialization.MissingFieldException: Field 'language' is required for type with serial name 'Project', but it was missing.
  

• All referenced objects in class properties must also be serializable.

  class User(val name: String)
    
  @Serializable
  class Project(
    val name: String, 
    val owner: User // Invalid code, User class is not serializable
  )
  

With that, you’re done with this tutorial!

Where to Go From Here?

You can download the final version of this project by clicking Download Materials at the top or bottom of this tutorial.

Congratulations! You’ve learned how to use the Kotlin Serialization library. If you’re wondering where to go next, here’s a list of things to try:

• The Kotlin Serialization library’s official Serialization Guide.
• YAML support for the Kotlin Serialization library.
• Deep Dive Into Kotlin Data Classes for Android.

We hope you enjoyed this tutorial. If you have any questions or comments, please join the forum discussion below!