Cinchoo ETL - Deserializing JSON to Polymorphic Classes without Type Information

1. Introduction

ChoETL is an open source ETL (extract, transform and load) framework for .NET. It is a code based library for extracting data from multiple sources, transforming, and loading into your very own data warehouse in .NET environment. You can have data in your data warehouse in no time.

This tip talks about generating CSV file from JSON format using Cinchoo ETL framework. It is very simple to use, with few lines of code, the conversion can be done. You can convert large files as the conversion process is stream based, quite fast and with low memory footprint.

2. Requirement

This framework library is written in C# using .NET 4.5 / .NET Core Frameworks.

3. How to Use

3.1 Sample Data

Let's begin by looking into a simple example of converting the below JSON input file, containing Gallery Item and Gallery Album objects represented in JSON.

Listing 3.1.1. Sample JSON Data Input File (Img.json)

[
  {
    "id": "OUHDm",
    "title": "My most recent drawing. Spent over 100 hours.",
    "is_album": false
  },
  {
    "id": "lDRB2",
    "title": "Imgur Office",
    "is_album": true,
    "images_count": 3,
    "images": [
      {
        "id": "24nLu",
        "link": "http://i.imgur.com/24nLu.jpg"
      },
      {
        "id": "Ziz25",
        "link": "http://i.imgur.com/Ziz25.jpg"
      },
      {
        "id": "9tzW6",
        "link": "http://i.imgur.com/9tzW6.jpg"
      }
    ]
  }
]

You can see there is no type information available to deserialize automatically. 'is_album' property is used to deserialize the JSON to either Gallery Item (if 'is_album' is false) or Gallery Album objects (if 'is_album' is true).

.NET Framework

Install-Package ChoETL.JSON

.NET Core

Install-Package ChoETL.JSON.NETStandard

Now add ChoETL namespace to the program.

using ChoETL;

3.2 Deserialization (Approach 1)

Here are the steps to follow to successfully deserialize the JSON into polymorphic classes. In this approach, you take full control of scanning the node and choose the type to deserialize based on specific node value. 

First, define classes to match the JSON as below.

Listing 3.2.1. POCO Class Tree

public interface IGalleryItem
{
}

public class GalleryItem : IGalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public bool is_album { get; set; }
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}

public class GalleryImage
{
    public string id { get; set; }
    public string link { get; set; }
}

Where defined two concrete classes, GalleryItem and GalleryAlbum derived from IGalleryItem interface to hold JSON contents.

Next use Cinchoo ETL, to deserialize the JSON as below.

Listing 3.2.2. Deserialization

private static void QuickConversion()
{
    using (var r = new ChoJSONReader<IGalleryItem>("img.json")
        .Configure(c => c.SupportsMultiRecordTypes = true)
        .Configure(c => c.RecordTypeSelector = o =>
        {
            dynamic kvp = o as dynamic;
            dynamic jObject = kvp.Item2 as dynamic;
            switch ((bool)jObject.is_album)
            {
                case true:
                    return typeof(GalleryAlbum);
                default:
                    return typeof(GalleryItem);
            }
        })
        )
    {
        foreach (var rec in r)
            Console.Write(rec.Dump());
    }
}

Create an instance of ChoJSONReader object for reading img.json file. Use the below configuration setup to parse JSON successfully.

• RecordTypeSelector - function to choose the record type to deserialize based on is_album property value. This callback is passed with Tuple<int, object> parameter, where item1 is node index, item2 is JObject. From item2, you can do check on is_album property value to determine the class type to be serialized.
• SupportsMultiRecordTypes - true, to tell the parser to handle polymorphic deserialization.

Sample fiddle: https://dotnetfiddle.net/IRZOu3

3.3 Deserialization (Approach 2)

In this approach, bit simplified way to choosing the subtype to deserialize based on discriminator (ie. is_album) field.

First, define classes to match the JSON as below.

Listing 3.3.1. POCO Class Tree

public interface IGalleryItem
{
}

public class GalleryItem : IGalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public bool is_album { get; set; }
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}

public class GalleryImage
{
    public string id { get; set; }
    public string link { get; set; }
}

Where defined two concrete classes, GalleryItem and GalleryAlbum derived from IGalleryItem interface to hold JSON contents.

Next use Cinchoo ETL, to deserialize the JSON as below.

Listing 3.3.2. Deserialization

private static void QuickConversion()
{
    using (var r = new ChoJSONReader<IGalleryItem>("img.json")
        .Configure(c => c.KnownTypeDiscriminator = "is_album")
        .Configure(c => c.RecordTypeSelector = o =>
        {
            var isAlbum = o.CastTo<bool>();
            return !isAlbum ? typeof(GalleryAlbum) : typeof(GalleryItem);
        })
        .Configure(c => c.SupportsMultiRecordTypes = true)
        )
    {
        r.Print();
    }
}

Create an instance of ChoJSONReader object for reading img.json file. Use the below configuration setup to parse JSON successfully.

• KnownTypeDiscriminator - Specify discriminator field (is_album)
• RecordTypeSelector - function to choose the record type to deserialize based on is_album property value. This callback is passed with Tuple<int, object> parameter, where item1 is node index, item2 is discriminator value. From item2, you can do check on is_album property value to determine the class type to be serialized.
• SupportsMultiRecordTypes - true, to tell the parser to handle polymorphic deserialization.

Sample fiddle: https://dotnetfiddle.net/xGMMIt

3.4 Deserialization (Approach 4)

In this approach, simplified way to choosing the polymorphic subtype by defining class attributes to base abstract / interface class.

First, define classes to match the JSON as below.

Listing 3.4.1. POCO Class Tree

[ChoKnownTypeDiscriminator("is_album")]
[ChoKnownType(typeof(GalleryAlbum), "true")]
[ChoKnownType(typeof(GalleryItem), "false")]
public interface IGalleryItem
{
}

public class GalleryItem : IGalleryItem
{
    public string id { get; set; }
    public string title { get; set; }
    public bool is_album { get; set; }
}

public class GalleryAlbum : GalleryItem
{
    public int images_count { get; set; }
    public List<GalleryImage> images { get; set; }
}

public class GalleryImage
{
    public string id { get; set; }
    public string link { get; set; }
}

Where defined two concrete classes, GalleryItem and GalleryAlbum derived from IGalleryItem interface to hold JSON contents.

Use ChoKnownTypeDiscriminatorAttribute to define the discriminator field. Use ChoKnownTypeAttribute to define the polymorpic subtypes for matching discriminator.

Next use Cinchoo ETL, to deserialize the JSON as below.

Listing 3.4.2. Deserialization

private static void QuickConversion()
{
    using (var r = new ChoJSONReader<IGalleryItem>("img.json")
        .Configure(c => c.SupportsMultiRecordTypes = true)
        )
    {
        r.Print();
    }
}

Create an instance of ChoJSONReader object for reading img.json file. Use the below configuration setup to parse JSON successfully.

• SupportsMultiRecordTypes - true, to tell the parser to handle polymorphic deserialization.

Sample fiddle: https://dotnetfiddle.net/mCl4NJ

Please refer to other similar articles for conversion of JSON to CSV:

For more information about Cinchoo ETL, please visit the other CodeProject articles:

• Cinchoo ETL - Articles

History

• 4th October, 2021: Initial version