How to Write Fewer Lines of Code with the OpenAPI Generator

Hey! I'll start with the main thing - I'm a lazy person. I'm a very, very lazy developer. I have to write a lot of code - both for the backend and for the frontend. And my laziness constantly torments me, saying: You could not write this code, but you write ... This is how we live.

But what to do? How can you get rid of the need to write at least part of the code?

There are many approaches to solving this problem. Let's take a look at some of them.

OpenAPI

Let's say your backend is a collection of REST services. The first place to start is to study your backend's documentation in the hope of stumbling upon the OpenAPI specification. The ideal situation would be when your backing provides the most complete spec, which will describe all the methods used by clients, as well as all transmitted and received data and possible errors.

In fact, I am writing these lines and I think this is self-evident: it seems obvious that if you are developing an API, then there must be a specification. Not in the form of a simple enumeration of methods, but as complete as possible - and, most importantly, generated from code and not written by hand. But this is not the case everywhere, so we must strive for the best.

Well, ok, so we found our spec; it is full-fledged, without dark spots. Great - it's almost done. Now it remains to use it to achieve our insidious goals 😈. It just so happens that I also write applications on Flutter and as a client, I would consider it but the approach used here is also suitable for web clients (and for any others, there is also something to use).

Client-initiated generation

I think it will not be a revelation that there is no magic. For a feature to appear, some code must appear anyway. And yes, we will not write it, but there will be a code generator. And this is where the fun begins. There are libraries for Flutter (and not only for it), which will generate code for working with the backend based on annotations that you can throw on pseudo-services (which you still have to write).

It looks something like this:

import 'package:dio/dio.dart';
import 'package:json_annotation/json_annotation.dart';
import 'package:retrofit/retrofit.dart';

part 'example.g.dart';

@RestApi(baseUrl: "https://5d42a6e2bc64f90014a56ca0.mockapi.io/api/v1/")
abstract class RestClient {
  factory RestClient(Dio dio, {String baseUrl}) = _RestClient;

  @GET("/tasks/{id}")
  Future<Task> getTask(@Path("id") String id);

  @GET('/demo')
  Future<String> queries(@Queries() Map<String, dynamic> queries);

  @GET("https://httpbin.org/get")
  Future<String> namedExample(@Query("apikey") String apiKey, @Query("scope") String scope, @Query("type") String type, @Query("from") int from);

  @PATCH("/tasks/{id}")
  Future<Task> updateTaskPart(@Path() String id, @Body() Map<String, dynamic> map);

  @PUT("/tasks/{id}")
  Future<Task> updateTask(@Path() String id, @Body() Task task);

  @DELETE("/tasks/{id}")
  Future<void> deleteTask(@Path() String id);

  @POST("/tasks")
  Future<Task> createTask(@Body() Task task);

  @POST("http://httpbin.org/post")
  Future<void> createNewTaskFromFile(@Part() File file);

  @POST("http://httpbin.org/post")
  @FormUrlEncoded()
  Future<String> postUrlEncodedFormData(@Field() String hello);
}

@JsonSerializable()
class Task {
  String id;
  String name;
  String avatar;
  String createdAt;

  Task({this.id, this.name, this.avatar, this.createdAt});

  factory Task.fromJson(Map<String, dynamic> json) => _$TaskFromJson(json);

  Map<String, dynamic> toJson() => _$TaskToJson(this);
}

After starting the generator, we will get a working service ready to use:

import 'package:logger/logger.dart';
import 'package:retrofit_example/example.dart';
import 'package:dio/dio.dart';

final logger = Logger();
void main(List<String> args) {
  final dio = Dio(); // Provide a dio instance
  dio.options.headers["Demo-Header"] = "demo header";
  final client = RestClient(dio);
  
  client.getTasks().then((it) => logger.i(it));
}

This method (applicable to all types of clients) can save you a lot of time. If your backing does not have a normal OpenAPI scheme, then you sometimes do not have a very large choice. However, if there is a high-quality scheme, then in comparison with that method code generation (which we will talk about further) the current version has several disadvantages:

• You still need to write code - less than before, but a lot

• You must independently track changes in the backend and change the code you write after them

It is worth dwelling on the last point in a little more detail - if (when) changes occur on the back in the methods that are already used in your application, then you need to track these changes yourself, modify the DTO models, and, possibly, the endpoint. Also, if for some incredible reason backward-incompatible changes to the method occur, then you will learn about this only at runtime (at the time of calling this method) - which may not happen during development (especially if you do not have or do not have enough tests) and then you will have an extremely unpleasant bug in the production.

A generation without "fog of war"

Have not you forgotten that we have a high-quality OpenAPI scheme? Fine! The whole battlefield is open to you and there is no point in groping (* I added this phrase in order to somehow justify the title of this block, which, with a scratch, you have to invent yourself; generation will not help here *). Then you should pay attention to those tools that the entire OpenAPI ecosystem offers in principle!

Of all the variety of hammers and microscopes, we are now interested in only one. And its name is OpenAPI Generator. This rasp allows you to generate code for any language (well, almost), as well as for both clients and the server (to make a mock server, for example).

Let's get to practice already:

As a diagram, we will take what the Swagger demo offers. Next, we need to install the generator itself. Here is a great tutorial for that. If you are reading this article, then with a high degree of probability you already have Node.js installed, which means that one of the easiest installation methods would be to use npm-versions.

The next step is the generation itself. There are a couple of ways to do this:

• Using a pure console command

• Using a command in conjunction with a config file

In our example, the 1st method will look like this:

openapi-generator-cli generate -i https://petstore.swagger.io/v2/swagger.json -g dart-dio -o .pet_api --additional-properties pubName=pet_api

An alternative way is to describe the parameters in the openapitools.json file, for example:

{
  "$schema": "node_modules/@openapitools/openapi-generator-cli/config.schema.json",
  "spaces": 2,
  "generator-cli": {
    "version": "5.1.1",
    "generators": {
      "pet": {
        "input-spec": "https://petstore.swagger.io/v2/swagger.json",
        "generator-name": "dart-dio",
        "output": ".pet_api",
        "additionalProperties": {
          "pubName": "pet_api"
        }
      }
    }
  }
}

And then running the command:

openapi-generator-cli generate

A complete list of available parameters for Dart is presented here. And for any other generator, the list of these parameters can be found by running the following console command:

# <generator-name>, dart-dio - for example
openapi-generator-cli config-help -g dart-dio

Even if you choose the complete console option, after the first start of the generator, you will have a configuration file with the version of the generator used written in it, as in this example - 5.1.1. In the case of Dart / Flutter, this version is very important, since each of them can carry certain changes, including those with backward incompatibility or interesting effects.

So, since version 5.1.0, the generator uses null-safety, but implements this through explicit checks, and not the capabilities of the Dart language itself (for now, unfortunately). For example, if in your schema some of the model fields are marked as required, then if your backend returns a model without this field, then an error will occur in runtime.

flutter: Deserializing '[id, 9, category, {id: 0, name: cats}, photoUrls, [string], tags, [{id: 0, na...' to 'Pet' failed due to: Tried to construct class "Pet" with null field "name". This is forbidden; to allow it, mark "name" with @nullable.

And all because the name field of the Pet model is explicitly specified as required, but is absent in the request response:

{
  "Pet": {
    "type": "object",
    "required": [
      "name", // <- required field
      "photoUrls" // <- and this too
    ],
    "properties": {
      "id": {
        "type": "integer",
        "format": "int64"
      },
      "category": {
        "$ref": "#/definitions/Category"
      },
      "name": {
        "type": "string",
        "example": "doggie"
      },
      "photoUrls": {
        "type": "array",
        "xml": {
          "wrapped": true
        },
        "items": {
          "type": "string",
          "xml": {
            "name": "photoUrl"
          }
        }
      },
      "tags": {
        "type": "array",
        "xml": {
          "wrapped": true
        },
        "items": {
          "xml": {
            "name": "tag"
          },
          "$ref": "#/definitions/Tag"
        }
      },
      "status": {
        "type": "string",
        "description": "pet status in the store",
        "enum": [
          "available",
          "pending",
          "sold"
        ]
      }
    },
    "xml": {
      "name": "Pet"
    }
  },
  "Category": {
    "type": "object",
    "properties": {
      "id": {
        "type": "integer",
        "format": "int64"
      },
      "name": {
        "type": "string"
      }
    },
    "xml": {
      "name": "Category"
    }
  },
  "Tag": {
    "type": "object",
    "properties": {
      "id": {
        "type": "integer",
        "format": "int64"
      },
      "name": {
        "type": "string"
      }
    },
    "xml": {
      "name": "Tag"
    }
  }
}

Well, the generator is running - the job is done, but there are a few simple steps left in which we hardly have to write code (for the sake of this, everything was started!). The standard openapi-generator will generate only the basic code, which uses libraries that already rely on code generation by means of Dart itself. Therefore, after completing the basic generation, you need to start Dart / Flutter:

cd .pet_api
flutter pub get  
flutter pub run build_runner build --delete-conflicting-outputs

At the output, we get a ready-made package, which will be located where you specified in the configuration file or console command. It remains to include it in pubspec.yaml:

name: openapi_sample
description: Sample for OpenAPI
version: 1.0.0
publish_to: none

environment:
  flutter: ">=2.0.0"
  sdk: ">=2.12.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter
  pet_api: # <- our generated library
    path: .pet_api

And use this library as follows:

import 'package:dio/dio.dart';
import 'package:pet_api/api/pet_api.dart';
import 'package:pet_api/model/pet.dart';
import 'package:pet_api/serializers.dart'; // <- we must use [standartSerializers] from this package module

Future<Pet> loadPet() async {
  final Dio dio = Dio(BaseOptions(baseUrl: 'https://petstore.swagger.io/v2'));
  final PetApi petApi = PetApi(dio, standardSerializers);
  const petId = 9;
  final Response<Pet> response = await petApi.getPetById(petId, headers: <String, String>{'Authorization': 'Bearer special-key'});
  return response.data;
}

An important part of it is the need to write down which serializers we will use in order for JSONs to turn into normal models. And also drop the Dio instances into the generated ... Api, specifying the base server URLs in them.

Dart nuances

It seems that this is all that can be said on this topic, but Dart recently received a major update, it added null-safety. And all packages are being actively updated, and projects are migrating to a new version of the language, more resistant to our errors.

However, at the moment, the generator does not support this new version and in several directions at once:

• Language version in the package (in the latest version of the generator - 5.1.1, Dart 2.7.0 is used)

• Outdated packages

• Backward incompatibility of some of the packages used (in the current version of Dio, some methods have different names)

name: pet_api
version: 1.0.0
description: OpenAPI API client

environment:
  sdk: '>=2.7.0 <3.0.0' # -> '>=2.12.0 <3.0.0'

dependencies:
  dio: '^3.0.9' # Actual -> 4.0.0
  built_value: '>=7.1.0 <8.0.0' # -> 8.1.0
  built_collection: '>=4.3.2 <5.0.0' # -> 5.1.0

dev_dependencies:
  built_value_generator: '>=7.1.0 <8.0.0' # -> 8.1.0
  build_runner: any # -> 2.0.5
  test: '>=1.3.0 <1.16.0' # -> 1.17.9

And this can cause several problems at once - if you have already switched to Flutter 2.0+ and Dart 2.12+, then in order to start the code generation of the second stage (which is on Dart), you will have to switch the language to the old version, FVM allows you to do this pretty quickly, but it's still an inconvenience.

The second disadvantage is that this generated API package is now a legacy dependency, which will prevent your new project from starting with sound-null-safety. You will be able to take advantage of null-safety when writing code, but you will not be able to check and optimize runtime, and the project will only work if you use the additional Flutter parameter: --no-sound-null-safety.

There are three options for correcting this situation:

• Make a pull request with the openapi-generator update

• Wait until someone else does it - in half a year it will most likely happen

• Correct the generated code so that it now becomes sound-null-safety

The third point sounds like we will have to write code ... It will have to be a little, but not the same.

Before starting our manipulations, I will show you the bash script that has turned out at the moment and which runs all our code generation logic:

openapi-generator-cli generate  
cd .pet_api || exit 
flutter pub get  
flutter pub run build_runner build --delete-conflicting-outputs

This script also relies on the configuration file we discussed above. Let's update this script so that it immediately updates all the dependencies of our generated package:

openapi-generator-cli generate  
cd .pet_api || exit  
echo "name: pet_api  
version: 1.0.0  
description: OpenAPI API client  
  
environment:  
 sdk: '>=2.12.0 <3.0.0'  
dependencies:  
 dio: ^4.0.0 built_value: ^8.1.0 built_collection: ^5.1.0  
dev_dependencies:  
 built_value_generator: ^8.1.0 build_runner: ^2.0.5 test: ^1.17.9" > pubspec.yaml  
flutter pub get  
flutter pub run build_runner build --delete-conflicting-outputs

Now - our generator will start correctly with the new version of Dart (>2.12.0) in the system. Everything would be fine, but we still won't be able to use our api package! First, the generated code is replete with annotations that bind it to the old version of the language:

//  
// AUTO-GENERATED FILE, DO NOT MODIFY!  
//  
// @dart=2.7  <--
  
// ignore_for_file: unused_import

And secondly, there is a backward incompatibility in the logic of Dio and packages that are used to serialize / deserialize models. Let's fix it!

To fix it, we need to write quite a bit of utility code that will fix the incompatibilities that appear in our generated code. I mentioned above that I would advise installing the generator via npm, as the easiest way, if you have Node.js, respectively, by inertia - and the utility code will be written in JS. If desired, it is easy to rewrite it in Dart, if you do not have Node.js and do not want to mess with it.

Let's take a look at these simple manipulations:

const fs = require('fs');
const p = require('path');

const dartFiles = [];

function main() {
  const openapiDirPath = p.resolve(__dirname, '.pet_api');
  searchDartFiles(openapiDirPath);
  for (const filePath of dartFiles) {
    fixFile(filePath);
    console.log('Fixed file:', filePath);
  }
}

function searchDartFiles(path) {
  const isDir = fs.lstatSync(path).isDirectory();
  if (isDir) {
    const dirContent = fs.readdirSync(path);
    for (const dirContentPath of dirContent) {
      const fullPath = p.resolve(path, dirContentPath);
      searchDartFiles(fullPath);
    }
  } else {
    if (path.includes('.dart')) {
      dartFiles.push(path);
    }
  }
}

function fixFile(path) {
  const fileContent = fs.readFileSync(path).toString();
  const fixedContent = fixOthers(fileContent);
  fs.writeFileSync(path, fixedContent);
}

const fixOthers = fileContent => {
  let content = fileContent;
  for (const entry of otherFixers.entries()) {
    content = content.replace(entry[0], entry[1]);
  }
  return content;
};

const otherFixers = new Map([
  // ? Base fixers for Dio and standard params
  [
    '// @dart=2.7',
    '// ',
  ],
  [
    /response\.request/gm,
    'response.requestOptions',
  ],
  [
    /request: /gm,
    'requestOptions: ',
  ],
  [
    /Iterable<Object> serialized/gm,
    'Iterable<Object?> serialized',
  ],
  [
    /(?<type>^ +Uint8List)(?<value> file,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +String)(?<value> additionalMetadata,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +ProgressCallback)(?<value> onReceiveProgress,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +ProgressCallback)(?<value> onSendProgress,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +ValidateStatus)(?<value> validateStatus,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +Map<String, dynamic>)(?<value> extra,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +Map<String, dynamic>)(?<value> headers,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +CancelToken)(?<value> cancelToken,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(@nullable\n)(?<annotation>^ +@.*\n)(?<type>.*)(?<getter> get )(?<variable>.*\n)/gm,
    '$<annotation>$<spaces>$<type>?$<getter>$<variable>',
  ],
  [
    'final result = <Object>[];',
    'final result = <Object?>[];',
  ],
  [
    'Iterable<Object> serialize',
    'Iterable<Object?> serialize',
  ],
  [
    /^ *final _response = await _dio.request<dynamic>\(\n +_request\.path,\n +data: _bodyData,\n +options: _request,\n +\);/gm,
    `_request.data = _bodyData;
    
    final _response = await _dio.fetch<dynamic>(_request);
    `,
  ],
  // ? Special, custom params for concrete API
  [
    /(?<type>^ +String)(?<value> apiKey,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +String)(?<value> name,)/gm,
    '$<type>?$<value>',
  ],
  [
    /(?<type>^ +String)(?<value> status,)/gm,
    '$<type>?$<value>',
  ],
]);

main();

Most of all these regulars fix the basic logic of the generated code, but there are three custom ones that are needed for a specific API. Each specific one will have its own custom regulars, but it is very likely that adding them will not be difficult, and all basic ones will work on any API.

Conclusions

The approach to generating client code, in the presence of a high-quality OpenAPI scheme, is an extremely simple task, regardless of the client's language. In the case of Dart, there are still certain inconveniences caused, especially, by the transition period to * null-safety *. But as part of this article, we have successfully overcome all the troubles and got a fully functional library for working with the backend, the dependencies of which (and itself) have been updated to the newest version and can be used in a Flutter project with sound-null-safety without any restrictions.

An additional plus of the approach when the source of truth is the schema - if it changes with the loss of backward compatibility, our generated code will immediately react to this and show all errors at the stage of static analysis, which will save your nerves from catching bugs at runtime.