How to Measure Your Type Coverage

2022-11-07

That's where test coverage gives us a hint. Which project would you join if you could pick: the one with 20 % test coverage or the one with 80 % test coverage? I'd always go with the latter, as tests give great confidence.

Yet, tests are not the only thing that can help us access code quality quickly. With PHP 7.0, 7.4, and 8.0, type declarations became a sign of project health. But how can we measure those?

Do you measure your type declaration completeness with @phpstan already?

You should 😉

It's such a great and safe feeling to see 99 % type-coverage 😎 pic.twitter.com/cYyDVYKqG8

— Tomas Votruba 🇺🇦 (@VotrubaT) October 26, 2022

What is the "Type Coverage"?

If the test coverage is % of all possible code runs, what is the "type coverage" then?

function run($name)
{
    return $name;
}

Here we have 1 param and 1 function return. That's 2 possible type declarations that we're missing:

• 0/2 = 0 % type coverage

How can we increase it? We add a type declaration into the function param:

function run(string $name)
{
    return $name;
}

Here we have 1 param with type declaration, and 1 return without it.

• 1/2 = 50 % type coverage

How do we get to 100 %? Exactly, we add the return type declaration:

function run(string $name): string
{
    return $name;
}

We do the same for typed properties as well:

private $name;

private $surname;

private $age;

• 0/3 types are completed = 0 % type coverage

What about this code?

private string|Name $name;

private ?string $surname = null;

/**
 * @var callable
 */
private $addressCallable;

This code has 100 % type declaration coverage. How is that possible? Nullable, union, and callable docblock type declarations are valid and the most strict types.

Pretty simple, right?

We can run the type coverage check quickly with PHPStan on any project. Even if it's legacy or full of magic - no autoloading is needed.

I love this Metric, Because...

• it is fast - we know instantly the result
• it is simple - we know the value is between 0-100
• it is explanatory - we know the potential type is missing, and where exactly can we fix it
• it is code sustainability predictor - based on this number, we know how easy or complicated it will be to work with the codebase

3 Steps to Measure it?

The type coverage is measured by 3 custom PHPStan rules with 3 custom collectors. They work the same way as described above in the code sample.

1. Install the symplify/phpstan-rules package

composer require symplify/phpstan-rules --dev

The package is available on PHP 7.2+, as downgraded.

1. Add Rules to phpstan.neon

The easiest type declaration to add is a return one, then the param one. On the other hand, the typed property is available as late as PHP 7.4. That's why we have 3 different rules for them, with one collector per each:

services:
    -
        class: Symplify\PHPStanRules\Rules\Explicit\PropertyTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

    -
        class: Symplify\PHPStanRules\Rules\Explicit\ParamTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

    -
        class: Symplify\PHPStanRules\Rules\Explicit\ReturnTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

The minimalLevel argument defines minimal required type coverage in every rule. Notice the value 0.99, meaning at least 99 % type coverage is required. We'll get back to that later.

1. Add Collectors to phpstan.neon

At the moment, we've registered the rules, but they do not have any effect. We have to add collector services too:

services:
    -
        class: Symplify\PHPStanRules\Collector\FunctionLike\ParamTypeSeaLevelCollector
        tags: [phpstan.collector]
    -
        class: Symplify\PHPStanRules\Collector\FunctionLike\ReturnTypeSeaLevelCollector
        tags: [phpstan.collector]
    -
        class: Symplify\PHPStanRules\Collector\ClassLike\PropertyTypeSeaLevelCollector
        tags: [phpstan.collector]

Now run to see the results:

vendor/bin/phpstan

The failed error message is more than meets the eye. It shows you where you can complete the type declarations, so you can find them in the code and improve.

How to Find Your Current Type Coverage

Now we get back to the 0.99 resp. 99 % required type coverage. The CI fails on such a high value, but that's our intention. The error message actually tells us the current type coverage value:

Out of 81 possible param types, only 60 % actually have it. Add more param types to get over 99 %

In this case, we take the current value of 60 and put it into the config, so our codebase will remain on this code coverage:

 services:
     -
         class: Symplify\PHPStanRules\Rules\Explicit\ParamTypeDeclarationSeaLevelRule
         tags: [phpstan.rules.rule]
         arguments:
-            minimalLevel: 0.99
+            minimalLevel: 0.60

This value can be different for param, return, and property, so adjust it accordingly to make the CI pass.

Now we re-run PHPStan, and everything is fine. We commit, open pull-request, and merge.

Lean Type Coverage Improvement

Once a week, we run the same command again, trying to bump it 2-3 % (depending on your codebase size) and open a pull request. This way, we can improve the codebase gradually, without any big bang.

We also use these rules to monitor type coverage improvement on the project we work on. That way, both developers and the client knows we're going in the right direction.

Happy coding!