Using a Framework and the ZDT Test Suite

Preamble¶

# used to create block diagrams
%reload_ext xdiag_magic
%xdiag_output_format svg
    
import numpy as np                   # for multi-dimensional containers
import pandas as pd                  # for DataFrames
import platypus as plat              # multi-objective optimisation framework

Introduction¶

When preparing to implement multi-objective optimisation experiments, it's often more convenient to use a ready-made framework/library instead of programming everything from scratch. There are many libraries and frameworks that have been implemented in many different programming languages, but as we're using Python we will be selecting from frameworks such as DEAP, PyGMO, and Platypus.

With our focus on multi-objective optimisation, our choice is an easy one. We will choose Platypus which has a focus on multi-objective problems and optimisation.

Platypus is a framework for evolutionary computing in Python with a focus on multiobjective evolutionary algorithms (MOEAs). It differs from existing optimization libraries, including PyGMO, Inspyred, DEAP, and Scipy, by providing optimization algorithms and analysis tools for multiobjective optimization.

As a first look into Platypus, let's repeat the process covered in the earlier section on "Synthetic Objective Functions and ZDT1", where we randomly initialise a solution and then evaluate it using ZDT1.

%%blockdiag
{
    orientation = portrait
    "Problem Variables" -> "Test Function" -> "Objective Values"
    "Test Function" [color = '#ffffcc']
}

blockdiag{ orientation = portrait "Problem Variables" -> "Test Function" -> "Objective Values" "Test Function" [color = '#ffffcc'] }

Problem VariablesTest FunctionObjective Values

The ZDT test function¶

Similar to the last time, we will be using a synthetic test problem throughout this notebook called ZDT1. It is part of the ZDT test suite, consisting of six different two-objective synthetic test problems. This is quite an old test suite, easy to solve, and very easy to visualise.

Mathematically, the ZDT11 two-objective test function can be expressed as:

f1(x1)=x1f2(x)=g⋅hg(x2,…,xD)=1+9⋅∑d=2Dxd(D−1)h(f1,g)=1−f1/g(1) \begin{aligned} f_1(x_1) &= x_1 \tag{1} \\ f_2(x) &= g \cdot h \\ g(x_2,\ldots,x_{\mathrm{D}}) &= 1 + 9 \cdot \sum_{d=2}^{\mathrm{D}} \frac{x_d}{(D-1)}\\ h(f_1,g) &= 1 - \sqrt{f1/g} \end{aligned} f1(x1)f2(x)g(x2,…,xD)h(f1,g)=x1=g⋅h=1+9⋅d=2∑D(D−1)xd=1−f1/g(1)

where xxx is a solution to the problem, defined as a vector of DDD decision variables.

x=⟨x1,x2,…,xD⟩(2) x= \langle x_{1},x_{2},\ldots,x_{\mathrm{D}} \rangle \tag{2} x=⟨x1,x2,…,xD⟩(2)

and all decision variables fall between 000 and 111.

0≤xd≤1,d=1,…,D(3) 0 \le x_d \le 1, d=1,\ldots,\mathrm{D} \tag{3} 0≤xd≤1,d=1,…,D(3)

For this bi-objective test function, f1f_1f1 is the first objective, and f2f_2f2 is the second objective. This particular objective function is, by design, scalable up to any number of problem variables but is restricted to two problem objectives.

Let's start implementing this in Python, beginning with the initialisation of a solution according to Equations 2 and 3. In this case, we will have 30 problem variables D=30\mathrm{D}=30D=30.

D = 30
x = np.random.rand(D)
print(x)

[0.40700054 0.04071341 0.2019138  0.79142779 0.9140935  0.03581037
 0.62395881 0.85908972 0.59781233 0.12436693 0.67729958 0.31991784
 0.54294362 0.62353553 0.70098228 0.03701772 0.08176195 0.70208965
 0.40398321 0.41584089 0.71320878 0.46511698 0.28813617 0.86793829
 0.65494094 0.5299217  0.98657349 0.46780006 0.23001727 0.41083113]

Now that we have a solution to evaluate, let's implement the ZDT1 synthetic test function using Equation 1.

def ZDT1(x):
    f1 = x[0]  # objective 1
    g = 1 + 9 * np.sum(x[1:D] / (D-1))
    h = 1 - np.sqrt(f1 / g)
    f2 = g * h  # objective 2
    
    return [f1, f2]

Finally, let's invoke our implemented test function using our solution xxx from earlier.

objective_values = ZDT1(x)
print(objective_values)

[0.4070005418391035, 3.9526573425595815]

Now we can see the two objective values that measure our solution xxx according to the ZDT1 synthetic test function, which is a minimisation problem.

Using a Framework¶

We've quickly repeated our earlier exercise, where we move from our mathematical description of ZDT1 to an implementation in Python. Now, let's use the Platypus' implementation of ZDT1, which would have saves us from having to implement it in Python ourselves.

We have already imported Platypus as plat above, so to get an instance of ZDT1 all we need to do is use the object constructor.

problem = plat.ZDT1()

Just like that, our variable problem references an instance of the ZDT1 test problem.

Now we need to create a solution in a structure that is defined by Platypus. This solution object is what Platypus expects when performing all of the operations that it provides.

solution = plat.Solution(problem)

By using the Solution() constructor and passing in our earlier instantiated problem, the solution is initialised with the correct number of variables and objectives. We can check this ourselves.

print(f"This solution's variables are set to:\n{solution.variables}")
print(f"This solution has {len(solution.variables)} variables")

This solution's variables are set to:
[None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None]
This solution has 30 variables

print(f"This solution's objectives are set to:\n{solution.objectives}")
print(f"This solution has {len(solution.objectives)} objectives")

This solution's objectives are set to:
[None, None]
This solution has 2 objectives

Earlier in this section we randomly generated 30 problem variables and stored them in the variable x. Let's assign this to our variables and check that it works.

solution.variables = x
print(f"This solution's variables are set to:\n{solution.variables}")

This solution's variables are set to:
[0.40700054 0.04071341 0.2019138  0.79142779 0.9140935  0.03581037
 0.62395881 0.85908972 0.59781233 0.12436693 0.67729958 0.31991784
 0.54294362 0.62353553 0.70098228 0.03701772 0.08176195 0.70208965
 0.40398321 0.41584089 0.71320878 0.46511698 0.28813617 0.86793829
 0.65494094 0.5299217  0.98657349 0.46780006 0.23001727 0.41083113]

Now we can invoke the evaluate() method which will use the assigned problem to evaluate the problem variables and calculate the objective values. We can print these out afterwards to see the results.

solution.evaluate()
print(solution.objectives)

[0.4070005418391035, 3.9526573425595815]

These objectives values should be the same as the ones that were calculated by our own implementation of ZDT1, within some margin of error.

Conclusion¶

In this section, we introduced a framework for multi-objective optimisation and analysis. We used it to create an instance of the ZDT1 test problem, which we then used to initialise an empty solution. We then assigned randomly generated problem variables to this solution and evaluated it with the ZDT test function to determine the objective values.

Exercise

Using the framework introduced in this section, evaluate a number of randomly generated solutions for ZDT2, ZDT3, ZDT4, and ZDT6.

E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation, 8(2):173-195, 2000 ↩

Preamble¶

Introduction¶

The ZDT test function¶

Using a Framework¶

Conclusion¶

Recommend

Fixing Rust's test suite on RISC-V

Librsvg's test suite is now in Rust

How I Structure my Laravel Test Suite

Running the Django Test Suite

Optimizing Your Test Suite

Clean up your Swift test suite with XCTUnwrap

Making our Laravel test suite ready for parallel testing

How we made our Rails test suite about 75% faster

CSS1 Test Suite

Using ML to rewrite a test suite to Playwright

About Joyk