Feature selection algorithms in machine learning

Feature selection algorithms in machine learning

By Roberto Lopez, Artelnics.

Many typical applications in machine learning, from customer segmentation to medical diagnosis, arise from complex interactions between all types of variables.

But analyzing multiple internal and external variables is very complicated. Data scientists might have huge data sets, and they need innovative methods that can select relevant variables.

In this post, we formulate the model selection problem and describe the algorithms most used in practice.

Contents:

Neural Designer implements all the model selection algorithms described here. To use these algorithms in practice, you can download the Neural Designer.

Problem formulation

In this way, model selection is applied to find the topology of a neural network that minimizes the error on new data.

Input selection algorithms are responsible for finding the optimal subset of inputs.

We define the selection error as the error of a neural network for new data. It measures the ability of the model to predict the result in a new case.

Input selection is a method to improve the quality of the predictions. It consists in extracting the subset of inputs that have more influence on a particular physical, biological, social, etc. process.

Growing inputs

The growing inputs method starts by calculating every input's correlation with every output in the neural network.

The growing inputs method starts with the most correlated input and keeps adding well-correlated variables until the selection error starts increasing.

Pruning inputs

The pruning input method also starts by calculating the correlations among every input and output in the neural network.

The pruning inputs algorithm starts with all the variables of the data set and removes the inputs with little correlation with the outputs.

Genetic algorithm

A different class of inputs selection method is the genetic algorithm.

This is a stochastic method based on the mechanics of natural genetics and biological evolution.

The genetic algorithm implemented includes several methods to perform fitness assignment, selection, crossover, and mutation operators.

The following figure shows a simplified flow diagram of the genetic algorithm.

The genetic algorithm starts with a population of different subsets of variables.

In every generation, the fitness of every individual in the population is computed as the selection error for that subset of inputs.

Then, the method evolves the population by selecting some individuals to generate the new population, performing a crossover with the selected population, and mutating the offspring generated during the crossover.

Conclusions

But model selection algorithms are very expensive in computational terms, so a big drawback here is the performance.

Neural Designer includes an advanced model selection framework capable of representing very complex data sets.

This system procures high added value to data scientists, providing them with results in a way previously unachievable.

Do you need help? Contact us | FAQ | Privacy | © 2022, Artificial Intelligence Techniques, Ltd.