Improving Classifier Performance by Changing the Difficulty of Images

We propose a difficulty translation model that modifies colorectal histopathology images to become more challenging to classify, finding that image classifiers trained with generated images as augmented data performed better.

General Overview.In this study, I worked with a team of researchers and created a difficulty translation model for histopathology images. In other words, given some image of cancer, we modified it into an image that was harder to classify. This is based on the motivation that these images have a range of features that determines how they would be classified, which differs from general computer vision datasets like ImageNet (e.g. there is no range of cats and dogs).

If you’d like a more technical description of our work, the full paper can be found here .

Generated images had lower agreement among pathologists. The left set of images shows translated images that were indeed harder to classify. The right set of images shows image translations that did not become harder to classify.

The Dataset.The dataset is made of colorectal cancer images from the Dartmouth-Hitchcock Medical Center — it has been split into a training set of 2,051 images and a testing set of 1,101 images. Each of these images were labeled as either a hyper-plastic polyp (HP) or a sessile serrated adenoma (SSA).

The Model.Our module consists of a scorer, which predicts the difficulty of a given image, and a cycle-consistent generative adversarial network (CycleGAN) image translator, which translates images that are easy to classify into images that are harder to classify. With this configuration, the module is able to translate a given image into a similar example that would be classified with the same label but would be harder to classify.