The Allen Institute for AI (AI2), the division within the nonprofit Allen Institute focused on machine learning research, today published its work on an AI system, called Unified-IO, that it claims is among the first to perform a "large and diverse" set of AI tasks. Unified-IO can process and create images, text and other structured data, a feat that the research team behind it says is a step toward building capable, unified general-purpose AI systems.

"We are interested in building task-agnostic [AI systems], which can enable practitioners to train [machine learning] models for new tasks with little to no knowledge of the underlying machinery," Jaisen Lu, a research scientist at AI2 who worked on Unified-IO, told TechCrunch via email. "Such unified architectures alleviate the need for task-specific parameters and system modifications, can be jointly trained to perform a large variety of tasks and can share knowledge across tasks to boost performance."

AI2's early efforts in building unified AI systems led to GPV-1 and GPV-2, two general-purpose, "vision-language" systems that supported a handful of workloads including captioning images and answering questions. Unified-IO required going back to the drawing board, according to Lu and designing a new model from the ground up.

Unified-IO shares characteristics in common with OpenAI’s GPT-3 in the sense that it’s a "Transformer." Dating back to 2017, the Transformer has become the architecture of choice for complex reasoning tasks, demonstrating an aptitude for summarizing documents, generating music, classifying objects in images and analyzing protein sequences.

Like all AI systems, Unified-IO learned by example, ingesting billions of words, images and more in the form of tokens. These tokens served to represent data in a way Unified-IO could understand.

Unified-IO can generate images given a brief description. Image Credits: Unified-IO

"The natural language processing (NLP) community has been very successful at building unified [AI systems] that support many different tasks, since many NLP tasks can be homogeneously represented -- words as input and words as output. But the nature and diversity of computer vision tasks has meant that multitask models in the past have been limited to a small set of tasks, and mostly tasks that produce language outputs (answer a question, caption an image, etc.)," Chris Clark, who collaborated with Lu on Unified-IO at AI2, told TechCrunch in an email. "Unified-IO demonstrates that by converting a range of diverse structured outputs like images, binary masks, bounding boxes, sets of key points, grayscale maps and more into homogenous sequences of tokens, we can model a host of classical computer vision tasks very similar to how we model tasks in NLP."