The AI computing boom: OctoML targets machine learning workload deployment

AI

The artificial intelligence scene is currently at a major inflection point, and solutions providers are cropping up to address specific pain points.

One of those is OctoML Inc., whose value promise is improved speed and efficiency in deploying enterprise machine learning workloads. OctoML was spun out of the University of Washington by the creators of Apache TVM, an open-source stack for ML portability and performance.

“We saw a major pain point and built OctoML, which is about three and a half years old now,” said Luis Ceze (pictured), co-founder and chief executive officer of OctoML. “And the company’s mission is to enable customers to deploy models very efficiently in the cloud and enable them to do it quickly, run fast, and run at a low cost, which is something that’s especially timely right now.”

Ceze spoke with theCUBE industry analyst John Furrier ahead of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event, during a CUBE Conversation. They discussed OctoML and signals on the growing need for purpose-built ML deployment solutions. (* Disclosure below.)

Massive AI-driven acceleration currently underway in CS

The timely pairing of advancements in hardware (as with microchips) and software capabilities have resulted in the perfect storm of AI ideas. A massive acceleration is taking place in the computer sciences, and this can only bring more powerful computing capabilities to reality, according to Ceze. In turn, it will create benefits across multiple stakeholders, from researchers to entrepreneurs.

“Advances in AI are enabling applications that we thought to be highly futuristic, and now they’re just right there today,” Ceze said. “[You have] AI that can generate photorealistic images from descriptions and can write text that’s pretty good, improving human creativity in a really meaningful way. And this is all very exciting, both from a research point of view as well as an entrepreneurial point of view.”

Another crucial element tying everything together is the improved scale of data integrations for use in these models. And, interestingly, a lot of it has come from taking preexisting foundational models and fine-tuning them to expertise with the data, Ceze added.

 “The ability to either take the data that exists and train a model to do something useful with it and, more interestingly, using baseline foundational models and with a little bit of data to turn them into something that can do a specialized task really well is exciting,” he stated. “It’s created this really fast proliferation of really impactful applications.”

Inevitably, AI and ML will become table stakes for most businesses to stay competitive in their industry. And companies that come early to the party stand to enjoy some of the first-mover benefits. For its part, OctoML is helping companies make that transition, according to Ceze.

“We’re enabling customers to incorporate AI into their applications and use cases accounting for the people that they have and the costs that they can afford, such that it creates the maximum impact possible,” he pointed out. “Also, getting the right hardware to run these incredibly hungry models is hard. So, we also help customers deal with hardware availability problems, as well as the solutions part.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event:

(* Disclosure: OctoML Inc. sponsored this segment of theCUBE. Neither OctoML nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE