What Google has in store for AI and LLM hardware

AI

In the technology business, it’s always prudent to strike while the iron is hot. Google LLC is running with that mantra, bursting through the gate with a flurry of artificial intelligence-related announcements to bring expanded capabilities to the enterprise.

But what’s the deal with all these announcements, and how does each one move the needle of innovation and drive real impact?

“Google is getting into their mojo, which is their greater search — they are merging search, they are grounding AI with search and that’s big,” said Sarbjeet Johal (pictured, right), technology analyst and go-to-market strategist. “Their new AI model, Gemini 1.5 Pro is in public preview, that’s huge. The demo was awesome. I think the equation is simple for any vendor, and they’re getting that — more partner-friendly plus more developer-friendly means more customer friendly and they get that.”

Johal, along with Andy Thurai (second from right), vice president and principal analyst at Constellation Research Inc., and Dustin Kirkland (left), vice president of engineering at Chainguard Inc. and theCUBE contributor, spoke with theCUBE Research analyst John Furrier at Google Cloud Next 2024, during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed the pivotal topics shaping the future of AI, cloud infrastructure and developer ecosystems. (* Disclosure below.)

Google’s strong AI showing signals innovative intent

As the event progressed, it became evident that Google had hit its stride, merging search capabilities with AI advancements. The unveiling of Gemini 1.5 Pro in public preview has sparked excitement and shows the company’s commitment to AI development. However, the real intrigue lay in Google’s approach to cater to diverse needs, balancing large AI models with nano models suitable for mobile devices, according to Thurai.

“What’s more impressive is because everyone else is starting to build massive models, what Google has realized is that’s not the only way we’re going to serve,” Thurai said. “They also came up with distillation of models and smaller language models. So, they’re able to take the nano models and put in their phone and the Google Pixel phones, and now it’s also running on Samsung models too. They’re going both sides of the large models, as well as the nano and smallest possible model.”

Google also made significant strides in catering to developers — a demographic often overlooked in previous event iterations. By spotlighting the Java and Spring frameworks on the main stage, it’s signaling a shift toward embracing enterprise language and empowering developers with versatile API-driven solutions.

“Google has always been about, ‘I give you everything as a platform or as an API level service,'” Thurai said. “They were never concentrating on developers to come in [and] never went to woo the developers. Now the day two, the developer was completely in focus.”

This shift from software development kits to application programming interfaces reflects a broader trend towards developer freedom and interoperability across platforms, a welcome change in the developer community.

Kubernetes’ leading role, model efficiency and future trends

Google’s cloud infrastructure has been built with Kubernetes as its backbone.  The seamless integration of Kubernetes with AI capabilities and workspace applications highlighted Google’s will to provide a comprehensive cloud solution tailored to diverse business needs, according to Kirkland.

“Google Cloud, in general, owes a lot to Kubernetes being the backbone of what makes Google Cloud work, what makes it different,” Kirkland said. “It’s taken 10 years to take something that was an internal Google implementation for all of G-suite, Gmail and YouTube, put that into open source and make that available to everyone.”

One key differentiator for Google is its emphasis on model efficiency and customizability. The concept of model cascading, where specialized models are deployed across edge locations, showcased Google’s innovative approach to optimizing AI workloads.

“When you take a larger model [and break it] into small models, one of the things that they’re doing is called a model cascading,” Thurai said. “You don’t have to have a bigger model at the edge locations to do something with that. You can have a model that’ll work well for you in that particular location. You can cascade it to another model either on cloud or the next location.”

As the industry looks toward the future, the topic of Tensor Processing Units vs. graphics processing units has emerged. A key factor in that interplay is whether Google plans to make TPUs available outside its cloud ecosystem, according to Kirkland.

“I think there’s going to be a real drag race between TPUs and GPUs, and I don’t know which one is going to win,” he said. “I think TPUs have some advantages here. The question is whether or not Google takes TPUs and makes those available outside of Google Cloud.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE Research’s coverage of Google Cloud Next 2024: 

(* Disclosure: TheCUBE is a paid media partner for Google Cloud Next 2024. Neither Google, the primary sponsor for theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE