LLMs like ChatGPT will win in business when they start learning from humans, says Emergence Capital

What’s the business case for Large Language Models (LLMs) like ChatGPT? Will they ever be reliable enough to support business decisions — or even autonomously take them and act on them? Emergence Capital is a VC firm that’s long held the belief that AI will fuel the next generation of enterprise applications. But Jake Saper, General Partner at the firm, foresees a rocky start for LLM technology in the business world, although he’s optimistic for the long term. He says:

There's going to be some really high profile mistakes that are made ... that freak people out. I also think that some of the AI startups that had been funded at a really high valuation and grew really quickly may not have sustainable business models, because they're effectively just reselling OpenAI’s technology. And those companies may explode ...

That's the creative destruction necessary to bring about positive change.

The problem with generative AI technologies like ChatGPT in a business context is that they sound persuasive, but they're often wrong. He explains:

It's scary how accurate it sounds. Because we as humans evolved to cue off, in many ways, how credible is my counterpart? And often that's, ‘How do they sound? Who are they?’ Much less than, ‘What is the accuracy of the content? ...’ If you're chatting with a dead celebrity, and it gets it wrong, who cares? There's no such thing as wrong when you're using it for a consumer use case.

But if I'm making a decision on what medical procedure to follow, or how to close this contract, or whatever it is, getting that wrong, and trusting it blindly because it sounded right, is really bad.

For a business context, these tools need more human supervision and guidance, he believes:

Deploying them on their own without a feedback loop, without contextually specific data, and without a human to help oversee and ensure accuracy, is likely to result in some some scary outcomes ...

When I'm skeptical of these tools, it's not that I don't think that they're going to be really helpful for business. They will be. But if you use them without guardrails, bad things will happen.

Humans and AI can learn from each other

This is where Emergence Capital’s concept of coaching networks comes in — a system in which humans and AI iteratively learn from each other, building on each participant’s strengths. Saper explains:

Humans are generally not great at quantitative forecasting. But we are good at adaptation, and creativity, and those types of things. We've got to find a way to integrate those things into the great skills that AI is bringing to us.

He sees the advances in LLMs being built by OpenAI and others as an important step that will contribute to the success of the coaching networks concept. Emergence is calling its approach 'iterative AI' because of the closed feedback loop that it builds in, rather than the open-ended, unmoderated output of generative AI. The big flaw in the current use cases for ChatGPT is the lack of a feedback loop. A skilled salesperson who edits a prospect email prepared by AI based on data held in the CRM system is probably going to do a better job than anything that individual could write unaided. But the human element is essential. "If you just have the emails written by AI, it's going to suck," remarks Saper.

As well as more proactive supervision, the other key to success in a business context will be marrying the LLMs up with deeper datasets. He explains:

The business use cases will start with making a call to some of these LLMs. But then they're going to marry that with their own proprietary data, and potentially proprietary models that are more use-case specific.

This is a concept called chaining ... You take the general model and then chain it together with use-case specific, very focused, data and models, and you're likely to get a better outcome.

As an example of how this might work, he cites the example of Ironclad, a SaaS company for digital contract management. The company has built an AI coach which is already quite smart in how it redlines contracts, and which learns from its interactions with human legal experts. Saper explains:

You don't want to just ship that contract with the AI redlines, you actually want a human in there to confirm the ideas, and then also to make the model better based on the things they edit.

Using a suitably trained LLM service, in the future it might be possible for the AI to suggest the most mutually acceptable wording, based on the patterns it finds in historic contracts — rather than each side's lawyers arguing for their own preferred wording. He elaborates:

If you and I are negotiating a clause, it could query all of the contracts that it's worked on in the past and say, this is actually the mutually acceptable outcome that is most frequently come to, when you and I are negotiating this clause ...

You actually create, using AI and data, an arbiter that says, this is where this clause fits in terms of the frequency. And if you both can agree on this thing ... it's more likely to be a successful outcome.

That would be impossible with ChatGPT on its own, because first of all, they don't have that specific contract negotiation data, and the models just aren't trained to be able to give that answer. So the exciting thing is .. it's leveraging OpenAI. But it's doing it in the context of Ironclad-specific workflow and data.

At an inflection point

This technology is bringing the industry to an inflection point and it's not clear at this moment where the winners will come from. He says:

It's an exciting moment. This technology is causing a lot of established players to question their moats, to question their dominance ... Where does the value accrue with this new technology? Does it accrue to the giants? Does it accrue to the kind of growth stage companies like Ironclad who can integrate it quickly? Does it accrue to startups that are building around this de novo? I don’t have a good answer to that yet.

My guess is there'll be value that accrues at all levels. I think a lot of startups that are getting started right now are unlikely to be defensible, because they're effectively reselling someone else's technology with a really thin UI layer on top of it. That's unlikely to be a sustainable business. I think the ones that get started now to build a really complex workflow, just like boring SaaS 1.0, but are injecting AI intelligently into that workflow, are likely to have more durability.

In other words, the integration of AI into business applications, for all the furore we're currently hearing, will seem rather mundane when it happens. He says:

It's much more likely to look like what I described with Ironclad, which is they've got a SaaS workflow that allows you to negotiate a contract and they're intelligently injecting AI in different parts of the workflow to help speed up that workflow, to help people make less errors, to help people get to agreement faster.

It's unlikely to be a situation where, boom! SaaS looks completely different tomorrow. Over time, it's going to become indistinguishable with SaaS, the same way that cloud has become indistinguishable. It'll just be weaved in, throughout, and people will be able to do their workflows faster, better, etc.

My take

We're currently experiencing a huge rise in hype around the emergence of generative AI, with many enterprise IT vendors leaping onto the fast-moving bandwagon. As a company that has seen this coming for more than five years, it's worth listening to Emergence's view. The reality may not be as exciting as the hype currently makes out, but I'm reminded of the old adage that we overestimate how much can change in one year and hugely underestimate how much can change in a decade. In ten years' time, will the enterprise vendor landscape look much the same as it does now, or will there be some unfamiliar names rising up the ranks on the back of AI technologies that are serving unmet needs? That was how it turned out with the shift to cloud computing. It seems likely that AI is capable of producing a similar shift. But the history of cloud computing and SaaS tells us that the conventional wisdom at the start of the trend is rarely a guide to the ultimate outcome. Right now, it's anyone's guess how this will all turn out.