Fixing Audit Won’t Fix Fraud: An Interview with Kris Bennatti of Bedrock AI

This week I have a really interesting interview to share. I talked with Kris Bennatti, the founder & CEO of Bedrock AI, about one of my all-time favourite topics: fraud. 

Kris is a data scientist by trade, and started her career in external audit as a CPA at KPMG. She’s now the founder of Bedrock AI, and leads a team of researchers who use unstructured data to understand and predict corporate malfeasance, fraud, and other bad things. She spends a lot of her time thinking about fraud, earnings manipulation, and low-integrity management teams.

If you’ve read this newsletter for a while, one thing you’ll know about me is that I’m absolutely fascinated by the topic of fraud. (For instance, see: here, here, here, here, here and here.) 

If you want to really understand how something works, study the scams and why they work. Fraudsters (the successful ones, anyway) have to understand exactly how a system works – not how it works on paper, but how it really works – in order to pull off their con. By studying scams carefully, you get a direct line of insight into how these complex systems actually work in real life. 

Auditors and short-sellers are both important parts of the fraud-industrial-complex. They’re both motivated to uncover frauds, but in different ways and armed with different resources. In this interview, Kris and I got to talk about these approaches, and why “looking at the data” doesn’t work nearly as well as you’d think for finding fraud. But scrutinizing the stories people tell in their financial statements and their investor reports can work a lot more powerfully, if you know how to look.

It’s a great time to be talking about this, since more than a few auditors are in a hot seat recently. EY Germany is under the gun after the Wirecard scandal. The U.S. is on track to force Chinese companies to get audits that can be overseen by the PCAOB. So please enjoy this chat; I think you’ll learn a lot from it.

---

Auditing isn’t something I know a lot about. But I have a sense that specific scandals like Wirecard have accelerated what was already an ongoing discussion about whether the auditing firms are actually doing a good job, or the right job. There have been a few high profile scandals and there’s a general sense that the Big Four audit firms are falling from grace. You seem to disagree with the basic premise of these discussions, is that a fair starting point?

Well not exactly. I think it would be great if we could improve the audit function and make the Big Four better. However, fixing the Big Four won’t fix fraud. So if their discussion is supposed to be about how we protect investors and prevent corporate fraud then we’re having the wrong discussion. 

The implicit assumptions that I see in most of the pieces I read is that a) the role of external audit is to ferret out fraudsters and b) the reason they aren’t doing so is because auditors are in bed with their clients. 

The biggest problem with these assumptions is that independence and competence aren’t the problem. It’s just really hard to identify fraud through an audit of financial information, even for a vigilant, independent entity. (I’m not making a judgment call on whether or not the Big Four are independent/vigilant.) The other problem is that audit firms don’t actually think it’s their job to find fraud. In my opinion, they have somewhat reasonable grounds for believing that. 

That’s quite the notion, that audit firms don’t actually think it’s their job to uncover fraud; let’s get back to that in a second. But first, why is it hard to find fraud through a financial audit? What do you mean by that?

Great question. Fraud, in most cases, doesn’t come through in the numbers. We’re in the business of modelling fraud but at Bedrock AI we mostly ignore financial ratios. Instead we focus on long-form text. Think about it, the whole point of earnings manipulation to make the numbers look normal. Any competent fraudster will put a lot of effort into making a normal-looking set of financial statements. 

This is especially true in the early years of a fraud. When the numbers start to unwind, suddenly everyone notices. By then it’s usually too late. All that to say, financial ratios are a pretty weak early predictor of fraud. (The papers that claim great results don’t hold up well in the “wild”.) For the same reason, when an auditor looks at fraudulent financial statements, they will see numbers that appear reasonable. 

I assume that the people who read your newsletter have, at some point in their life, taken a look at a short seller’s report arguing that a company is a sham. Very few short-reports focus solely or even primarily on financial statement metrics. Harry Markopolous’ 2019 GE report is the exception not the rule. Short-reports focus on relationships, customers, suppliers and operations. Auditors, on the other hand, do not audit operational metrics. 

I’m sure my friends still working at KPMG would say otherwise but auditors have a pretty superficial understanding of a client’s operations. When I worked in audit, I often wouldn’t even read my client’s Management Discussion & Analysis (MD&A). A junior associate would tie in the financial figures and that’s where we left it. I assume the partner looked at it but checking the MD&A isn’t our job, that’s how far removed auditors are. 

I completely buy your point that financial information isn’t all that informative if it isn’t supplemented with operational info. By the time fraud actually makes it into the numbers themselves, it’s probably too late. So then if financial audits are failing to do what they’re supposed to, could you expand the audit scope so that auditors actually have a chance of catching this stuff?

Audits are already really bloated, in the U.S. in particular. Anyone who’s been involved in an audit who hears the words “expand” and “audit” in the same sentence will start yanking out their hair. New requirements keep being added to the audit process e.g. Sarbanes Oxley control testing, COSO 2013, ever expanding PCAOB guidance, and none are taken away. 

Simultaneously clients are becoming more price sensitive and putting downward pressure on audit fees. Public company audits have become pretty burdensome for management teams, one of many reasons that companies are choosing not to go public at all. The average auditor works much longer hours than they did 10 years ago. It’s not great for anyone involved. 

I recently watched a lecture from a symposium put on by the University of Texas Salem Center for Policy. During the lecture an accounting professor from Columbia suggests that auditors should be more like short sellers in proactively identifying frauds like Luckin (minute 37) and points to the example of counting customers entering the coffee shops. 

That is perfectly fine to do if you’re a short-seller with suspicions about a specific company; you cannot task auditors with doing something like that. Take a moment to consider the cost of adding audit procedures like sending multiple audit associates to count customers at multiple client locations for multiple days. Most companies only have 1-8 audit associates assigned to their audit teams, generally for less than 2 months. If you want to expand the audit to include operational metrics, you need close to twice the team, twice the time and, most importantly, auditors would need a different skillset, particularly in specialized industries like mining and pharmaceuticals. 

Okay, let’s say audit isn’t the answer to curbing fraud. Fraud is clearly still a big problem, particularly in a bull market like this one where it’s hard to distinguish between good and bad actors.  If improving audit isn’t the solution, what is? Do you have one?

Well, if you’re interested in avoiding large losses as an investor or lender, the answer is to use Bedrock AI tools to find early red flags. I may be biased. 

Bigger picture, the answer is short-sellers, active regulators with big budgets and more capital market actors who are willing to look past EBITDA. Short sellers play a really important role in investor protection that I think is overlooked. They’re the only market actor with the economic incentives to actually check how many customers are entering coffee shops. 

Most people don’t know this but the SEC actually doesn’t do a lot of proactive enforcement. They primarily rely on tips and a lot of those tips come from short sellers and hedge funds. Canada, in particular, could do a better job of being supportive of the role of short sellers. In Canada we tend to demonize them as market manipulators. It irks me that short shellers get such a bad rap while stock promoters get away scot free. Stock promoters don’t contribute to efficient capital markets, short sellers (mostly) do. 

It seems like you’re letting auditors off the hook. Surely they have some role to play here.

Yes, they absolutely do. Auditors can and should do a better job at a lot of things. One area where they’re not performing is identifying/stopping aggressive accounting that essentially amounts to earnings manipulation but stops short of fraud. Many people don’t realize that accounting is very judgment based. There’s a lot of wiggle room before you cross any bright lines. When there’s grey area, the auditor will almost always side with management.

You said before that auditors don’t think it’s their job to find fraud? I’d love if you could go into that a little bit.  

An auditor’s role is to identify material misstatements in the financial statement, whether due to fraud or error. Audits are designed around identifying misstatement. In audit we used to talk about “professional scepticism” a lot but auditors don’t think of an audit as something specifically designed to find fraud. 

One reason for this is that even a well designed audit will have to rely on management’s representations at one point or another. An inventory count is a great example of this. If an auditor goes into a warehouse and says show me the widgets you’ve listed on your balance sheet, they will show you some widgets. Whether those widgets actually belong to the company is anyone’s guess. They could be on consignment, they could be stolen, they could be other widgets that look like the widgets you’re trying to find but have a red screw instead of a yellow screw. The widgets could all be defective but without trying each one, you would never know. 

Needless to say, testing widgets is beyond the scope of what is expected of an auditor. Essentially, if every audit assumed underlying malfeasance, a lot more work would be required. That’s not to say auditors completely ignore fraud risk. Every audit does involve testing that is fraud-specific, it’s just somewhat limited. 

I had a call with a Big Four audit partner a few months ago that really drove home the point that auditors don’t see audits as “fraud-centric”. At Bedrock AI we do two things, we score filings in real-time in order to provide company-specific risk ratings based on the likelihood of malfeasance and more importantly, we highlight in-text red flags that are indicative/predictive of this risk. (This was a really hard problem to solve.) 

The in-text red flags include things that are outside of the auditor’s domain, like executive departures and aggressive Non-GAAP metrics, but also financial considerations like aggressive accounting policies, reversals to reserves etc. During the call, the partner in question gave me some product advice.  One of the things he said was more or less, “It would be great if there were a way you could sell this to auditors, but we don’t really do fraud, so that would be hard.”  I wasn’t surprised to hear him say that but I believe many other people would have been.  

To end, there’s a question I am obviously going to ask you: what is your all-time favourite fraud? The most interesting, or noteworthy in some way, or just generally what specific instance of fraud should more people know about? 

Hmm, that’s a hard question. I have many favourites but I’m particularly fond of the General Electric (GE) case. Just recently, GE got a Wells notice from the SEC which means they’re soon to be under investigation. This isn’t the first time. The SEC accused GE of fraud in 2009, related to “earnings smoothing” in fiscal years 2003 and 2004. GE is interesting to me because they’ve always flirted with the line between earnings manipulation and fraud. GE isn’t a sham. No one stole money or bribed a terrorist or booked non-cash transactions to a related party. The GE fraud was insidious and therefore, most people ignored it.  Smart money has been investing in GE as a “blue chip” stock for years. 

Simultaneously, everyone seemed to be aware that they were manipulating their numbers, before and after they got investigated by the SEC. Harry Markopolos put out a short report on GE in 2019. The report made less of a splash than expected because there were a lot of people saying “hey, everyone already knows this, it’s priced in”. I find that fascinating. 

As far as frauds people should know more about, in my opinion, Valeant (now Bausch Health) should be a household name the same way that Enron is. Valeant had a larger market capitalization than the Royal Bank of Canada at one point and it was all wiped out very quickly. If you want to learn more, I suggest the Dirty Money episode “Drug Short”. It’s on Netflix.

This has been fascinating and I am sure there are more than a few readers of this newsletter who could probably benefit directly from reaching out to you. How can people learn more about Bedrock AI and what you do?

We have three more spots open in our pilot program, starting in mid-January. If you are a pension fund, hedge fund or securities regulator and you’re interested in being part of our pilot, send a note to [email protected].

Follow us on twitter @AIBedrock and on LinkedIn.  Our website is www.bedrock-ai.com and we occasionally post updates through our newsletter. You can subscribe here. 

Thanks Kris for taking the time to come on the newsletter! 

---

Like this post? Get it in your inbox every week with Two Truths and a Take, my weekly newsletter enjoyed by 20,000 people each week.