Linux Kernel Developer Criticizes Intel for Meltdown, Spectre Response

Vancouver, B.C.—At the Open Source Summit North America here on Aug. 29, Greg Kroah-Hartman warned attendees about the severe impact the Meltdown and Spectre CPU vulnerabilities could have on them, as well as detailed how Linux kernel developers are dealing with the flaws.

Kroah-Hartman is one of the world's leading Linux kernel developers, with responsibility for maintaining the stable Linux kernel, and is employed by the Linux Foundation as a Fellow. During his talk, Kroah-Hartman detailed the root impact and the response of Linux kernel developers for seven variants of Meltdown and Spectre, though he saved his strongest criticism for Intel's initial disclosure.

"Jann Horn discovered the first issues in July of 2017, but it wasn't until Oct. 25 of last year that some of us in the kernel community heard rumors of the flaw," he said. "That's a long time, and we only heard rumors because another very large operating system vendor told Intel to get off their tails and tell us about it."

The Meltdown and Spectre vulnerabilities were first publiclyacknowledged by Intel on Jan. 3. The original research was reported to Intel by multiple researchers, including Horn from Google's Project Zero effort. Kroah-Hartman said that when Intel finally decided to tell Linux developers, the disclosure was siloed.

"Normally when we get a kernel security bug, it goes to the Linux kernel security team, we drag in the right people, we work with the distributions getting everyone on the same page and push out patches," he said. "Intel siloed SUSE, they siloed Red Hat, they siloed Canonical. They never told Oracle, and they wouldn't let us talk to each other."

For an initial set of vulnerabilities, Kroah-Hartman said the different Linux vendors that typically work together. However, in this case they ended up working on their own, and each came up with different solutions.

"It really wasn't working, and a number of us kernel developers yelled at [Intel] and pleaded, and we finally got them to allow us to talk to each other the last week of December [2017]," he said. "All of our Christmas vacations were ruined.

"This was not good. Intel really messed up on this," Kroah-Hartman said.

The Linux kernel is developed and run by a wide variety of organizations, and there are both corporate kernels that are backed by large vendors as well as community kernels. Kroah-Hartman said the majority of the world does not run on a corporate-backed Linux kernel. He noted that Intel is used to working with companies and only initially worked with corporate vendors on Meltdown and Spectre.

"The majority of the world runs Debian or they run their own kernel," Kroah-Hartman said. "Debian was not allowed to be part of the disclosure, so the majority of the world was caught with their pants down, and that's not good."

Intel Responds

To Intel's credit, Kroah-Hartman said that after Linux kernel developers complained loudly to the company in December 2017 and into January 2018, it fixed its disclosure process for future Meltdown- and Spectre-related vulnerabilities.

With the most recent variant of Meltdown and Spectre, which has been dubbed Foreshadow and was publiclydisclosed on Aug. 14, Kroah-Hartman said Linux kernel developers were properly notified ahead of time, so that fixes could be made in a collaborative way by the Linux community.

"Intel has gotten better at this," he said.

An interesting side effect of the Meltdown and Spectre vulnerabilities is that Linux and Windows developers are now working together, since both operating systems face similar risks from the CPU vulnerabilities.

"Windows and Linux kernel developers now have this wonderful back channel. We're talking to each other and we're fixing bugs for each other," Kroah-Hartman said. "We are working well together. We have always wanted that."

Fixing the Flaws

All the reported Meltdown and Spectre vulnerabilities have been fixed in modern supported Linux kernels, according to Kroah-Hartman. He warned, however, that not all fixes have been backported to the older Linux 4.4 kernel and advised users not to run any untrusted code on systems running that kernel.

While there have been many patches made in Linux, he strongly advised users to update with Intel's microcode fixes as well, as they provide an additional layer of protection beyond what an operating system can provide.

Kroah-Hartman also warned that the Meltdown and Spectre issues will be around for a long time, as new variants are likely to emerge.

The fixes for the initial Meltdown and Spectre variants have required a lot of effort from kernel developers, and Kroah-Hartman said that there are efforts underway to develop more automated fixes.

"We are not going to be playing whack-a-mole for the next 10 years," he said.

Kroah-Hartman added that there needs to be a reliable way to automatically find the bugs, or to have an updated compiler that detects vulnerable code patterns and removes the flaws. Research is ongoing on different ways to accurately detect Meltdown and Spectre type issues in code, he said.

"Security has always been important," Kroah-Hartman said. "This was just yet another bug, and any bug can be a security bug."

Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.