3

The fast kernel headers tree

 2 years ago
source link: https://lwn.net/Articles/880175/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

The fast kernel headers tree

[Posted January 2, 2022 by corbet]
Kernel developer Ingo Molnar has been quiet for a while; now we know why. He has just announced a massive set of patches (touching over half of the files in the kernel tree) reworking how header files are handled.

The fast-headers tree offers a +50-80% improvement in absolute kernel build performance on supported architectures, depending on the config. This is a major step forward in terms of Linux kernel build efficiency & performance.

A justified question would be: why on Earth 2,200 commits??

It seems likely that interesting conversations will follow; stay tuned.

(Log in to post comments)

The fast kernel headers tree

Posted Jan 2, 2022 23:39 UTC (Sun) by Paf (subscriber, #91811) [Link]

John,

I was surprised to see no replies to Inyo’s posting, then checked a few timestamps. Did you really post this five minutes after his message or am I reading something wrong? 😂

Timing

Posted Jan 3, 2022 0:04 UTC (Mon) by corbet (editor, #1) [Link]

It's part of my routine to check in Sunday afternoon/evening (my time) for Linus's -rc posting, and I happened to notice that as well. Had Ingo posted it any other time it would have sat for longer.

Wait a little while, I'm sure you'll see replies ;)

Timing

Posted Jan 3, 2022 3:26 UTC (Mon) by jbailey (subscriber, #16890) [Link]

Nonono, you're supposed to say something enigmatic, like "I felt a great disturbance in the Force, as if millions of continuous builders suddenly cried out in terror and were suddenly silenced."

The fast kernel headers tree

Posted Jan 3, 2022 1:02 UTC (Mon) by dankamongmen (subscriber, #35141) [Link]

corbet was also explicitly cc'd on the mail

The fast kernel headers tree

Posted Jan 3, 2022 1:18 UTC (Mon) by pebolle (subscriber, #35204) [Link]

> 😂

One would hope that lwn.net subscribers would not use emojis. Since they actually do we might as well ask Jon to go all out and accept animated GIFs too. Why reject what the rest of the world is using so feverishly?

The fast kernel headers tree

Posted Jan 3, 2022 1:26 UTC (Mon) by banana (guest, #144773) [Link]

What’s wrong with emojis? They work OK on Linux.

The fast kernel headers tree

Posted Jan 3, 2022 1:40 UTC (Mon) by JoeBuck (subscriber, #2330) [Link]

Let's not derail the conversation arguing about emojis.

To get back on topic: is there any performance cost to the uninlining?

The fast kernel headers tree

Posted Jan 3, 2022 8:49 UTC (Mon) by smurf (subscriber, #17840) [Link]

Yes. Your kernel build is now too fast to wait for the coffee machine, so you need another justification to hang out there.

The fast kernel headers tree

Posted Jan 3, 2022 17:12 UTC (Mon) by flussence (subscriber, #85566) [Link]

Easy: just build the kernel with LTO and you'll have enough time to go get lunch ;-)

The fast kernel headers tree

Posted Jan 3, 2022 16:06 UTC (Mon) by Sesse (subscriber, #53779) [Link]

I believe Ingo posted somewhere that he had run performance tests with no ill effects.

In general, people do inline too much. Some things absolutely must be inlined, and many things are just fine without. Even worse, some things are faster locally when inlined, but slow down the rest of the system (due to code bloat).

The fast kernel headers tree

Posted Jan 3, 2022 23:53 UTC (Mon) by atnot (subscriber, #124910) [Link]

> In general, people do inline too much.

Agreed, especially these days with LTO being very widespread, it turns out that humans are usually significantly worse than compilers at deciding what would benefit from inlining.

The fast kernel headers tree

Posted Jan 3, 2022 2:27 UTC (Mon) by Paf (subscriber, #91811) [Link]

Kids these days! Terrible!

The fast kernel headers tree

Posted Jan 3, 2022 4:15 UTC (Mon) by felixfix (subscriber, #242) [Link]

Many systems turn text emojis like colon dash close-paren into gifs. Let's find out if that is the case here.

Emojis began in the text world, probably on typewriters, and I say WE TAKE BACK OUR EMOJIS!!!!

The fast kernel headers tree

Posted Jan 3, 2022 8:21 UTC (Mon) by nilsmeyer (subscriber, #122604) [Link]

Let's do one better and convert all gifs back to text.

The fast kernel headers tree

Posted Jan 3, 2022 8:31 UTC (Mon) by Karellen (subscriber, #67644) [Link]

"text emojis" - we already have the word "emoticons" which predates "emojis" (in English) by at least 5 years. And they can pry my emoticons off of my cold, dead keyboard!!!! Or something.

The fast kernel headers tree

Posted Jan 3, 2022 17:42 UTC (Mon) by EnigmaticSeraph (subscriber, #50582) [Link]

While I understand the sentiment, Emojis are far better than emoticons. For example, screen readers for the blind cannot interpret emoticons, but Emoji are fine. To be inclusive, we should really move past emoticons.

The fast kernel headers tree

Posted Jan 3, 2022 18:20 UTC (Mon) by Wol (subscriber, #4433) [Link]

Are you *trying* to us exclude dinosaurs!?

I won't say I use emoticons much, but I don't use emojis AT ALL. I neither know, nor care, how to ... :-)

Cheers,
Wol

The fast kernel headers tree

Posted Jan 3, 2022 21:04 UTC (Mon) by NYKevin (subscriber, #129325) [Link]

Wikipedia claims the following:

> Mary Kalantzis and Bill Cope write that the first digital emojis were created by Bruce Parello, a student at the University of Illinois, on PLATO IV, the first e-learning system, in 1972.

It cites a book as well as this open-access PDF: https://www.researchgate.net/publication/351400910_A_Litt...

If that year is correct, then it would predate the introduction of (digital) emoticons in 1982 by a full ten years (although Wikipedia again claims that emoticons have appeared in non-digital text as early as 1648).

However, there are a few caveats here:

1. To the best of my understanding, these emoji did not get incorporated into Unicode and therefore lack historical continuity with modern emoji.
2. The PLATO emoji can't have been very influential, because everybody and their dog incorrectly claims that Shigetaka Kurita created the first emoji in 1999.

The fast kernel headers tree

Posted Jan 4, 2022 1:48 UTC (Tue) by sfeam (subscriber, #2841) [Link]

I'm not sure that implementing a custom character on a PLATO system counts as "creating an emoji" as that would commonly be understood now. PLATO used a vector display; any arbitrary shape could be defined, used, and reused. The same was true for the PDP/PDS-IMLAC terminals of that same era. My first summer job project back in 1971 involved teaching an PDP+IMLAC system to draw logic diagrams, and that started with creating a library of routines to draw individual logic gates (AND NOR XOR ...) essentially as reusable custom characters. Those were 絵文字 (emoji) in the literal sense of being "picture (絵) characters (文字)" but obviously not in the current social media sense of "punctuation marks conveying emotion".

On the other hand, I concede that if Bruce Parello's PLATO emoji were promoted for use by students or instructors as markup annotations for PLATO assignments, they might qualify under the modern sense of emoji as well as under the original technical sense.

The fast kernel headers tree

Posted Jan 3, 2022 20:50 UTC (Mon) by KJ7RRV (subscriber, #153595) [Link]

How hard would it be to create a font that shows emojis as emoticons? That, combined with existing emoticon-to-emoji converters, should make everyone happy.

The fast kernel headers tree

Posted Jan 3, 2022 20:54 UTC (Mon) by KJ7RRV (subscriber, #153595) [Link]

A browser extension would also work for Web sites (and any chat platforms accessed through a browser, as well as Web mail); that way it would actually be emoticons and not just look like them. An IRC bouncer could do the same for IRC.

The fast kernel headers tree

Posted Jan 4, 2022 3:19 UTC (Tue) by flussence (subscriber, #85566) [Link]

The "☺" in most DOS codepages predates Unicode by a few decades too...

The fast kernel headers tree

Posted Jan 3, 2022 14:09 UTC (Mon) by atnot (subscriber, #124910) [Link]

I am pleased to say for the future of the kernel that there are in fact people below the age of 40 using this website

The fast kernel headers tree

Posted Jan 2, 2022 23:48 UTC (Sun) by rgmoore (✭ supporter ✭, #75) [Link]

I guess we know who's going to be the leader in the LWN stats for the next kernel.

The fast kernel headers tree

Posted Jan 3, 2022 2:54 UTC (Mon) by JMB (guest, #74439) [Link]

So fast ... very unlikely ... this is an RFC for a reason - so not for the next kernel ...
The reviewing process will be ... interesting ... ;)

The fast kernel headers tree

Posted Jan 3, 2022 8:22 UTC (Mon) by adamg (subscriber, #42260) [Link]

>I'm pleased to announce the first public version of my new "Fast Kernel Headers" project that I've been working on since late 2020 (...)

This is mind blowing example of how determined kernel developers are. Imagine spending more than one year developing a feature you are not sure will be even accepted. And that's just a start of journey, merely first RFC, not even pull request. Guys like Ingo, Jason (for wireguard took about 4 years to be merged) and many others.

Hats off!

The fast kernel headers tree

Posted Jan 3, 2022 8:54 UTC (Mon) by zdzichu (subscriber, #17118) [Link]

Think about employer who tolerated working on such feature. Someone had to pay for this year of work :)

The fast kernel headers tree

Posted Jan 3, 2022 9:15 UTC (Mon) by metan (subscriber, #74107) [Link]

Actually this makes a perfect sense from an employers point of view, as long as CI is concerned speeding up the build twice significantly shortens the CI feedback loop. That alone a big win for developers. It also allows you to test twice as much patches or enable more automated testsuites without increasing the hardware lab size and power consumption.

The fast kernel headers tree

Posted Jan 3, 2022 11:34 UTC (Mon) by micka (subscriber, #38720) [Link]

There is probably some gain on test duration but I expect most test suite to run far longer than the kernel build part.

The fast kernel headers tree

Posted Jan 3, 2022 12:17 UTC (Mon) by metan (subscriber, #74107) [Link]

That really depends on the usecase. Some of the CI systems can identify, based on the patch, which parts of kernel were changed and tries to select minimal subset of tests that covers that exact area. Of course this is not bulletproof, but on the other hand it can identify bugs very fast and in that case the kernel build time may be significant part of the CI runtime. This is really about developers getting feedback fast in case that something is not right. Of course you want to run all the tests once this minimal set has passed, which will possibly identify more subtle problems, but that will also take much longer and the results will be produced probably after the developer has already started to work on a different task.

Posted Jan 3, 2022 21:48 UTC (Mon) by david.a.wheeler (subscriber, #72896) [Link]

I agree, this is a crazy amount of work. Serious hat tip here.

The fast kernel headers tree

Posted Jan 3, 2022 8:24 UTC (Mon) by tdz (subscriber, #58733) [Link]

Fantastic! I can't wait for this to get merged.

The fast kernel headers tree

Posted Jan 3, 2022 8:50 UTC (Mon) by smurf (subscriber, #17840) [Link]

Yes you can. In fact you'll have to, as it's somewhat far from finished.

The fast kernel headers tree

Posted Jan 3, 2022 17:25 UTC (Mon) by jhhaller (subscriber, #56103) [Link]

That's going to put a substantial drop in performance in my post-retirement plan to modify gcc to use openat for -I directives instead of concatenating each of the -I paths and trying to open the files from root, which cuts out a number of inode lookups and the string concatenation. I doubt it would have improved performance more than a few percent, and now there will be a lot fewer files to open. At least for the kernel, anyway, it could still help other compilations. But, someone may beat me to the change, or perhaps someone has already done it.

The fast kernel headers tree

Posted Jan 3, 2022 19:28 UTC (Mon) by jezuch (subscriber, #52988) [Link]

As I understand it, this is mostly what maven-enforcer-plugin does in Java land. Good dependency hygiene is important, hard to maintain (especially without any tooling support!) and close to impossible to attain in an older project.

Titanic work!


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK