Release of Oil 0.8.9

blog | oilshell.org

Release of Oil 0.8.9

This is the latest version of Oil, a Unix shell that's our upgrade path from bash:

To build and run it, follow the instructions in INSTALL.txt. The wiki has tips on How To Test OSH.

If you're new to the project, see Why Create a New Shell? and posts tagged #FAQ.

This release announcement is sprinkled with links to external stories, like other posts tagged #comments. Feel free to let me know if you find them helpful or not.

Summary of Changes

Oil Language

I tightened up the semantics of the Oil language. It now relies less on Python internals, which is a step toward entirely removing the runtime dependency on Python. For example:

# Create an array, which is like a Python list
oil$ var x = [42, 3.14, 'foo bar']

# Now an error instead of using Python's repr().
# Only primitive types can be stringified.
oil$ echo $x
fatal: Expected a bool, int, float, or string.

# Splicing an array makes sense.
oil$ write -- @x
42 
3.14
foo bar

(The error message is rough and needs polish.)

TODO: We should probably have a JSON function as well as the builtin, so you can do this:

# A stable format to exchange with processes written
# in other languages
oil$ echo $json(x)
[42, 3.14, "foo bar"]

Feedback on the language is always welcome!

OSH Compatibility

For compatibility with GNU libtool, we now accept return '' to mean return 0 when the strict_control_flow option is off. As usual, it's off by default in OSH, but turned on by the oil:basic option group, and bin/oil.

This behavior difference pertains to one of the riskier known differences we have:

• break / continue / return are keywords, not builtins

It's possible that we should revert this difference entirely, but I hope not. I like the statically-parsed versions: they can help catch unused code, and make it possible to compile shell statically to bytecode.

Removed libc Locale Hacks

We now have a simpler implementation of Unicode. Oil's semantics have always been centered around UTF-8, though we still use libc, which complicates the situation.

I explain why in this comment. Summary: the encoding is a property of data, not the code that processes it! Context:

Moving Toward Stable Performance Metrics

I've noted the problem with our performance metrics in recent posts tagged #metrics. So I started using cachegrind to measure instructions simulated rather than wall time, e.g. for the shell parsing benchmark:

• osh-parser benchmark for 0.8.9

This was inspired by a recent post, which references sqlite's usage of the technique:

The Oil release process is still tied to two of my personal machines, and cachegrind measurements will make us less reliant on them. The release process should be in the cloud, or really multiple clouds. See my recent post about CI services and build systems.

Acknowledgements

The last release got more feedback than is typical. Thank you to everyone who tried Oil, reported bugs, and contributed!

• Travis Everett (return semantics)
• rsteube on Github (locale issues)
• Gabriel Arazas (who found the Eggex bug, along with providing language feedback)
  • I also want to acknowledge good feedback from Danilo Spinella on recent releases
• Ilya Sher (of Next Generation Shell)
• Elisha Hollander

Closed Issues

#925 eggex crash bug in a loop

#920 Quick Reference link is broken

#917 $maybe(empty) instead of @maybe(empty) does the wrong thing

#914 return $x is too strict for libtool

#912 archlinux: Unhandled exception 'Invalid locale for LC_CTYPE'

#871 Use cachegrind for stable performance measurements?

Commit Log

Here are the commits from other contributors. You can also view the full changelog.

88e9505 Elisha Hollander Remove unnecessary line in osh-to-oil.js (#910) b6dd617 Ilya Sher Fix typo (#921) 7a593cf Ilya Sher Update qtsv.md (#922)

What's Next?

Working on the garbage collector will move us toward oil-native.

But I also want to work on Shell as an Engine for a TUI or GUI (issue 738), which will enable parallel development on the #interactive-shell. I want Oil to be a solid foundation for alternative UIs.

I hope this will also lead into work on the coprocess protocol, which I mentioned in this 2018 blog post.

Specifically, Subhav Ramachandran and I discussed using file descriptor passing between a UI and shell, and I now realize that this mechanism should also be used for coprocesses. Importantly, the shell itself should be a coprocess, in addition to invoking them.

There's some more color on coprocesses in this comment analyzing the problem with Python startup time. It's basically a fancy name for "single-threaded server". Context: