Probably the last update of the year. I had hoped to do it this weekly in 2019. That didn’t quite happen but since this is the 25th update of the year, I average about one every 2 weeks, which I’ll still claim is a pretty good result. Hope they were useful to you. I’ll try to keep it up in 2020.

Dep graphs

I released a new version of tools.deps.graph today (0.2.24) that has a –size flag to add jar sizes to your dep graphs. Maybe useful, let me know. I’m thinking about adding this into the clj -Stree output too.

Here’s the core.async deps with sizes:

If you have graphviz installed already, you should be able to try this in your own deps.edn project with this command line:

clj -Sdeps '{:aliases {:g {:deps {org.clojure/tools.deps.graph {:mvn/version "0.2.24"}} :main-opts ["-m" "clojure.tools.deps.graph" "--size"]}}}' -A:g

See the docs for better ways to install and other things to do.

S3 Transporter

I’ve been spending most of my time recently working on a new S3 repository access implementation for tools.deps. Maven repositories are mostly just a bunch of files that accessible and are usually served over http(s), but it’s become common to serve them from s3 buckets too.

“Wagons” are the old school pluggable ways to get to different repository types in Maven. These days, the modern Maven resolver has a new framework called “transporters”. The main project people use for access to S3 is the Spring AWS Maven Wagon. This project has not been actively maintained in 4-5 years and is archived on GitHub. Because of the inactivity, there are a bunch of forks and wrappers floating around trying to limp it along. Because it’s a wagon, it doesn’t work directly in the new transporter framework so you need a transporter-wagon adapter as well. The S3 access happens through an old version of aws-java-sdk and there have been various issues with region access, credential chains, bloated deps, etc.

In case it is not yet obvious, this is a big pile of janky barely-working stuff and I cannot wait to drop the whole pile on the floor. In its place, I’ve written a new Transporter impl that uses the Cognitect aws-api library to talk directly to S3. This is currently in a tools.deps branch until I’ve got another chunk of testing in various environments. Nicely, it’s just about 120 lines of Clojure code thanks to the much friendlier things I’m plugging together.

Anyhow, this is not quite done yet, but it does mostly work, and removes about 3-4 mb from the clj install (which maybe nobobdy cares about but me).

One question I anticipate is whether this code will work for non-clj users, and currently the answer is no. I have not updated the “put” path (as clj doesn’t need it), and I have not done whatever dependency injection magic is needed for this code to work properly as a Maven extension (or Leiningen plugin, etc). Maybe at some point if I have some free time, I will look at that.

Spec 2

In spec 2 world, when last I wrote I was talking about the addition of nonflowing s/and, aka s/and-. We’re now contemplating renaming that to s/union (which I know currently is being used for schema combination, but that would change).

I also implemented a new s/defcompop that takes a spec name, and additional predicates and creates a new spec op that takes the args of the first spec, then s/and-‘s them with the predicates. That was really in service of making it easy to define s/catv and s/cats which are vector-only and sequence-only variants of s/cat. Those are all now available in spec. I’ve seen the vector-only and sequence-only needs come up many times when spec’ing macros, so this is a good tool to fix a bunch of issues in the current core.specs.

Users who have dabbled a bit with spec 2 may also notice that I have renamed all of the namespaces in the current impl as well, trying out a better strategy for this “alpha while in dev” practice. Specifically, the api namespaces are now clojure.alpha.spec, clojure.alpha.spec.gen, and clojure.alpha.spec.test. We will probably rename the library as well to clojure.alpha.spec. The idea is that when we leave “alpha”, the alpha will be removed from all of these names. Timing of all that and how it interacts with Clojure itself is all still TBD, just starting to work towards that.

Also, Rich has been hard at work on design work around function specs and possible integration into defn. Lots of exciting ideas and I’m sure that will be a focus in January.

Some other miscellaneous updates:

• com.cognitect.aws/api - I noticed while working on the combination of the s3-transporter and the tools.deps.graph size enhancement that the aws api depended on Apache commons-codec, solely for one call to hex encode, so I wrote a Clojure-only version and killed off that dependency. It’s not as fast, but it’s something like 1.7x which seemed sufficiently good. If anyone wants to perf golf it, let me know what you find! Hasn’t yet been released but should be in next.
• clojure/data.xml - I also noticed that data.xml was using data.codec for base64 encoding, which is now built into the JDK (since Java 8) in the Base64 class so I swapped that out and killed off the dep (and left a note in the data.codec readme). Didn’t release that but it will be in next.
• clojure/core.async - there was a question on Slack about why (chan 0) errors, but (buffer 0) doesn’t, and indeed there is no good reason - the assertion was in a bad place, so I fixed that. Didn’t release it yet.
• clojure/tools.deps.alpha - did some minor cleanups and bumped to latest deps, not released
• Some misc JIRA stuff

Other stuff I enjoyed this week…

I’m totally digging Broke by Samm Henshaw this week - hat tip to my daughter for sending that one to me. I was hooked by the first horn hit 3 seconds in. Also, I imagine that’s what my sons’ apartments will look like when they’re older.