Item 21: Understand what semantic versioning promises

"If we acknowledge that SemVer is a lossy estimate and represents only a subset of the possible scope of changes, we can begin to see it as a blunt instrument." – Titus Winters, "Software Engineering at Google"

Cargo, Rust's package manager, allows automatic selection of dependencies (Item 25) for Rust code according to semantic versioning (semver). A Cargo.toml stanza like

[dependencies]
serde = "1.0.*"

indicates to cargo what ranges of semver versions are acceptable for this dependency (see the official docs for more detail on specifying precise ranges of acceptable versions).

When choosing dependency versions, Cargo will then generally pick the most recent version that's within the combination of all of these semver ranges. However, if the -Z minimal-versions flag is passed to Cargo, it will instead pick the oldest version of each dependency that satisfies the semver ranges; consider including a -Z minimal-versions build in your CI system (Item 32) to confirm that the lower bounds of the semver ranges are accurate.

Because semantic versioning is at the heart of cargo's dependency resolution process, this Item explores more details about what that means.

The essentials of semantic versioning are given by its summary

Given a version number MAJOR.MINOR.PATCH, increment the:

  • MAJOR version when you make incompatible API changes,
  • MINOR version when you add functionality in a backwards compatible manner, and
  • PATCH version when you make backwards compatible bug fixes.

An important detail lurks in the details:

  1. Once a versioned package has been released, the contents of that version MUST NOT be modified. Any modifications MUST be released as a new version.

Putting this in different words:

  • Changing anything requires a new PATCH version.
  • Adding things to the API in a way that means existing users of the crate still compile and work requires a MINOR version upgrade.
  • Removing or changing things in the API requires a MAJOR version upgrade.

There is one more important codicil to the semver rules:

  1. Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

Cargo adapts these rules slightly, "left-shifting" the rules so that changes in the left-most non-zero component indicate incompatible changes. This means that 0.2.3 to 0.3.0 can include an incompatible API change, as can 0.0.4 to 0.0.5.

Semver for Crate Authors

"In theory, theory is the same as practice. In practice, it's not."

As a crate author, the first of these rules is easy to comply with, in theory: if you touch anything, you need a new release. Using Git tags to match releases can help with this – by default, a tag is fixed to a particular commit and can only be moved with a manual --force option. Crates published to crates.io also get automatic policing of this, as the registry will reject a second attempt to publish the same crate version. The main danger for non-compliance is when you notice a mistake just after a release has gone out, and you have to resist the temptation to just nip in a fix.

However, if your crate is widely depended on, then in practice you may need to be aware of Hyrum's Law: regardless of how minor a change you make to the code, someone out there is likely to depend on the old behaviour.

The difficult part for crate authors is the later rules, which require an accurate determination of whether a change is back compatible or not. Some changes are obviously incompatible – removing public entrypoints or types, changing method signatures – and some changes are obviously backwards compatible (e.g. adding a new method to a struct, or adding a new constant), but there's a lot of gray area left in between.

To help with this, the Cargo book goes into considerable detail as to what is and is not back-compatible. Most of these details are unsurprising, but there are a few areas worth highlighting.

  • Adding new items is usually safe, but may cause clashes if code using the crate already makes use of something that happens to have the same name as the new item.
  • Rust's insistence on covering all possibilities means that changing the set of available possibilities can be a breaking change.
    • Performing a match on an enum must cover all possibilities, so if a crate adds a new enum variant, that's a breaking change (unless the enum is marked as non_exhaustive).
    • Explicitly creating an instance of a struct requires an initial value for all fields, so adding a field to a structure that can be publically instantiated is a breaking change. Structures that have private fields are OK, because crate users can't explicitly construct them anyway; a struct can also be marked as non_exhaustive to prevent external users performing explicit construction.
  • Changing a trait so it is no longer object safe (Item 2) is a breaking change; any users that build trait objects for the trait will stop being able to compile their code.
  • Adding a new blanket implementation for a trait is a breaking change; any users that already implement the trait will now have two conflicting implementations.
  • Changing the license of an open-source crate is an incompatible change: users of your crate who have strict restrictions on what licenses are acceptable may be broken by the change. Consider the license to be part of your API.
  • Changing the default features (Item 26) of a crate is potentially a breaking change. Removing a default feature is almost certain to break things (unless the feature was already a no-op); adding a default feature may break things depending on what it enables. Consider the default feature set to be part of your API.
  • Changing library code so that it uses a new feature of Rust might be an incompatible change, because users of your crate who have not yet upgraded their compiler to a version that includes the feature will be broken by the change. However, most Rust crates treat a MSRV increase as a non-breaking change, so consider whether the minimum supported Rust version (MSRV) forms part of your API.

An obvious corollary of the rules is this: the fewer public items a crate has, the fewer things there are that can induce an incompatible change (Item 22).

However, there's no escaping the fact that comparing all public API items for compatibility from one release to the next is a time-consuming process that is only likely to yield an approximate (major/minor/patch) assessment of the level of change, at best. Given that this comparison is a somewhat mechanical process, hopefully tooling (Item 31) will arrive to make the process easier1.

If you do need to make an incompatible MAJOR version change, it's nice to make life easier for your users by ensuring that the same overall functionality is available after the change, even if the API has radically changed. If possible, the most helpful sequence for your crate users is to:

  • Release a MINOR version update that includes the new version of the API, and which marks the older variant as deprecated, including an indication of how to migrate.
  • Subsequently release a MAJOR version update that removes the deprecated parts of the API.

A more subtle point is: make breaking changes breaking. If your crate is changing its behaviour in a way that's actually incompatible for existing users, but which could re-use the same API: don't. Force a change in types (and a MAJOR version bump) to ensure that users can't inadvertantly use the new version incorrectly.

For the less tangible parts of your API – such as the MSRV or the license – consider setting up a continuous integration check (Item 32) that detects changes, using tooling (e.g. cargo-deny, see Item 31) as needed.

Finally, don't be afraid of version 1.0.0 because it's a commitment that your API is now fixed. Lots of crates fall into the trap of staying at version 0.x forever, but that reduces the already-limited expressivity of semver from three categories (major/minor/patch) to two (effective-major/effective-minor).

Semver for Crate Users

As a user of a crate, the theoretical expectations for a new version of a dependency are:

  • A new PATCH version of a dependency crate Should Just Work™.
  • A new MINOR version of a dependency crate Should Just Work™, but the new parts of the API might be worth exploring to see if there are cleaner/better ways of using the crate now. However, if you do use the new parts you won't be able to revert the dependency back to the old version.
  • All bets are off for a new MAJOR version of a dependency; chances are that your code will no longer compile and you'll need to re-write parts of your code to comply with the new API. Even if your code does still compile, you should check that your use of the API is still valid after a MAJOR version change, because the constraints and preconditions of the library may have changed.

In practice, even the first two types of change may cause unexpected behaviour changes, even in code that still compiles fine, due to Hyrum's Law.

As a consequence of these expectations, your dependency specifications will commonly take a form like "1.4.*" or "0.7.*"; avoid specifying a completely wildcard dependency like "*" or "0.*". A completely wildcard dependency says that any version of the dependency, with any API, can be used by your crate – which is unlikely to be what you really want.

However, in the longer term it's not safe to just ignore major version changes in dependencies. Once a library has had a major version change, the chances are that no further bug fixes – and more importantly, security updates – will be made to the previous major version. A version specification like "1.4.*" will then fall further and further behind, with any security problems left unaddressed.

As a result, you either need to accept the risks of being stuck on an old version, or you need to eventually follow major version upgrades to your dependencies. Tools such as cargo update or Dependabot (Item 31), can let you know when updates are available; you can then schedule the upgrade for a time that's convenient for you.

Discussion

Semantic versioning has a cost: every change to a crate has to be assessed against its criteria, to decide the appropriate type of version bump. Semantic versioning is also a blunt tool: at best, it reflects a crate owner's guess as to which of three categories the current release falls into. Not everyone gets it right, not everything is clear-cut about exactly what "right" means, and even if you get it right, there's always a chance you may fall foul of Hyrum's Law.

However, semver is the only game in town for anyone who doesn't have the luxury of working in a highly-tested monorepo that contains all the code in the world. As such, understanding its concepts and limitations is necessary for managing dependencies.

1: rust-semverver is a tool that attempts to do something along these lines.