Mar 5, 2015

As some of you know I’m the unofficial maintainer of the apt module from Puppetlabs. Together with Morgan from the Puppetlabs Module Team we try and keep that module up to date and in good shape.

The apt module has not seen a significant revision since its inception in 2010. Over the past 4 years it has accumulated feature after feature without anyone taking a hard looking at what was going on inside. We merged features that in hindsight have no business being in the apt module and added switches on the main class for all kinds of bizarre things.

What needs to go

What we decided was that we need to reduce the apt module back to its core. The purpose of the apt module is:

  • To manage configuration entries in /etc/apt/conf.d
  • To manage repository entries in /etc/apt/sources.list.d
  • To manage GPG keys for package validation through apt-key
  • To manage apt preferences in /etc/apt/preferences.d

The apt module should provide these basic building blocks to allow you to configured apt as you need. It should not be a collection of user-specific preferences on how apt is to be configured.

We identified a few things that we were unhappy with:

  • unattended-upgrades: Even though it stores configuration in /etc/apt/conf.d it has no business being part of the apt module itself. Maybe people want to use cron-apt instead, or another way of doing this. We should however provide types that allow people to easily and correctly write those configuration files from an external module.
  • fancy progress: For some reason I merged a change that allowed you to configure apt fancy progress through a switch on the main class. This is stupid. If anything user interaction should be limited with the system, we’re automating things, so fancy progress seems completely out of place. However, we never had a good way for users to be able to write those settings, safe for coming up with a file resource definition yourself.
  • builddep: A way to install the build dependencies of a package. This can now be solved by the install_options on the package type and can therefor be removed.
  • Wrapper classes like backports or releases: The only thing they do is create an apt::source entry. It seems harmless but then it starts. “I need to configure a pin”, “I need the pin to be configured at a different priority”, “I need the pin to be pinned on release and not on origin” and so on. We get pull request after pull request for mostly user-specific features. These things are very easily managed yourself but currently cause unnecessary churn in the module. It’s time for them to go.

What needs to be added

We’re removing quite a few things but in return we’re introducing some of the building blocks that will help you easily recreate the old behaviour.

  • apt::setting: A new defined type to write configuration entries for apt. Based on the prefix of the resource name conf-, pref- or list- it will write the file to either /etc/apt/conf.d, /etc/apt/preferences.d or /etc/apt/sources.list.d. You’ll almost never need an apt::setting { 'list-somerepo': } but it’s used internally by apt::source and any other place we write configuration. It supports things as priority, passing in a source or content and knows to notify the apt_update exec.
  • sources, keys, settings and ppas on the main class. Every of those options expects a hash of resources and leverages create_resources() to create resource entries in the catalog. This means you can now fully drive all apt configuration changes from a data binding backend like Hiera.

Things that have remained but changed

  • Major spring cleaning in apt::ppa.
  • Overall spring cleaning and bringing the module in line with our current best practices of module design.
  • Raise our dependencies. We’ve been stuck with stdlib 2.x and old Puppet versions for too long.
  • The purge_* options and their defaults have changed. We now accept a purge hash with four keys, sources.list, sources.list.d, preferences and preferences.d. They all default to true now too. You can pass a partial hash to purge. Instead of needing to specify all entries if you want to override only one of the purge entries, only pass in a hash of the entries you want to override and we’ll take care to merge the rest in there.
  • The key option on apt::source can now take a hash allowing you to pass additional parameters through to the underlying apt::key and apt_key that ends up getting called.
  • The update_* options have been consolidated into a single hash with the same behaviour as the purge hash.
  • The proxy_* options have been consolidated into a single hash with the same behaviour as the purge and update hash. Additionally those proxy options are now also used by apt::ppa and you can turn the option to proxy HTTPS requests on or off. It defaults to off.
  • We’ve removed a set of prefixes on apt::key and apt_key. Things that were named along the lines of key_server are now simply called server as the prefix was redundant.

Status

We’re not done, yet. Most of the things I’ve described here are reflected on the next branch. We’re also hard at work to getting the documentation back into order, write the necessary examples and provide a migration guide for when we release 2.0.0.

We also have plans to release a 3.0.0 version once Puppet 4 has seen some uptake leveraging the new Type system to do parameter validation and other features. This release will likely not break anything on the API level of apt, we’ll just require newer features in Puppet itself.

Working with Morgan on this has been a very rewarding experience. We had some excellent discussions about what we were doing and managed to go through most of the changes at break-neck speed. Except when Travis-CI decided not to run tests because yolo. We’re in a good place and this experience will hopefully set a precedent for many more collaborations to come.

puppet  puppetlabs apt