:::: MENU ::::

GitConsensus now available as a Github App

Last year I introduced a way to manage open source projects with GitConsensus, an open source Github bot that anyone can download and run.

Today running GitConsensus is even easier with the availability of a new GitConsensus GitHub App. Developers can now add GitConsensus directly to their repositories simply by enabling it in GitHub and then adding the consensus files. There’s even example consensus rules covering a variety of options, from anarchy to oligarchy (with a set of recommended rules that is good for most projects).

In order to get this all working I also had to build a new GitHub Apps Python Library. This library extends the excellent github3.py library, making it easier to extend projects already built using it into full-fledged GitHub Apps.

Manage Github Pull Requests with gitconsensus

This weekend I dug into the Github API to build gitconsensus, which lets communities create truly democratic projects using Reactions as a voting mechanism. Projects can define consensus rules (minimum age of pull request, quorum for votes, threshold needed for passing) using a yaml file in their project root. Pull Requests that meet the consensus rules will get merged, and those that do not meet them after a certain amount of time will get closed.

The yaml file itself is pretty straightforward-

# .gitconsensus.yaml
# Add extra labels for the vote counts and age when merging
extra_labels: false

# Do not count any vote from a user who votes for multiple options
prevent_doubles: true

# Minimum number of voters
quorum: 5

# Required percentage of yes votes (ignoring abstentions)
threshold: 0.65

# Only process votes by contributors
contributors_only: false

# Only process votes by collaborators
collaborators_only: false

# When defined only process votes from these github users
  - alice
  - bob
  - carol

# Number of days after last action (commit or opening the pull request) before issue can be merged
mergedelay: 3

# Number of days after last action (commit or opening the pull request) before issue is autoclosed
timeout: 30

The project is available now on pypi and can be installed using pip.

Introducing jsonsmash – work with large json files easily

Over the last year I’ve run into some pretty massive JSON files. One recent examples is from AWS, which publishes a 120mb file containing a list of their available services that they have yet to provide documentation for. Attempting to open that in a standard editor is not going to be pleasant, and while tearing it apart with something like jq is certainly an option it doesn’t feel like the best approach.

That’s why I’ve build jsonsmash, an emulated shell that lets users browse through their massive json files as if they were actually filesystems. It uses shell commands any linux user would already be familiar with to make it very straightforward to figure out the schema of a json object (even if it is a few hundred megabytes) and pull out specific data.

Development is on github and the package is published on npmjs.

Stash v0.14 Released with PSR-6 Support

Release v0.14.1 is a major update to Stash, and quite likely the last line of releases before v1.0.0.

The biggest addition is support for PSR-6. Stash now implements the various interfaces natively, allowing it to be directly injected into PSR-6 compliant libraries.

This did require a few API changes (which prompted some cleanup of deprecated functions), so please make sure to review the release notes.

Stash remains one of the best tested caching libraries out there, and this improved on that significantly. In addition there have been optimizations and improvements in PHP7 support, including behind the scenes switching between the APC and APCu functions.

The Symfony Stash Bundle has also been updated, with version v0.6.1 utilizing the Stash v0.14 line.

New Release of Stash and Stash Bundle

Stash 0.13.1

  • Dropped support for PHP 5.3.
  • Updated dependencies.
  • Removed various PHP warnings (exceptions are still thrown where needed).
  • Various optimizations, such as reduced function calls during repeated operations.
  • Added “isPersistent” method to driver classes.

Stash Bundle v0.5.1

  • Dropped support for PHP 5.3.
  • Added ‘logger’ config parameter to caches so that they may be injected when each cache is created.
  • Compatibility updates for our upstream projects.

Backing Up with Puppet and rsnapshot

One of my favorite backup tools has always been rsnapshot. It’s based off of rsync and uses a nice trick with hardlinks to maintain incremental updates that are also full updates. It runs using a basic configuration and a series of cron jobs. This is unix as it’s meant to be- extremely light weight while also being very powerful.

I am rather picky with how it is set up though. I don’t like leaving root open over ssh, which means a sudo based solution is needed on the client side. I’m also rather paranoid, which means I like my backup solutions to be read only. I also don’t like all of my machines running off of a single rsnapshot configuration, as this means a failure for the script to run on one means it won’t run on the ones after.

For years I had a set of scripts to handle this, but in the days of configuration management that seems almost silly. To make life easier I’ve put this all in a Puppet module.

There are quite a few features to this module that make it stand out-
Continue Reading

Github Enterprise Backups with Puppet

The amount and value of data stored on Github Enterprise servers is quite large, and backing them up is rather important. To keep people from doing desperate restores from cloned repositories Github has a tool for backing up and restoring GHE installs. To make this even easier I’ve put together a Puppet module that takes care of the basic setup and configuration of the GHE backups tool.

Combining this module with your existing Puppet environment makes GHE backups easy-

class { 'ghebackups':
  ghe_hostname => 'github.example.net',

In fact if your network will resolve “github” on your search domains you can even skip the ghe_hostname parameter.

Storing backups with a custom location and number of snapshots is not much more difficult.

file { '/backups/github':
  ensure => 'directory'

class { 'ghebackups':
  ghe_hostname      => 'github.example.net',
  ghe_data_dir      => '/backups/github',
  ghe_num_snapshots => 72,

This module also keeps the backup utility up to date and takes care of some other minor issues around installing this package.

For full documentation view the module on Puppet Forge, and as always contributions are always welcome on Github.

Hieratic – a new way to use Puppet and Hiera

Today I’m announcing the release of Hieratic, a new Puppet module designed to improve the integration between Puppet and Hiera.

Hiera tries to improve Puppet by taking site specific data out of manifests. Hiera makes it possible (and even easy) to define complex sites by making general configurations and then defining only the differences between each node. This can be done on a very granular (and configurable) level. If you haven’t played with Hiera yet take a minute to read through it’s overview and bask in the awesome.

As much as I love it though Hiera is not without flaws. Since it wasn’t shipped by default in Puppet until 3.0 many modules do not have built in support for it. Automatic Parameter Lookup partially solves this problem, but has some serious limitations that significantly reduces it’s usefulness- the biggest being that it is incapable of merging values from multiple levels and only provides the highest priority items.

Hieratic solves this problem. It lets you define a variety of a resources, including classes, while still allowing deep merging to occur. With Hieratic it’s possible to completely separate Puppet code from configuration.
Continue Reading

Fetch v0.6.1 Released

This update has a lot of bug fixes and optimizations, particularly around character encoding, that have been submitted by over a dozen contributors. This release also contains two new functions- listMailboxes and getOrderedMessages.

Puppet PSAD v1.1.2 Released

The PSAD Puppet Module has been updated. The major addition includes some better handling of the PSAD service checks, which should result in a faster check of the PSAD service and remove some potential warnings on versions of Ubuntu. Additionally some documentation and formatting updates have been made.

Special thanks to netson for their contributions!