There’s been a lot of discussion about PSR-6, the php-fig caching interfaces, so I thought it was time to step in and describe what this system is all about. Be prepared to read far more about caching interfaces than you probably thought possible.
I’m a firm believer in hosting my own email and other services, and after running some updates to deal with Shellshock I realized it was time to replace my provisioning scripts and bring my personal systems under proper configuration management. It was while doing this that I found out there isn’t a PSAD module for Puppet! This is something I just couldn’t allow, so I’ve spent some time over the last few days fixing that problem.
PSAD is one of my favorite security tools for services. It works with the existing firewall to detect post scans and actively block them. It has the ability to persist data, meaning it can catch scans that are occurring over long periods of time, and it has a variety of configuration options to suite all needs. Further, because it’s using the firewall logs themselves you can add and remove new firewall rules without needing to change any of the PSAD configurations- whitelisting an IP address is a simple matter of adding that change to your firewall.
The PSAD module handles everything you need to get PSAD up and running.
- It can be used to control the entire configuration of PSAD.
- It adds the firewall logging rules that are needed.
- It adds a cronjob to keep PSADs signatures up to date.
- Of course, it also installs PSAD and keeps the service running.
It’s hard to stress how much work went into these two releases. For Stash there was 139 commits and the Stash Bundle itself had another 103. Much of this was internal changes designed to improve performance and ease future development, but there are also a lot of goodies that you’ll want to start exploring.
New Documentation Site and Github Organization
Stash and the Stash Bundle have both been moved out of my personal Github account and over to a new organization. At the same time we’re launching a brand new site, StashPHP.com, for all of the Stash documentation. A special thanks to the folks at the Guzzle project for releasing their Sphinx theme to the public, as that was the basis for the new site.
Full Support for HHVM
This latest release is the first to commit to supporting HHVM. Every new change will be tested against both the stable HHVM and the nightly builds, and any problems that come up will be resolved before being released.
Stash has already has great support for grouping cached items together using Stacks, and with this latest release it adds Namespaces to the mix. These features are redesigned to work together, allowing each namespace to contain it’s own completely independent groups of items.
For more information on how to use these features in your project check out the Groupings documentation on the Stash site.
Public API Additions
Besides the Namespace features there are a few new public functions available. The DriversList class has been expanded to allow more data to be pulled out about what drivers are available on the system as well as making it possible to register custom drivers. The Item class also contains new functions which pull out metadata about when an Item was first created and when it is expected to expire.
Intensely Expanded Test Suites
Testing is by far one of the most important parts of the Stash library. Stash’s test suite contains 162 tests and over 4,200 assertions, which act to make sure that all drivers behave in a consistent manner and that all data that goes into Stash comes out as it should.
The StashBundle had not, until this point, been tested nearly as thoroughly. That has changed with this update, and the StashBundle now has 98% code coverage, with 94 tests and 834 different assertions.
Major Rewrite and Expansion of Documentation
All of the documentation has been reviewed and updated, including the PHPDoc and inline commenting. New sections have been added to help people Extend Stash for their purposes, Add New Drivers for systems not currently supported, and to Integrate Stash into their projects.
Lots of Little Things
Although a lot of big things have been mentioned, there’s numerous smaller changes that help make Stash better. For more information as well as a list of all potential BC breaks please check out the Changelogs for Stash and StashBundle.
In the most recent release of JShrink I pledged full HHVM support, and since then have been running our full test suite against the 3.0 release as well as the nightly builds. Well, it seems that HHVM has itself committed to supporting JShrink, as they’ve added it to their own framework compatibility test suite.
For the last week JShrink has been present on the HHVM Compatibility Dashboard, where it is one of 24 projects that are working at 100%. I’m proud to publish code that supports HHVM, and am very excited that JShrink is one of the projects used to test compatibility of HHVM against the existing php ecosystem.
JShrink has been in development for several years now. It’s migrated from Google Code to Github and modernized with support for things like Composer and the PSR standards. It’s been adopted by numerous projects (SugarCRM and Piwik are two examples), and extended by other developers for use with Assetic and Symfony2. Through Packagist along it has had over 32k downloads (over 5k of which was just last month). In recent months a lot of work has been put into increasing test coverage, adopting standards, and making contributions easier. This latest version of JShrink is the fastest, best documented, and most comprehensively tested version yet.
All of which leads to this, JShrink 1.0.
Thank you to everyone who has contributed to and adopted JShrink!
This release of Stash deals with changes between the APC and Opcode Caching system in PHP 5.4, specifically how opcode caches get invalidated. As of this release the Filesystem Driver explicitly invalidates the opcode cache for the files it creates or updates, preventing PHP from including an older version of it’s data.
This release handled a few bugs and improves integration with the testing suite, particularly for people running it via Vagrant. The biggest benefit with testing is that the staging environment is now created and reset automatically when phpunit is run, rather than needing to be initialized separately. This release also adds better support for messages with special encoding.
As I was poking around the Packagist website this morning I found that Stash has reached 40k downloads through composer! What’s really exciting is that 2,135 of those downloads were just in the last 10 days.
This is an exciting milestone for me. Stash has been in active development since 2009, but was only moved to Github 2 years ago- around the time that it was made available through Packagist. Since then the community has rallied behind it- with 23 different contributors, a fork supporting PHP 5.2, bundles for popular frameworks like Symfony (thanks Josh!), and adoption of projects like EZPublish, the growth of this project has been phenomenal.
Thanks to all of the community for their help in developing Stash! I’m looking forward to my next update on Stash, where I’ll talk a bit more about what makes it such a great caching library.
In my previous post I introduced the testing suite I created for Fetch. Here I want to go through exactly what needed to be done to put that together. This post is a bit longer, and you really don’t need it to take advantage of this package, but it may provide some insight to anyone hoping to do something like this themselves.
For me testing has two major components- it should ease development on the local level by preventing regressions, and it should help me as the maintainer of an open source project by giving me trust in the pull requests that are coming in to me. Travis-CI, an application I’ve been using extensively for my projects now, makes that last problem much easier as it will run my test suites and tell me right in the pull request if any problems have come up. On the local level I decided to use Vagrant, as it makes adding virtual machines to source control (without adding a huge binary file) really simple.
Years ago I wrote a library, Fetch, which was designed to read email using the PHP IMAP Extension. At the time I did not expect it to get much reuse so I skipped out on much of the test suite- if I’m going to be perfectly honest here though I have to admit that a huge reason for not building out a test suite then was that I was unsure how to go about it. Building a test suite for an email library has some very interesting requirements, especially if you want to test it in as real world a setting as possible.
- You need a mail server.
- That mail server needs a variety of messages, folders, flags and attachments for all the for different types of tests that are needed.
- Those messages have to remain consistent between test runs.
- Resetting the mail server to it’s original state should be fast to encourage lots of testing during development.
- The test suite, and thus the mail server, need to be portable so other people can run tests on their changes.
- Continuous integration (integrated in with something like Github) is really really nice.
Unfortunately for me all of my great excuses have vanished, thanks in large part to Travis-CI and Vagrant. Both of these systems are fantastic for testing. Travis is a continuous integration environment that’s setup for a number of languages and has direct integration with a number of services such as Github. Vagrant makes sharing server environments as easy as passing along a configuration file. With the two of these tools I was able to put together a testing package for Fetch that met all of those requirements, and after a bit of additional work I’ve broken that up into a stand alone package for people to use in their own projects.
Using this is really simple- you just need to call SetupEnvironment.sh before each run of your test suite. Once that is finished running you have a fully functional Dovecot server with a consistent set of messages. It doesn’t matter if you’re running this from your home computer or if it’s run on Travis-CI, as the SetupEnvironment.sh script will recognize where it’s being run and work accordingly.
There are also some cool optimizations to make testing faster. The test environment will stay up and running for a half hour after the last test before shutting itself down, so you don’t have to wait for the virtual machine to boot between tests. Rather than start from a blank slate each time, the environment has a reset function that restores the original inboxes. With an already provisioned test environment there is almost no overhead by this package between tests, as resetting the inbox takes only a couple of seconds.
The project is up on Github at tedivm\DovecotTesting, so if you’re writing an email reading library there’s no more excuses. I’ve also put together a more detailed post describing how this was put together.