A couple times per year, WordPress plugin authors and owners get an email like this one:
WordPress 5.6 is imminent! Are your plugins ready?
You’re receiving this email because you have commit access or ownership of existing, open plugins hosted on WordPress.org. The next release of WordPress, 5.6, is scheduled for 08 December 2020.
We would like you to take this time to review your existing plugins and ensure their ongoing compatibility with WordPress. Once you’ve done so, you can update the readme “Tested up to:” value to 5.6. This information provides peace of mind to users and helps encourage them to update WordPress.
Here are the current “Tested up to:” values for each of your plugins:
The message goes on from there to list the plugins I’m responsible for and some notes and details about what’s new in the upcoming WordPress release.
In case it’s not clear, this is an important moment because the authors of tens of thousands of WordPress plugins are being asked to help ensure that when the many millions of WordPress sites out there upgrade to the upcoming release, that those sites continue to look and function as expected by their users. It’s an impressive example of how the WordPress developer community works together in the background to help sustain and grow the larger WordPress ecosystem.
For authors of widely used plugins, by the time this email goes out their plugin may already be fully ready, especially if they’ve been following or maybe even contributing to the development of the new WordPress core release. Some plugin authors rightly have an extensive automated test suite in place to confirm that every part of their plugin’s functionality works against the latest beta or release candidate version of the new version before it comes out.
Authors and maintainers of smaller plugins (like me) may not have the same infrastructure set up, and instead need to perform some manual testing of our plugins to ensure they’re ready.
So, here are the steps I follow every major release cycle to make sure my plugins have been tested and are ready for the new version.
Setting up a testing environment
First I make sure my local testing environment is ready to go. I use VVV for almost all of my WordPress-related development work, and I have a dedicated “release testing” site setup within VVV for this particular purpose. Here’s the site’s configuration in my config.yml file:
releasetest:
repo: https://github.com/ChrisHardie/jch-vvv-provision.git
skip_provisioning: false
hosts:
- releasetest.test
The repo value there defines a public GitHub repository I maintain with some scripts/files that define how my WordPress development sites are provisioned. As you’ll see from the vvv-init.sh file, here are some of the important things it handles for me when the environment is set up:
- Download, install and perform basic config for the latest version of WordPress
- Empty out the site so any test content is gone
- Set some config variables for development purposes, including WP_DEBUG, WP_DEBUG_LOG, WP_DEBUG_DISPLAY, SAVEQUERIES, FORCE_SSL_ADMIN, and JETPACK_DEV_DEBUG.
- Install some plugins I use in developing almost every site, including Jetpack, Query Monitor, WordPress Importer, and Post Meta Inspector.
It’s pretty nice to have all of this taken care of automatically when starting a new project or refreshing an environment.
There are a variety of other tools and configs related to setting up VVV and developing for WordPress that are out of scope for this post, including having a good IDE (I use PhpStorm) with a code standards sniffer (I usually use phpcs with the “WordPress” and “WordPress-Extra” rulesets).
The one other tool I want to mention explicitly is a way to deploy directly from a GitHub repository to the SVN-powered WordPress.org plugin repository. The slickest option is the GitHub action available from 10up, which will release a new plugin version for you when you do a release on GitHub.
I personally still use this similar script, which does the deployment work from my local environment. More on what it looks like when it runs in a bit, but here are the changes I make to it once I’ve downloaded it so that the default values work for all of my runs of the script:
-default_svnpath="/tmp/$PLUGINSLUG" +default_svnpath="/Users/chris/develop/wp-svn-temp/$PLUGINSLUG" -default_svnuser="GaryJ" +default_svnuser="ChrisHardie"
There should be a better way for the script to find changes to the default config (e.g. a config file it reads) but for now I have those two changes saved as a patch file so I can quickly apply it to future updates to the script:
$ patch < deploy.patch
Testing each plugin
Now it’s time to test the plugins themselves. For each plugin I’m responsible for, I follow these steps:
In the plugins directory of the releasetest WordPress install, I clone the GitHub repository of the plugin I want to test. Or, if I’ve already cloned it in the past, I do a git pull to make sure I have the latest version for testing.
In a separate terminal window, I start doing a tail -F debug.log on the log file that WordPress writes to when there are PHP warnings or errors, so that if any action I take in my testing generates an error message, I can investigate it further. I also open my browser’s Javascript console so I’ll see any errors going by there.
Then, I take a variety of actions to see if everything works:
- Activate and deactivate the plugin.
- If the plugin depends on other plugins, try various combinations of what happens when the dependencies are present/not-present, active/not-active.
- Test any admin-facing user interface elements to make sure they work as expected
- Test any user-facing interface elements
- Check the code against code standards with phpcs (e.g.
$ phpcs --standard=WordPress pet-adoption-search-widget.php)
The testing process for some plugins is more extensive than others.
Again, it would be wise to automate some of this testing for any non-trivial plugin functionality since having to remember or document each plugin’s edge cases and areas of functionality doesn’t scale very well.
Updating the plugin files
If it looks like I will need to make changes to the plugin to make it compatible with the upcoming WordPress release, then I create a git feature branch, make the updates, test some more, and then merge that into the main branch when it’s ready.
After that I tag that merge commit as a new version of the plugin, and push that release up to GitHub:
$ git checkout main $ git pull $ git tag -a 1.5.1 <merge_commit_hash_here> $ git push origin 1.5.1
If the only change I’m making is to indicate that the plugin has successfully been tested against the new version of WordPress, then I just edit the readme.txt file and change the “Tested up to:” header to the right value (in this case, 5.6). I still do that in a feature branch and push/merge it on GitHub, but no tagging or releasing is needed.
Deploying to the WordPress.org directory
As noted above, I use this script to handle the final step of deploying the plugin’s GitHub main branch to the WordPress.org SVN servers. Here’s what an example run looks like, where I accept all the default values after answering the first question:
$ sh deploy.sh WordPress Plugin SVN Deploy v4.1.0 Let's collect some information first. There are six questions. Default values are in brackets - just hit enter to accept them. Q1. WordPress Repo Plugin Slug e.g. my-awesome-plugin: wp-lookout Q2. Your local plugin root directory (the Git repo). (/Users/chris/develop/vagrant-local/www/releasetest/public_html/wp-content/plugins/wp-lookout): Q3. Your local repository directory for SVN assets. (.wordpress-org): Q4. Name of the main plugin file (wp-lookout.php): Checking version in main plugin file matches version in readme.txt file... wp-lookout.php version: 1.0.1 readme.txt version: 1.0.1 Versions match in readme.txt and wp-lookout.php. Let's continue... Q5. Path to a local directory where a temporary SVN checkout can be made. Don't add trunk (/Users/chris/develop/wp-svn-temp/wp-lookout): Q6. Remote SVN repo on WordPress.org. (https://plugins.svn.wordpress.org/wp-lookout): Q7. Your WordPress repo SVN username (ChrisHardie): That's all of the data collected. Slug: wp-lookout Plugin directory: /Users/chris/develop/vagrant-local/www/releasetest/public_html/wp-content/plugins/wp-lookout Main file: wp-lookout.php Temp checkout path: /Users/chris/develop/wp-svn-temp/wp-lookout Remote SVN repo: https://plugins.svn.wordpress.org/wp-lookout SVN username: ChrisHardie OK to proceed (Y|n)? .......................................... Preparing to deploy WordPress plugin .......................................... Changing to /Users/chris/develop/vagrant-local/www/releasetest/public_html/wp-content/plugins/wp-lookout Git tag 1.0.1 does exist. Let's continue... Creating local copy of SVN repo trunk... A /Users/chris/develop/wp-svn-temp/wp-lookout/assets A /Users/chris/develop/wp-svn-temp/wp-lookout/branches A /Users/chris/develop/wp-svn-temp/wp-lookout/tags A /Users/chris/develop/wp-svn-temp/wp-lookout/trunk Checked out revision 2429945. Ignoring GitHub specific files Using default .svnignore property 'svn:ignore' set on '/Users/chris/develop/wp-svn-temp/wp-lookout/trunk' Exporting the HEAD of master from git to the trunk of SVN Moving assets. A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/icon-128x128.png A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/icon-256x256.png A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/banner-1544x500.jpg A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/screenshot-1.png A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/screenshot-2.png A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/banner-772x250.jpg A (bin) /Users/chris/develop/wp-svn-temp/wp-lookout/assets/screenshot-3.png Changing directory to SVN and committing to trunk. Sending readme.txt Transmitting file data .done Committing transaction... Committed revision 2429946. Updating WordPress plugin repo assets and committing. Tree conflict at 'banner-1544x500.jpg' marked as resolved. Tree conflict at 'banner-772x250.jpg' marked as resolved. Tree conflict at 'icon-128x128.png' marked as resolved. Tree conflict at 'icon-256x256.png' marked as resolved. Tree conflict at 'screenshot-1.png' marked as resolved. Tree conflict at 'screenshot-2.png' marked as resolved. Tree conflict at 'screenshot-3.png' marked as resolved. Replacing banner-1544x500.jpg Replacing banner-772x250.jpg Replacing icon-128x128.png Replacing icon-256x256.png Replacing screenshot-1.png Replacing screenshot-2.png Replacing screenshot-3.png Transmitting file data .......done Committing transaction... Committed revision 2429947. Creating new SVN tag and committing it. Updating readme.txt to tag 1.0.1 D tags/1.0.1/readme.txt A tags/1.0.1/readme.txt Replacing readme.txt Committing transaction... Committed revision 2429948. Removing temporary directory /Users/chris/develop/wp-svn-temp/wp-lookout. *** FIN ***
That’s it! As soon as the script is done running, I confirm on the plugin page in the directory that the “tested up to” value is now displaying as 5.6. I also receive some email confirmations of my SVN commit.
Final thoughts
For someone with 9 active plugins that I maintain, this process typically takes me an hour or two each time. It varies depending on how long it’s been since I refreshed my release testing environment (environment updates can always lead to unexpected issues), how thoroughly I test, whether the WordPress release itself has substantial changes that affect my plugins, and whether I get sidetracked in making other improvements to my plugins while I’m in there.
Ideally there would be some better tooling for plugin maintainers and authors to complete this process. A basic improvement would be to have that notification email link to a guide reminding plugin authors how to test and update their plugins. Also, while there’s no substitute for manual testing I could imagine that some of the tests that WordPress core runs when it auto-updates a plugin in production could also be proactively run against a plugin using a beta or release candidate of WP core, and the summarized results of that included in the email message to authors.
Even as things are now, I’m happy to spend a few hours (more or less) a few times per year to ensure that my plugins are ready for users to start or continue using them without problems.
Do you maintain WordPress plugins and have your own process for testing against upcoming core releases? Please share!
