Chris Hardie's Tech and Software Blog

Tools and tech we’re using to publish a print, online newspaper

Wow, it’s been over a month since I took ownership of a print and online newspaper here in my community. There’s a lot to say about that experience and what I’ve learned so far. In this post, I’ll be focused on the tools and technology we’re using to operate this business. Some of these were in place before I came in, some are new in the last month.

I’m sharing this because (a) I generally enjoy the topic of if/how tools can make life and business easier, and (b) I hope it could be useful to someone else publishing a newspaper or building a media organization.

Print Layout and Design

It’s Adobe Creative Cloud all the way, for better or worse. InDesign for newspaper layout, Photoshop for image editing. Given the way our staff is set up and our weekly newspaper production process works, almost everyone touches the newspaper pages at some point or another, so the monthly license costs to cover all of that is somewhat ouch. If there were a viable alternative to InDesign, we’d probably switch to it.

Issue and Story Budget Planning

We’re using an Airtable base that helps us record story ideas and plan for our upcoming issues by tracking what articles are going to go where, what state their in, and all the associated data that goes with them such as photos, source info, internal notes, etc. It’s pretty great and the real-time collaboration that it makes possible is hard to beat. I think down the road we may move toward a custom Laravel-powered solution that allows for tighter integration of all of our business operations, but that’s a ways off.

Phone System

We’re using a self-hosted FreePBX (Asterisk) installation with the Sysadmin Pro and EndPoint Manager paid add-on modules. Digital Ocean had a 1-click installer on their marketplace that made it super fast to get going. We’re using VOIP.ms for our trunk lines and they made DID porting in very easy.

Having used Asterisk in a previous business I was already familiar with its architecture and features, but FreePBX meant I could configure everything via web interface instead of editing dialplan files – amazing. We have extensions, queues, interactive voice menus, voicemail speech to text transcription (using this tool) and more, and it sets up a nice foundation for future integration with other tools like our CRM data.

We’re using Yealink T31P and T33G VOIP phones and so far Counterpath’s Bria Mobile has been the most compatible/feature complete softphone for iOS that I’ve found.

Continue reading Tools and tech we’re using to publish a print, online newspaper

Define, fire and listen for custom Laravel model events within a trait

It took me some time to figure out the right way to define, fire and listen for custom events for a Laravel model using a trait, so I’m writing down how I did it in case it’s helpful to others.

Let’s say you have a model Post that you set up as having a “draft” status by default, but eventually will have a status of “publish”. Let’s also say you want to make the act of publishing a post a custom model event that can be listened for in addition to the standard model events like “created” or “updated”. And let’s say you want to do all of this using a trait so that you can apply the same logic to another model in the future, such as comments on the post, without repeating yourself.

Here’s what my Post model might look like:

<?php

namespace App\Models;

class Post extends Model
{
    //
}

Let’s create a Publishable trait that can be applied to this model:

<?php

namespace App\Models\Traits;

trait Publishable
{
    // Add to the list of observable events on any publishable model
    public function initializePublishable()
    {
        $this->addObservableEvents([
            'publishing',
            'published',
        ]);
    }

    // Create a publish method that we'll use to transition the 
    // status of any publishable model, and fire off the before/after events
    public function publish()
    {
        if (false === $this->fireModelEvent('publishing')) {
            return false;
        }
        $this->forceFill(['status' => 'publish'])->save();
        $this->fireModelEvent('published');
    }

    // Register the existence of the publishing model event
    public static function publishing($callback)
    {
        static::registerModelEvent('publishing', $callback);
    }

    // Register the existence of the published model event
    public static function published($callback)
    {
        static::registerModelEvent('published', $callback);
    }
}

This new trait can now be applied to the Post model:

CrowdTangle API SDK in PHP

After not finding anyone else who has done so, I created a minimal PHP implementation of the CrowdTangle API, which I needed anyway for a project I’m working on:

GitHub repo: https://github.com/ChrisHardie/crowdtangle-api
Packagist repo: https://packagist.org/packages/chrishardie/crowdtangle-api

Example usage syntax:

$client = new ChrisHardie\CrowdtangleApi\Client($accessToken);

// get lists
$client->getLists();

// get accounts in a list
$client->getAccountsForList($listId);

// get posts
$client->getPosts([
    'accounts' => '12345678',
    'startDate' => '2022-03-01',
]);

// get a single post
$client->getPost($postId);

I’m sure there’s plenty to improve but I hope it’s helpful to anyone working with CrowdTangle in PHP.

Locking Adobe InDesign files for editing in shared network or cloud folders

Today I had a chance to help a print publication solve a workflow challenge that is apparently very common.

If you open an Adobe InDesign layout file from a local folder on your computer, the software creates an Adobe InDesign Lock File (.idlk) in that folder, which prevents the same file from being opened by another copy of InDesign. But if the file exists in a folder that is shared via network or cloud service, InDesign does not create a lock file when the InDesign file is opened for editing. This includes Adobe’s own “Creative Cloud” file sharing option.

There may be good technical reasons for not creating or syncing lock files across network folders, but the end result is that multiple users can decide to open the same file at the same time, and whomever saves their changes last will “win,” with the other user’s changes being lost.

In researching this, I found it was not some edge case. There seem to be many newsrooms and other organizations struggling with this every day. They work around it by doing things like copying the InDesign files to a local folder, making their changes, and then uploading back to the shared folder, hoping that the internal communication about such things is sufficient along the way. This Adobe Community Support forum thread illustrates the pain points involved.

Thankfully, Adobe InDesign is a scriptable software tool, and so
Max Schmidt and the folks at t3n created a script that creates a locking system for network shared InDesign files. And there was much rejoicing!

After installing the script on all the devices that will be accessing shared InDesign files, anyone trying to open a file that someone is already editing gets an error message and then the file is closed. (I submitted a Pull Request to expand the README documentation so the installation process is a bit clearer.)

In the long run, Adobe needs to solve this problem in a more standard way for its users. But this script is a great alternative option and I hope bringing some additional attention to it helps out other publications or news rooms that might be struggling with the same challenge.

An example of why RSS is useful and important

Let’s say I want to be an informed, engaged local citizen. Let’s say my town has a website set up where they post local updates and alerts that I might care about.

Specifically, let’s say there was a water main break a few days ago that led to everyone in the town being asked to boil water, and that I wanted to know when that boil advisory was over.

Sure, the information is right there on their website, so one option is to just reload the website all day and wait for an alert to pop up:

But what if my goal and wish is to have access to that information in a programmatic way? Such as, say, RSS, the technology that has long allowed websites to share information with each other in a format that other websites and software can read?

Once I have that, I could read the alert in a news reader, I could have it piped to a Slack or Discord channel, I could make a little notification appear on my computer desktop, make the lights flash using IFTTT or Zapier, or whatever was most useful to me. The steps would be:

Get RSS feed link.
Do whatever I want to with the information in the feed.

But this town does not offer an RSS feed on its website. Not for alerts, not for regular community news, nothing. (To make matters worse, it’s powered by web software that appears aimed at helping smaller towns and cities have a web presence, so we know this missing feature is now being repeated in communities across the world!)

What are my other options?

Continue reading An example of why RSS is useful and important

My standard Laravel development tools

Now that I’ve been actively developing applications with the Laravel framework for a few years, I thought I’d write down the tools and services I tend to use on a regular basis in that work.

I’ve spent a fair amount of time researching and experimenting with these tools and their alternatives in order to make a choice, so maybe it will help someone else who hasn’t gone through that yet. I’m always glad when others share details about their development environments so that the rest of us who are just getting going can build on that foundation.

Hardware and Development Environment

Launching a New Project

composer global update laravel/installer
laravel new example-app --git --branch="main"
cd example-app
valet link
valet secure example-app

Then I create a database, add the DB info in .env, run artisan migrate, and I’m ready to develop. Sometimes I have to make sure PhpStorm has the right coding standards and PHP Code Sniffer config in place.

Sometimes I add a .psysh.php file to my project repo with these contents:

<?php
DB::listen(function ($query) {
    dump("[{$query->time}ms] {$query->sql}");
    if ($query->bindings) {
        dump($query->bindings);
    }
});

When using artisan tinker, this prints out any SQL queries that were run by a given command within the tinker session, for faster debugging.

For WordPress projects, I use a customized version of this “valetpress” bash script to initialize new projects.

Continue reading My standard Laravel development tools

Customizing Newsletter Glue for WordPress post notifications

I love the Newsletter Glue plugin for WordPress. Before I found it, I had tinkered for a long time with different imperfect solutions to what seemed like a simple need: quickly and easily send people an engaging, visually appealing email when I publish a new article on my website.

I’ve tried manually sending out Mailchimp campaigns after publishing (too slow and cumbersome to customize per post), using messages sent directly from my WordPress host server (too prone to being flagged as spam), automatically generating Mailchimp campaigns from an RSS feed (limited scheduling options and prone to errors), using WordPress.com new post notifications via Jetpack (not very customizable), developing my own plugin to send on demand via MailChimp’s API (very functional but not easily re-usable across sites), and others.

Alas, nothing felt quite like what I wanted.

I’d revisit the challenge now and then, especially when I’d see everybody over there loving on Substack like “oh isn’t amazing how you can send out a readable email newsletter from a website, what an innovation” and found myself rolling my eyes especially hard. “SURELY WE SHOULD BE ABLE TO DO THAT WITH WORDPRESS!” I’d exclaim. And then my family would look at me with tilted heads before going back to whatever they were doing.

I was about to start putting some serious time into turning my own proof of concept Mailchimp API plugin into something reusable and shareable, when I did one last round of research into existing options. And that’s when I found Newsletter Glue. The sky opened up. Light shone down. I did a dance. At least that’s what it felt like. (I see that Justin Tadlock at WP Tavern has had a similar experience.)

So, yes, ahem, where was I? Oh, right: Newsletter Glue is an elegant solution to a real need in the world of WordPress publishing. Go check it out if you haven’t already. I bought a 5-site license and sent Lesley Sim, one of the plugin’s co-creators, a note of appreciation.

The rest of this blog post is about a few additional Newsletter Glue customizations I set up. While some of this is possible via the Newsletter Glue UI, I did it via a small custom plugin that I could re-use across all of my WordPress sites without additional configuration.

Unlocking email content into RSS feeds redux with WordPress and Postie

As a part of some local journalism projects I’m exploring, I wanted to have a way to get information that is being emailed around (press releases, newsletters) into a publicly accessible RSS feed.

I’ve already explored this general “unlock email into an RSS feed” workflow using Zapier but Zapier’s limitations around translating HTML email messages into useful RSS entries led me to explore other options. For a while now I’ve been using Feedly’s paid feature that lets you receive email at a custom address and puts the content into your feed reading experience, and that’s actually been a good solution for me as an individual. (I made sure to set up an address at a domain I control and aliased that to Feedly’s provided address, in case I want to move to another solution later.)

But if we want to help a given audience have better access to information that’s only available in email but is intended to be public, I don’t think it scales well to ask them all to subscribe to the same email newsletter, or to all sign up for a Feedly paid plan. And yet so many organizations continue to use email as a way to distribute information, often instead of a website, and it doesn’t scale well to beg each of them to start (or go back to?) publishing their updates on a website with an RSS feed.

I started looking at using Mailparser for a more generalized solution. Receive the emails, have Mailparser extract the information into a structured, API-queryable format, and then download that information (including any attachments and images) and put it up on a publicly accessible URL somewhere. I knew I needed a way to organize the information coming in according to the email address of the person who sent it, so I’d have to build a small application that managed that categorization during the publishing process.

And then I realized I was basically getting into CMS territory. Publishing text and media on a website. Organizing content by categories and authors. Searchable and sortable. Yeah, I know a tool that already does all that stuff really well: WordPress.

But was I still going to need to build a glue application to process the emails and create WordPress posts?

I was aware of Jetpack’s post by email feature and I think it could work well for some scenarios, but I wanted something a little more purpose built. I did a little bit of Googling and found Postie, a WordPress plugin that has great features for bringing emails into WordPress posts. I exclaimed many words of delight upon finding it, and continued to be impressed as I looked through the thoughtful documentation, the developer-oriented options for extending and customizing it, and the active support and maintenance that goes into it. The WordPress community is amazing that way. I sent the developer a donation.

So, here’s the new workflow I would use:

Email is sent to an alias at my custom domain, which goes to a free email provider with IMAP access.
Every half hour, Postie goes out and checks for new email messages and creates pending WordPress posts.
I get a notification in a Slack channel about the WordPress post, and can publish it if appropriate.
The WordPress site provides a built in RSS feed of “emails” as posts on the site.

Amazing! But I still wanted a way to organize the incoming emails and resulting posts based on sender, without using Postie’s default method of creating WordPress users for each sender.

So, I created a custom taxonomy, Sources, with some term meta fields that allow me to associate email addresses and to select whether posts from that source should be pending or published by default. (Of course, email can be forged so it’s never safe to depend on the value of the “From” address in an email to authenticate anything important.)

Here are some code snippets used to accomplish this in a custom WordPress plugin I set up to complement Postie.

Ending support for some little-used WordPress plugins

Today I am making these three little-used WordPress plugins that I created inactive, in the sense that I will no longer update them, test them against newer versions of WordPress, or provide support for them. If after a time no one else wants to take over maintaining them, they will be closed.

Each plugin has around 10 or fewer active installations and minimal or no user reviews. Given that it takes time to test each plugin I maintain for upcoming WordPress releases and PHP changes, and to support any inquiries or feature requests they may receive, I have to balance the requirements of continuing that work against the value each plugin offers. In these cases, I’d like to free up that maintenance time for other things.

I will leave the GitHub repos public indefinitely in case anyone wants to fork them or take them over.

This change does not affect the other WordPress plugins I maintain.

If you’re interested in taking over responsibility for one or more of these plugins, please contact me.

Thanks to everyone who tried them out and gave feedback along the way.

Creating a personalized, private RSS feed for users in Laravel

In building WP Lookout I had the need to create user-specific, private RSS feeds as a way for users to get updates and information specific to their account. Here’s how I did it.

I started with the laravel-feed package from Spatie, which allows you to easily generate RSS feeds, either from a model’s records or from some other method that you define. I found that someone has proposed a feature and submitted a PR to add support for signed routes, which is Laravel’s way of adding a hash key to a given URL so that it can’t be accessed if it’s been modified, perfect for something like a user-specific RSS feed. But the Spatie folks decided not to include the feature, so I had to figure out how to do it on my own.

First, I added the package to my project and published the related config file:

$ composer require spatie/laravel-feed
$ php artisan vendor:publish \
    --provider="Spatie\Feed\FeedServiceProvider" \
    --tag="config"

Then, I added a user-specific feed identifier to the User model:

$ php artisan make:migration add_feed_id_to_users_table --table=users

The migration looked like this in part:

public function up()
{
    Schema::table('users', function (Blueprint $table) {
        //
        $table->string('feed_id');
    });

    DB::statement('UPDATE users SET feed_id = ...');
}

The UPDATE statement there is used to set the initial value of the feed identifier for existing users. An alternative would have been to make the field nullable and then set it elsewhere. For new users, I set the feed identifier as users are created, via the boot() method in the User model: