Adobe InDesign script to pull newspaper stories from Airtable API

Over on my newspaper publisher blog I recently shared about things we’ve been doing to automate pieces of the newspaper’s print layout process:

One of them that was in progress at the time was a script to fetch articles and story content from our Airtable-managed story database for faster placement on the page, instead of copying and pasting.

That script is now in production and while it still has some rough edges, is also open sourced on GitHub. I thought I’d go into a little more detail here about how it works:

  1. Layout editor opens a page file they want to do layout on
  2. Layout editor runs InDesign script
  3. Script determines what page is being worked on based on the filename
  4. Script makes a call to a remote API with the page number as a query parameter, to see what stories are available and ready to be placed on that page, and gets them as a JSON data structure
  5. Using a base story layer that exists in our page template, script creates a new story layer with the headline, subhead, byline, story content, etc. filled in from the JSON data
  6. Script finds and replaces Markdown syntax in the content with established InDesign styles for bold, italics, body subheaders, bullet points, etc.
  7. Layout editor drags the new layer(s) into place, adjusts dimensions, and marks the stories as placed
  8. Rinse and repeat

We’re also working toward adding support for images and captions/cutlines.

This turns what could be a tens or hundreds of clicks process for a given story into just a few clicks. It saves more time on some stories than others, but especially for the ones that involved a lot of applying inline character styles that were being lost or mangled during copy/paste, I think it’s a clear win.

A screen capture of an InDesign layout window running the discussed script and importing story layers onto the page.

I referred to a “remote API” above because even though our stories are managed in Airtable right now, I chose to introduce an intermediate API for the InDesign script to call for simplicity and so that we weren’t locked in to Airtable’s way of doing things.

In our case, it’s implemented as a single action controller in Laravel, which essentially proxies the query on to the Airtable API and maps out a new, simpler data structure from the result with some content cleanup thrown in along the way:

$airtableResults = $this->getAirtableStories($pageNumber, $nextIssueDate);

$stories = $airtableResults->map(function (array $item) {
	$content = $this->processContent($item['fields']['Story Content']);
	return [
		'id' => $item['id'],
		'headline' => $this->extractHeadlineFromContent($content, $item['fields']['Story']),
		'subhead' => $this->extractSubtitleFromContent($content),
		'byline' => $this->getByline($item),
		'body' => $content,
	];
});

return response()->json([
	'status' => 'success',
	'count' => $stories->count(),
	'data' => $stories->toArray(),
]);

You can tell that there are some hacky things going on with how we represent headlines, subheads and bylines in the content of our stories, but that’s a blog post for another time.

Here’s what a response might look like:

{
  "status": "success",
  "count": 2,
  "data": [
    {
      "id": "record_id_1",
      "headline": "My first headline",
      "subhead": null,
      "byline": "From staff reports",
      "body": "Lectus viverra cubilia.",
      "image_url": "https://placehold.co/600x400",
      "cutline": "My image caption 1 here"
    },
    {
      "id": "record_id_2",
      "headline": "A second story",
      "subhead": "You should really read this",
      "byline": "By David Carr",
      "body": "Lectus viverra cubilia.",
      "image_url": null,
      "cutline": null
    }]
}

The README at the root of the repository explains further how the API query and response should work and includes some basic installation instructions. (If you use this, I recommend including some light caching so that you don’t over-query the Airtable API, or wherever your stories are stored.)

I realize everyone’s layout and story management tools are different and that the likelihood that our workflow matches up with yours is slim…but just in case this is helpful to anyone else, I wanted to get it out there! Let us know what you think.

CrowdTangle API SDK in PHP

After not finding anyone else who has done so, I created a minimal PHP implementation of the CrowdTangle API, which I needed anyway for a project I’m working on:

Example usage syntax:

$client = new ChrisHardie\CrowdtangleApi\Client($accessToken);

// get lists
$client->getLists();

// get accounts in a list
$client->getAccountsForList($listId);

// get posts
$client->getPosts([
    'accounts' => '12345678',
    'startDate' => '2022-03-01',
]);

// get a single post
$client->getPost($postId);

I’m sure there’s plenty to improve but I hope it’s helpful to anyone working with CrowdTangle in PHP.

Quick hack to monitor the health of a collection of AWS services

My SaaS app WP Lookout uses Amazon Web Services for hosting. I’ve had plenty of monitoring set up using both internal and external tools notifications, and these mostly output to a combination of a dedicated Slack channel and/or a Pushover notification that goes to my mobile device.

The one thing I didn’t have in place until recently was monitoring for the health of the specific group of AWS services the app depends on. With AWS offering many services spread across many parts of the world (as illustrated in the Very Long Service Health Dashboard), I wasn’t about to start monitoring all of the AWS infrastructure just to see if my particular app was affected.

They do offer a “Personal Health Dashboard” that’s supposedly more tailored to incidents affecting my AWS account, but I find it unintuitive and confusing to use, especially when it comes to setting up notifications to external services like Slack. In my online searching I found various services and tools that supposedly integrated the PHD into more standard monitoring and alerting options, but they were also either too complicated or seemed to require additional levels of paid AWS support plans that I didn’t want to take on.

So I decided to keep things simple and use the tried and true technology for polling information about external resources: RSS!

Continue reading Quick hack to monitor the health of a collection of AWS services

My first CiviCRM extension: Slack notifications

I’ve started using CiviCRM for an organization I’m working with. If you’re not familiar with it, it’s an open source constituent relationship management tool for not-for-profits, political organizations and other entities that might have a need to manage relationships with supporters, volunteers, donors, members and so on.

I’ll try to reflect more soon on the process of getting started with CiviCRM; it was a little rocky. But to get myself familiar with CiviCRM’s inner workings, I decided to write a simple extension that would allow the organization to get Slack notifications when certain objects (contacts, contributions, pledges) were created or updated. I couldn’t find any similar existing extensions in their directory or in my searching.

Here’s the settings screen:

(with apologies for the weird extra bullets, can’t seem to make them go away yet) and here’s what a resulting Slack message might look like:

The extension is available on GitHub. I’m sure there are many ways to improve it, so issues and pull requests are welcome.

Update on April 20, 2021: the extension now provides a more flexible CiviRules action for Slack notifications instead of a standalone notification method, so some of the above information is outdated.