Adobe InDesign script to pull newspaper stories from Airtable API

Over on my newspaper publisher blog I recently shared about things we’ve been doing to automate pieces of the newspaper’s print layout process:

One of them that was in progress at the time was a script to fetch articles and story content from our Airtable-managed story database for faster placement on the page, instead of copying and pasting.

That script is now in production and while it still has some rough edges, is also open sourced on GitHub. I thought I’d go into a little more detail here about how it works:

  1. Layout editor opens a page file they want to do layout on
  2. Layout editor runs InDesign script
  3. Script determines what page is being worked on based on the filename
  4. Script makes a call to a remote API with the page number as a query parameter, to see what stories are available and ready to be placed on that page, and gets them as a JSON data structure
  5. Using a base story layer that exists in our page template, script creates a new story layer with the headline, subhead, byline, story content, etc. filled in from the JSON data
  6. Script finds and replaces Markdown syntax in the content with established InDesign styles for bold, italics, body subheaders, bullet points, etc.
  7. Layout editor drags the new layer(s) into place, adjusts dimensions, and marks the stories as placed
  8. Rinse and repeat

We’re also working toward adding support for images and captions/cutlines.

This turns what could be a tens or hundreds of clicks process for a given story into just a few clicks. It saves more time on some stories than others, but especially for the ones that involved a lot of applying inline character styles that were being lost or mangled during copy/paste, I think it’s a clear win.

A screen capture of an InDesign layout window running the discussed script and importing story layers onto the page.

I referred to a “remote API” above because even though our stories are managed in Airtable right now, I chose to introduce an intermediate API for the InDesign script to call for simplicity and so that we weren’t locked in to Airtable’s way of doing things.

In our case, it’s implemented as a single action controller in Laravel, which essentially proxies the query on to the Airtable API and maps out a new, simpler data structure from the result with some content cleanup thrown in along the way:

$airtableResults = $this->getAirtableStories($pageNumber, $nextIssueDate);

$stories = $airtableResults->map(function (array $item) {
	$content = $this->processContent($item['fields']['Story Content']);
	return [
		'id' => $item['id'],
		'headline' => $this->extractHeadlineFromContent($content, $item['fields']['Story']),
		'subhead' => $this->extractSubtitleFromContent($content),
		'byline' => $this->getByline($item),
		'body' => $content,
	];
});

return response()->json([
	'status' => 'success',
	'count' => $stories->count(),
	'data' => $stories->toArray(),
]);

You can tell that there are some hacky things going on with how we represent headlines, subheads and bylines in the content of our stories, but that’s a blog post for another time.

Here’s what a response might look like:

{
  "status": "success",
  "count": 2,
  "data": [
    {
      "id": "record_id_1",
      "headline": "My first headline",
      "subhead": null,
      "byline": "From staff reports",
      "body": "Lectus viverra cubilia.",
      "image_url": "https://placehold.co/600x400",
      "cutline": "My image caption 1 here"
    },
    {
      "id": "record_id_2",
      "headline": "A second story",
      "subhead": "You should really read this",
      "byline": "By David Carr",
      "body": "Lectus viverra cubilia.",
      "image_url": null,
      "cutline": null
    }]
}

The README at the root of the repository explains further how the API query and response should work and includes some basic installation instructions. (If you use this, I recommend including some light caching so that you don’t over-query the Airtable API, or wherever your stories are stored.)

I realize everyone’s layout and story management tools are different and that the likelihood that our workflow matches up with yours is slim…but just in case this is helpful to anyone else, I wanted to get it out there! Let us know what you think.

Tools and tech we’re using to publish a print, online newspaper

Wow, it’s been over a month since I took ownership of a print and online newspaper here in my community. There’s a lot to say about that experience and what I’ve learned so far. In this post, I’ll be focused on the tools and technology we’re using  to operate this business. Some of these were in place before I came in, some are new in the last month.

I’m sharing this because (a) I generally enjoy the topic of if/how tools can make life and business easier, and (b) I hope it could be useful to someone else publishing a newspaper or building a media organization.

Print Layout and Design

It’s Adobe Creative Cloud all the way, for better or worse. InDesign for newspaper layout, Photoshop for image editing. Given the way our staff is set up and our weekly newspaper production process works, almost everyone touches the newspaper pages at some point or another, so the monthly license costs to cover all of that is somewhat ouch. If there were a viable alternative to InDesign, we’d probably switch to it.

Issue and Story Budget Planning

We’re using an Airtable base that helps us record story ideas and plan for our upcoming issues by tracking what articles are going to go where, what state their in, and all the associated data that goes with them such as photos, source info, internal notes, etc. It’s pretty great and the real-time collaboration that it makes possible is hard to beat. I think down the road we may move toward a custom Laravel-powered solution that allows for tighter integration of all of our business operations, but that’s a ways off.

Phone System

We’re using a self-hosted FreePBX (Asterisk) installation with the Sysadmin Pro and EndPoint Manager paid add-on modules. Digital Ocean had a 1-click installer on their marketplace that made it super fast to get going. We’re using VOIP.ms for our trunk lines and they made DID porting in very easy.

Having used Asterisk in a previous business I was already familiar with its architecture and features, but FreePBX meant I could configure everything via web interface instead of editing dialplan files – amazing. We have extensions, queues, interactive voice menus, voicemail speech to text transcription (using this tool) and more, and it sets up a nice foundation for future integration with other tools like our CRM data.

We’re using Yealink T31P and T33G VOIP phones and so far Counterpath’s Bria Mobile has been the most compatible/feature complete softphone for iOS that I’ve found.

Continue reading Tools and tech we’re using to publish a print, online newspaper