Better WordPress Transients

Little Rascals' Spanky waiting on something

WordPress’s built-in transient system gets the job done, but it has a few drawbacks. Depending on how long it takes to refresh your transient and the volume of traffic you get, it may or may not be a problem. Here’s a quick overview of where WordPress falls short, and how you can fix it.

How WordPress Transients Work

WordPress transients are data stored in the database with an expiry date and time. If you have a query, request, or other action on your website that takes a long time, you can cache the result to a transient. For subsequent calls to that information, it’s a single database query, instead of whatever is involved in calculating that value.

When a user visits your page, WordPress will determine if the transient has expired. If it hasn’t, whatever value is in the database will be used. If it has expired, the user waits for the process that generates the transient’s data to take place. That data is saved new to the database and served to the user.

WordPress Transients Take Too Long to Refresh

For most cases, this is probably fine. I had a use case, however, that was much too time-consuming to expect anyone to wait.

If you visit the Sterner Stuff home page, you’ll see a section of blocks highlighting recent blog posts as well as recent activity on some of my social networks. It shows you the latest thing I’ve listened to on Spotify, the latest beer I checked in to Untappd, and the last trophy I earned on my Playstation.

Another blog post might go into how these work. The short version is this:

For Spotify, there’s a service called LastFM. Other music services can “scrobble” to your LastFM, which offers an aggregated history of your listening. LastFM offers a developer API, which makes it super quick for me to get this data. I originally cached it with WordPress transients. Because this request happened so quickly, I didn’t mind that an occasional user would have to wait for this to refresh.

Untappd and Playstation Network (PSN), however, don’t have a developer API (or not a quick and easy one). For both of these, I scrape the HTML of a public profile and crawl the DOM for what I need.

For users who hit the refresh of this transient, they’re waiting for my home page to load, which subsequently has to wait for those other two pages to load to get the information it needs. That’s like three page loads in one! Way too long.

The Original Code

Let’s get a quick look at what this code would have looked like originally:

function untappedCheckin() {
	$checkinNice = get_transient('sterner_untappd');
	if(!$checkinNice) {
		$dom = new DOMDocument;
		libxml_use_internal_errors(true);
		$profile = file_get_contents('https://untappd.com/user/eclev91');
		$dom->loadHtml($profile);
		$xpath = new DOMXPath($dom);
		$className = 'checkin';
		$query = '//*[@class="' . $className . '"]';
		$checkins = $xpath->query($query);
		$checkin = $checkins->item(0);
		$links = $checkin->getElementsByTagName('a');
		$beer = $links[2]->nodeValue;
		$brewery = $links[3]->nodeValue;
		$checkinNice = $beer.'<br>by '.$brewery;
		set_transient('sterner_untappd', $checkinNice, HOUR_IN_SECONDS);
	}
	return $checkinNice;
}

Ignoring the really gross HTML-parsing happening, you can see what’s going on. Line 2 gets the existing transient with WordPress’s built-in system. WordPress returns false when this is expired. So then we go through fetching the latest data.

Line 6 is the problem line. file_get_contents to a traditional, user-facing HTML page, during a synchronous task. This is slow. Then things are parsed, and on line 17 the transient is re-saved.

For simplicity, I’ll be abbreviating most of this logic for the rest of this tutorial:

function untappedCheckin() {
	$checkinNice = get_transient('sterner_untappd');
	if(!$checkinNice) {
		$checkinNice = get_untappd();
		set_transient('sterner_untappd', $checkinNice, HOUR_IN_SECONDS);
	}
	return $checkinNice;
}

Whew.

How Can We Solve This?

The simplest way around this should be pretty obvious. When the transient has expired, why not serve up the stale data and fetch the new data in the background, asynchronously? For this application, that strategy is gold. Our information isn’t time-sensitive. In other cases, this might not be an option.

I hate to write all that from scratch, since this is probably a pretty common use case. Fortunately, there’s a great library for this from the team at 10up. However, it isn’t wrapped in a neat little plugin. This is for advanced users, so here we go.

Implementing 10up’s Async Transients

This library is intended to be installed via Composer. Fortunately, I actually manage all my WordPress sites via Composer, so I’ve already got the autoloader working and just have to give it the ol’ composer require 10up/async-transients.

If your WordPress site isn’t already Composer-friendly, you can set up Composer and run the above command from your site root. Then require_once the `autoload.php` file it generates from your theme or plugin.

If even that sounds scary, you can download the library and require_once its Transients.php and `helpers.php` files. If I were you, I’d get to know Composer.

Next, it’s just a matter of being familiar with the syntax:

function untappedCheckin() {
	$checkinNice = \TenUp\AsyncTransients\get_async_transient( 'sterner_untappd', 'get_untappd' );
	if(!$checkinNice) {
		$checkinNice = 'Checking in with Untappd.<br>Try again shortly...';
	}
	return $checkinNice;
}

You can see we got rid of all WordPress’s built-in transient functions in exchange for 10up’s implementation. We might still get `false` back if there’s nothing in the database at all (so the first time I deployed this, for example), but that will rarely be the case. The `get_async_transient` call will return the stale database value every time, and launch a background process with the callback function to refresh it when appropriate. Of course, our callback function is still in charge of setting that value:

function get_untappd() {
	$dom = new DOMDocument;
	libxml_use_internal_errors(true);
	$profile = file_get_contents('https://untappd.com/user/eclev91');
	// ...parse the DOM into some variables
	$checkinNice = $beer.'<br>from '.$brewery;
	\TenUp\AsyncTransients\set_async_transient( 'sterner_untappd', $checkinNice, HOUR_IN_SECONDS );
}

Lickity-split for everyone.

How Could This Be Better?

There’s some room for improvement here. To make it a drop-in replacement, it’d be nice to see this wrapped into a plugin that hooks into WordPress’s existing transient functionality (on my to-do list).

This solution also has its own way of saving data to the wp_options table. So as far as I can tell, this won’t play nice with any plugins designed to let you manipulate WordPress’s built-in transients. Making the above improvement should also make this work. While you’d short-circuit WordPress’s transient logic, data could still be saved in the same way.

Thoughts? Praise? Complaints? Be sure to let me know in the comments below.


Are You a Do-It-Yourself WordPress-er?

If you aren't ready to invest in Sterner Stuff, I've got something for you! Check out my new (and free) WordPress for Small Business Owners email course!

Leave a reply

Your email address will not be published. Required fields are marked *

Folks are talking
  1. Thanks! I was looking for a solution like this and needed a pointer to 10up’s library.