Using XML, Twitter, and RSS Content in WordPress

Collecting External Content

You have weighed the pros and cons of aggregating your online reputation into your WordPress site. The next question is how are you going to do it? And which sites are you going to include?

Basically, you can only include sites that have an API to permit the consumption of the data. That is not entirely true; you could code up a spider that logs into the remote site and harvests the information you want, but that would be pretty obscure and not fun to maintain long term. For the purposes of this book we are going to focus on exposed APIs.

The first option to integrate services is to read the API documentation and create a plugin or other function to consume and convert the information into something WordPress can use, such as a post. Because the underlying architecture of WordPress is PHP, you can use whatever PHP tricks you have up your sleeve and code a nice solution. For more information about plugin development, see Chapter 7 of the book "Professional WordPress Design and Development". In truth, you do not even have to use PHP; you could use an intermediary language and interface with the WordPress table directly.

Generic XML Feed

Let’s take a look at a simple example for integrating a generic XML data source into your WordPress site. What you’ll do here is take an XML formatted feed, and consume it using PHP code. You’ll then convert it into a format suitable for a post on WordPress. In this case, you’re taking each node of the XML and making an individual post. Finally, you’ll make the WordPress post using the built in WordPress API. This is an automated way of taking the external source that generates the XML and make new posts, just as if you were hand retyping it in the WordPress Dashboard. This is an example that could be used for any XML source, but for this example we will use Twitter, which has a well-documented and simple-to-use API. Again, we are focusing on extracting the content and repurposing it for use in your WordPress site. This is not like the Twitter badges covered in the previous section which show the current Twitter updates. Those badges are more decoration than content and nothing persists on your site. This example turns tweets into posts. Getting the latest tweets for a single user in XML format is simple (assuming you have PHP5 with SimpleXML):

$twitterUser = "mirmillo";

$url = "http://twitter.com/statuses/user_timeline/$twitterUser.xml";

$xml = new SimpleXMLElement(file_get_contents($url));

header("Content-type: text/xml");

echo $xml->asXML();

Create a new PHP file with the above code (substitute your own Twitter account), and place it on a web server. If you use a web browser and view the served page, you will see the last 20 or so tweets from your account in a nice XML format. This gives you a chance to see what you’re working with. Remember, we are only using Twitter as an example; this could be any structured XML.

Rather than echoing this XML, you would in turn parse it and use the WordPress XML-RPC to create new posts. Unfortunately, the XML-RPC is phenomenally not documented. And, for some strange reason there is not a built-in WordPress API function for creating a new post, so you have to use the metaWebBlog API. Google will be your friend here.

Here is a simple example using the XML-RPC with the built-in WordPress client. First, you will retrieve the XML feed into a local variable on your site. Next you will include the built in XML-RPC client that comes with WordPress. This will allow you to create the new posts.

$twitterUser = "mirmillo";

$url = "http://twitter.com/statuses/user_timeline/$twitterUser.xml";

$xml = new SimpleXMLElement(file_get_contents($url));

 

include(‘../../../wp-includes/class-IXR.php’);

$client = new IXR_Client(‘http://localhost/wordpress/xmlrpc.php’);

Depending on your needs, you may want to sort your tweets by date or keep track of the last one posted. In order to keep this example simple, we will skip that. The next step is to loop over the XML nodes and pull the appropriate information that make up a WordPress post. The XML-RPC format for a new post is an associative array with title and description key/value pairs. There are other keys you can use to populate other WordPress fields should you need them. However, if you do not provide them, WordPress will fill in the blanks with defaults from your configuration.

foreach ($xml->status as $status)

{

    /* set up the post – there are many more keys you can include */

    $content[‘title’] = "Tweet from $status->created_at";

    $content[‘description’] = "<p>".$status->text."</p>";

Next, you will take this associative array and use the XML-RPC client to make a new WordPress post. Again, there is not a built-in way to do this in WordPress, so you use the metaWeblog.newPost method.

    /* post the tweet */

    $client->query(‘metaWeblog.newPost’, ”, ‘admin’, ‘password’, $content, true);

    if ($client->message->faultString)

    {

        echo "Failure – ".$client->message->faultString."<br />";

    } else {

        echo "Success – ".$status->text."<br />";

    }

Putting it all together, from retrieving the XML feed, instantiating the local XML-RPC client, formatting the data and creating the post, here is the file in its entirety:

$twitterUser = "mirmillo";

$url = "http://twitter.com/statuses/user_timeline/$twitterUser.xml";

$xml = new SimpleXMLElement(file_get_contents($url));

 

include(‘../../../wp-includes/class-IXR.php’);

$client = new IXR_Client(‘http://localhost/wordpress/xmlrpc.php’);  

 

foreach ($xml->status as $status)

{

    /* set up the post – there are many more keys you can include */

    $content[‘title’] = "Tweet from $status->created_at";

    $content[‘description’] = "<p>".$status->text."</p>";

    /* post the tweet */

    $client->query(‘metaWeblog.newPost’, ”, ‘admin’, ‘password’, $content, true);

    if ($client->message->faultString)

    {

        echo "Failure – ".$client->message->faultString."<br />";

    } else {

        echo "Success – ".$status->text."<br />";

    }

}

This file can be placed in the active theme folder of your WordPress installation. By placing it here, you can access it directly without messing with core files and your upgrade path. You would also want to set up some sort of cron job to make this trigger automatically depending on how often you want to aggregate. This method is a departure from using the WordPress Dashboard to generate content. In this case you are prompting the creation of the new posts without invoking WordPress directly. To make this work, you will need to access the URL of the PHP file you just created in a browser, which will kick the import process off. Once you have this manual process working, you can then set up a scheduled task via your operating system’s built in mechanism, such as cron, to launch the import on a regular basis.

What is really nice about the XML-RPC method is that there is no WordPress overhead involved. It accomplishes the goals outside of the WordPress framework, yet integrates the content appropriately. This method uses the application stack of PHP and the cron job, and only relies on WordPress for the XML-RPC client. It’s similar to posting from an external editor that invokes the XML RPC API to send its content to WordPress and have it turned into a post.

Alternatively, you could accomplish something similar with a full-fledged plugin and use the built-in WordPress API. For example, a traditional plugin could use wp_insert_post(), which is well documented in the codex and may provide you more flexibility. But in order to use the plugin method you will need to load all the database connectivity and API features. For more information on writing a plugin, check out Chapter 7 of the book "Professional WordPress Design and Development". We’ll explore a Twitter-specific plugin that performs this function in the next section.

Consuming XML to aggregate the content into your own posts is pretty simple. If you do not find a plugin that already exists to import your content, you can do it yourself with relative ease. You could create an XML-RPC consumer for each XML interface you want to aggregate into your WordPress site. If you designed it properly you could probably even reuse some of the code and keep it relatively lightweight. This flexibility allows you to consume nearly any type of XML feed, manipulate it and import the content.

Integrating Twitter

Even though we just used Twitter as an example of how to integrate any XML feed, let’s look at how you could integrate Twitter specifically. Twitter is currently the poster child for open web service APIs. The Twitter API is well documented and easy to use. In addition, it has tons of features. All of this makes integrating your Twitter activities with your WordPress installation a breeze. As such, there are several things you can do with Twitter integration. For example, you can show your latest tweets in a sidebar widget; this is an example of a simple social media badge. You could archive each tweet as its own WordPress post, or get fancier and have daily or weekly archives. You could grab specific tweets and use them in your header to create a dynamic first impression. Or finally, you could reverse the integration and automatically tweet every time you publish a new blog post.

The Twitter Tools plugin (http://wordpress.org/extend/plugins/twitter-tools/) by Alex King does almost all of this. And it is all contained in one plugin with a simple control panel so you can pick and choose how you want to use it.

For example, showing your latest tweets in the simple social networking sidebar widget is straightforward functionality with this plugin. First, install the plugin on your WordPress site and activate it. Using the new Twitter Tools dashboard, configure your Twitter username and password. Next, save your settings and jump over to the Widgets control panel. You will notice a new widget for Twitter Tools. This widget, by default, will show your last three tweets. Drag this widget to the appropriate sidebar for your theme to enable it. The benefit of this method is that it is trivially simple to implement; turn it on and it’s there. The downside is the transitive nature of this type of integration. You may generate some interest in people to follow you on Twitter, but it does not provide lasting content for your personal site. 

Twitter Tools can also convert each individual tweet into a post. Depending on how you use Twitter, this can be a nice way to back up, or archive your tweets on your own site. For example, if you use Twitter to create notes to yourself, having WordPress convert these to posts will give you the simplicity of Twitter, but then include the power of the structured content in WordPress.

An alternative approach would be to publish your tweets as asides intermingled around your regular blog posts. Think of asides in the same context as comments you make in the course of conversation that aren’t related to the main topic at hand. In this case, because they are originating from Twitter, they would be single-line posts, which make them ideal for asides. Simply take the tweets from Twitter and create new posts in a special category. Your theme would have to handle this special logic to create the right “asides” feel: shown in a sidebar, or highlighted, to differentiate them from the main narrative of your blog posts. Once you have the tweets making new posts, this could be as simple as using the Sandbox theme discussed in Chapter 8 (of the book "Professional WordPress Design and Development") and some creative CSS:

.category-twitter h2, .category-twitter .entry-date,

.category-twitter .entry-meta {

    display:none;

}

.category-twitter .entry-content p {

    background: #22739E url(‘images/twitter.png’) 5px 5px no-repeat;

    color: #fff;

    padding: 2px 5px 2px 25px;

}

If you’re publishing each tweet to its own individual post, Twitter Tools also has the capability to create daily or weekly digests. This is particularly interesting when you use Twitter to capture moments of your day through tweets. The hook with Twitter is how easy it is to use, and yet, because of the size restriction you are forced to keep your posts short—hence microblogging. In this case, you are telling the daily story of your life in a series of small takes. Allowing WordPress to import them creates another form of the same narrative. It is no different from writing an entire post entry and publishing it, except this way is self-assembled from the microblogging format used by Twitter.

One more feature of Twitter Tools is to reverse the integration; that is, tweet when you publish a new post. This feature has to be explicitly enabled in the Twitter Tools dashboard. Once turned on, Twitter Tools will automatically tweet on your behalf that a new post has been published and will include a link to your article. For some people this is a nice alternative to RSS syndication, which we will look at later in the chapter.

Alex King’s Twitter Tools plugin is very powerful and covers a broad spectrum of Twitter integration possibilities. However, sometimes you may need to do something unique, or something that a plugin was not designed for. Maybe you just have a different itch than the plugin author. Let’s take a look at how you could show your latest tweet as part of your WordPress header, without using a plugin.This would function similarly to a Facebook current status and would be like a “What I am doing right now” visual hook in the masthead of your web site. To accomplish this, you’ll create a special function that will retrieve the latest tweet from your Twitter timeline. We should note that Twitter does impose rate limits on the number of times the Twitter API can be accessed per hour, so depending on your traffic levels, you may want to consider augmenting this function with caching or rate limiting of your own to avoid triggering these limits.

The first step is to create the function that will retrieve your latest tweet. This functionality would be best placed in your functions.php file. You will notice some similarity to the previously discussed example for integrating a generic XML feed. We are using the same API, just fine tuning what is being retrieved, and abstracting it into a unique function. Here is the function:

function ddamstra_getLatestTweet($twitterUser = "mirmillo") {

    $url = "http://twitter.com/statuses/user_timeline/$twitterUser.xml?count=1";

    $xml = new SimpleXMLElement(file_get_contents($url));

    $status = $xml->status->text;

    return $status;

}

This function gets the text of the latest tweet from Twitter. It only retrieves one tweet, as seen in the query string of the URL. This status text is then returned to the function caller Next, you will need to access this information by calling the function. In your header.php template file, you can add some code to the latest tweet:

<div id="header">

    <h1 id="blog-title"><?php bloginfo(‘name’) ?></h1>

    <div id="blog-description"><?php bloginfo(‘description’) ?></div>

    <div id="twitter-current"><?php echo ddamstra_getLatestTweet(); ?></div>

</div><!–  #header –>

Supplement this sample with a simple link to your Twitter account and you have added a nice touch of personality to your site. Assuming, of course, that your audiences overlap and you are a responsible Twitter user.

The Twitter API is very robust and open. You can do pretty much whatever you want with it; you just need to read the documentation and this is what makes playing with the Twitter API so much fun. The result is tons of plugins to integrate with Twitter including posting to Twitter from the WordPress Control Panel, adding a “tweet this” to your blog posts to encourage visitors to spread your word. There is even a plugin to add Twitter information to comment posts and one to use Twitter pictures in lieu of gravatars.

Integrating RSS and ATOM Feeds

For us, RSS is becoming the glue that ties many sites together. Despite its shaky start and competing standards, RSS is a solid means to communicate certain information between different sites.

One reason you might want to import RSS feeds from another site into your own is to create your own RSS aggregator site like a “Planet” (a la http://planet.wordpress.org/) or like Google Reader. RSS is an easy way to keep tabs on multiple sites in an easy-to-digest location. Make sure you have permission to republish the content; consult a lawyer if you have questions. Also, make sure you are courteous and cache the feeds. Your web site should not adversely affect the performance of another.

If the originating content is your content, or content you have permission to repurpose, building a feed aggregator is fine. Typically, companies do this by deciding to build a “planet” of all of their employee blogs, no matter where or how they are hosted. You might also do this if you have a few quasi-independent people who all want to share content to build the sense of community akin to early points about getting noticed. Finally, you could do this if you have more than one blog and periodically want to post a summary of “what’s going on over in XYZ land.” For example if you run a food site and a sports site. By the very nature of their core subject matters they do not cross-reference each other, but each could summarize the other from time to time to drive cross-readership.

The easiest way to consume an RSS or ATOM feeds from another site for republishing in your site is to use a plugin. The FeedWordPress plugin (http://wordpress.org/extend/plugins/feedwordpress/) by Charles Johnson is a simple and powerful plugin for consuming these feeds.

After installing and activating this plugin you will have a new set of administration dashboards to configure the syndicated sites. After entering a valid feed URL, you have options to configure scheduling of when the feed is re-visited, as well as, authors, posts and tag information. This plugin really simplifies the means to republish content from one site to another via RSS or ATOM feeds.

Finally, the SimplePie Core WordPress plugin is a PHP library that gives you a bit more control over feed processing and filtering, but without the simpler-to-use interface. If you want to extract content from an RSS feed and perform some cleanup, for example removing images or resorting items before publishing, then you can use SimplePie as a container for your PHP content processing scripts.

This article is excerpted from chapter 9 "Content Aggregation " of the book "Professional WordPress Design and Development" by Hal Stern, David Damstra, Brad Williams (ISBN: 978-0-470-56054-9, Wrox, 2010, Copyright Wiley Publishing Inc.)

Tags:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *