Making a Raspberry Pi Twitter Bot Part 2 – Reading RSS feeds

Making a Raspberry Pi Twitter Bot Part 2 – Reading RSS feeds

Raspberry Pi Twitter Bot

raspberrypi24_main
Let’s continue working on our Raspberry Pi Twitter bot!
In the part 1, we figured out how to post tweets through the Twitter API. Next, we will need the actual content for the tweets! Ideally, we should tweet info as up to date as possible. So, we will be posting updates from an RSS feed.

What is RSS?

raspberrypi24_img01

Figure 1 From Wikipedia

When we think of RSS, the first thing that comes to mind is this icon!
You probably already know that it is some kind of URL (?) that provides update information, so here is a quick explanation!

RSS – Wikipedia
RSS (Rich Site Summary; originally RDF Site Summary; often called Really Simple Syndication) uses a family of standard web feed formats to publish frequently updated information: blog entries, news headlines, audio, video.

Currently, both RSS 1.0 and RSS 2.0 are used. Each uses a different format, so be careful when writing your program.The differences between each can be read about on the Wikipedia pages linked below.

RSS 1.0 specifications
RSS 2.0 specifications

Speaking of which, DevicePlus’ RSS feed can be found on the right sidebar.

raspberrypi24_img02

Figure 2

Over here! (Referring to the layout on July 2015)

An RSS feed is XSS-based, so you can view it from within a browser. Some browsers will display it differently from others, so you may have to look directly at the page source. In Chrome, the XML will display as is.

raspberrypi24_img03

Figure 3

The version of RSS used in a feed can be confirmed in the first tag. The end of the third line reads “version=“2.0””, meaning that this RSS feed uses RSS 2.0.
“Title”, “link”, and “description” seen under the “channel” tag are the only mandatory tags. Each article’s information can be seen in the “item” tag within it. The same tags are used here, so be careful not to confuse them.

Reading RSS feeds through PHP

An RSS feed comes in the XML file format, meaning that it can be read through a single PHP function. Below is a sample program to output the four fields of “title”, “link”, “description”, and “pubDate2.”

/var/www/news/rss.php

raspberrypi24_img04

Figure 4

Figure 4 shows data read from DevicePlus’ RSS feed

Along with the title and URL, we can add a “description” with a character limit to create a more detailed tweet.
“pubDate” shows the set publishing time. We might be able to use it as a measure for our tweet timing (but let’s leave that for next time!). The time is listed in RFC822 format, but if you like, you can change it to something else to suit your preference.

raspberrypi24_img05

Figure 5

Figure 5 is like Figure 4, but displays the RSS feed from the official Raspberry Pi blog. Here, we can see that “description” may sometimes contain tags. According to RSS 2.0 specifications, either “title” or “description” must be set for the item to work. “Link” is not absolutely necessary, meaning that the URL can either be:

  • Added in “link”
  • Added in the “description” HTML

Changing this setting would involve altering the code, so we are going to skip ahead and assume that the URL is going to be set under “link.”

About Twitter’s character limit

If you have used Twitter before, then you probably know that a single tweet can only contain 140 characters . Setting our program to output tweet in under 140 characters is simple enough, but do URLs count?

About Twitter’s link service (http://t.co)
Links shared on Twitter, including links shared in Direct Messages, will automatically be processed and shortened to an http://t.co link.

How to post a link in a Tweet on the web
A URL of any length will be altered to 23 characters, even if the link itself is less than 23 characters long. Your character count will reflect this.

Just to be sure, let’s confirm this on Twitter itself.

raspberrypi24_img06

Figure 6

Figure 6 shows the tweet posting box with a URL inputted. The URL appears as is, but taking a look at the character count in the bottom right corner reveals 23 characters!

140 – 117 = 23

 Regardless of what is shown, the link is automatically shortened.

raspberrypi24_img07

Figure 7

While not listed on the help page, note that URLs that begin with “https” will become 23 characters.

raspberrypi24_img08

Figure 8

Figure 8 shows a tweet made by our automatic tweeting program from last update. The text shows the entire URL, but mousing over it shows the shortened version.

Summing things up, our 23 character shortened URL + 1 space gives us a total of 24 characters! This means that we have a total of 116 characters free to use for the tweet!

Tweeting from Raspberry Pi!

Now that we know how the Twitter’s character limit system works, we can hook up the tweeting program that we made last time!

We added a function that makes an LED light up if the tweet was successful.

raspberrypi24_img09

The GPIO port uses the pin 10 “GPI015”. Refer to this article for further instructions on LED lights.

/var/www/news/tweet_rss.php

Execute command
php /var/www/news/tweet_rss.php

Like before, do not forget to replace the “*****” segments with your Twitter API keys (Consumer key, Consumer secret, Access Token, Access Token Secret).
Each field will be printed out on a new line. Note that a line break also counts as 1 character.

We will use up to 116 of the 140 characters available (subtracting 23 for the URL and 1 for the space/line break), and add an ellipsis (…) to the end if we run out of space.
The sample seen above posts only one update from the RSS loaded. While it is possible to post every update, you may end up posting dozens of tweets at once depending on the feed’s content. Confirm the number of updates your feed has through the “item” tag beforehand (DevicePlus has only around 10, so posting them all did not result in an error).
Other tweeting limits will be covered in the next article!
The LED will flash three times if the tweet went through successfully. This flashing design choice was made so that we could place the device on our office desk and not bother other people by having it making noise.

raspberrypi24_img10

Here is what our tweet looks like.
We used the full 140 character limit, but it just looks like a jumble of text with an ellipsis at the end. When using our RSS feed, it may be better to aim for a simple output that stays short.
In the end, projects like this need your personal touch! Adjust the tweet content depending on the RSS feed you use!
If your program has trouble reading the RSS feed, check if the feed’s XML file has not been corrupted. Of course, there is also the common problem of the program working on your computer, but not on the Raspberry Pi. In that case, you might be able to solve things by simply updating the apache2 package (this actually solved our problem!).

Summary

RSS feeds turned out to be a lot more complicated than we thought. There was a lot of research, such as the completely different versions of RSS 1.0 and 2.0, and the very few mandatory tags.
Simply reading from an RSS feed and posting it on Twitter failed make us feel like engineers, so we added a good old fashioned LED. The flashing light is rather noticeable when sitting at our desks! You can set it up to not only flash when a tweet has been made, but also when the RSS feed gets an update!
Now that we have laid down the framework, all that remains is to make it automatic!
Our current setup will tweet everything at once, so we will need to set up something up to monitor what has been tweeted before and what has not. Having an error log for an automatic program like this will also prove handy.
In the final article in our series, we’ll see how to set up the environment!