Making a Raspberry Pi Twitter Bot Part 3 – Auto-posting

Making a Raspberry Pi Twitter Bot Part 3 – Auto-posting

Raspberry Pi Twitter Bot

raspberrypi25_main
Here’s the last part of our series on making a Raspberry Pi Twitter bot.
Above is the picture of our completed device. When it gets new feeds, the LEGO gentleman on the sofa blinks his LED to notify you.

Today, we will set up the environment to auto post smoothly. First, let’s research the post limit for Twitter! Let’s try to avoid errors regarding things we can find out beforehand. We made it so that things get recorded in a log file so that when an error occurs during a post, we can check the details. Now that we are ready, let’s set up the auto post. We recently started using “cron”. Aside from being able to set the day and time, it also allows us to auto-execute when Raspberry Pi launches, so we gave it a shot!

Limits for Twitter

We touched upon the limits in the last article and the one before that, but let’s take a closer look here (This information is as of July 2015).

Tweet posts
Definition: A Tweet may contain photos, videos, links and up to 140 characters of text.

We’re familiar with this limit. When trying to post over 140 strings through the API, we get an error saying, “Status is over 140 characters.” We can solve this with the method we covered in the last post, by considering the shortened URL to make it 140, remember?

I’m seeing an error message after posting a Tweet

  • If you see a “Whoops, you already said that” error message when you post a Tweet, you’ve posted the exact same text in another Tweet recently, and should try writing something new in your next Tweet.
  • You should only see this error if you post exactly the same text as a recent tweet.

As we discussed in “Making a Raspberry Pi Twitter Bot Part 1 – Twitter API“, Twitter has a function to prevent double posts. If 1 character is different, we were able to post. But when the text is exactly the same, we had to wait a few hours to post it again. When we use API, we get an error message saying, “Status is a duplicate.”

Twitter limit (API, Tweet posts, follow)

  • Tweets: 2,400 per day. The daily update limit is further broken down into smaller limits for semi-hourly intervals. Retweets are counted as Tweets.
  • The Tweet limit of 2,400 updates per day is further broken down into semi-hourly intervals. If you hit your account update/Tweet limit, please try again in a few hours after the limit period has elapsed.

2,400 tweets in a day! That’s more than I thought! It seems we do not have a problem. But if you want to use something that gets updated frequently like a news website RSS feed, you might want to be careful.
It seems there are no details about the “30 minute intervals” as of now (July 2015).

List of error codes

For more details on errors and error codes, refer to the official website.

In this case, as long as we avoid character limit and double posts, we probably won’t have a problem. But just in case they change the spec in the future, let’s make it so that the log files keep a record of your posts and errors.

Outputting Log Files

We created two log files, one to record the time of execution of the last post and another to record the error logs. Below is the data that will be saved.

Execution log file
(log.csv)
RSS URL
Time executed of last session
Error log file
(erroe.csv)
RSS URL
Time executed
Post
Error message

Let’s manage this data in CSV format. We previously went over how to input/output CSV files.
All we had to do was customize the source of this article: “DIY Raspberry Pi Thermometer

/var/www/news/tweet_rss.php

Execute command
php/var/www/news/tweet_rss.php

Please replace the “*****” part with the appropriate TwitterAPI key (Consumer key, Consumer secret, Access Token, Access Token Secret).

We added a simple check feature that prevents double posts based on an updated day and time.
Whether you posted a Tweet or not, the date and time will be recorded in the execution log file (log.csv) after the process.
In the next session, it will only post information that was updated after the recorded date (the day before the first time).

When an error occurs, it will be recorded in the error log file (error.csv). Just in case, we made it so that the error content will be saved.

The multiple RSS check has also been improved. The URL and the strings attached to the Tweet text will be managed in array.

Once executed, check your Twitter! If your post has been posted, it’s a success!

raspberrypi25_img01

Figure 1

The LED blinking time is used as sleep, so it will be posted every 3-5 seconds. Check the log file accordingly.

raspberrypi25_img02

Figure 2

This is the screen of the execution log. Are the URL and date/time recorded properly?

raspberrypi25_img03

Figure 3

This is the error log. And here is an actual error we were able to find because of this log file (the program above has been fixed).
On the 4th row, it reads “Status is over 140 characters”. The character length is supposed to be adjusted, so why did this happen? We looked at the string that we attempted to post and pasted it into the Twitter post box, and it appears that it’s over the 140 characters limit.

raspberrypi25_img04

Figure 4

At a closer look, “Raspi.TV” on the 3rd row is shown in blue, so it is recognized as an URL. When we post contents with two or more URLs, the 1st one gets treated as an URL and the others get counted as characters. The text was supposed to be 22 characters with the shortened URL, but instead it got counted as a normal character and resulted an the error.
Aside from this, there were many errors we noticed because of the error log file. We added the error log file because we thought it would come in handy when the API spec changes, but it already helped us in the testing stage!

Auto-Post Setting with Cron

Cron is our tool for Auto-execute! We already used Cron settings several times in our series. Let’s say you want to execute the command every hour. You would do the following.

crontab -e

First, bring up the cron settings screen from the LX Terminal.

00 * * * * php /var/www/news/tweet_rss.php

Then, write it like this!
Hit [Ctrl]+[O] to save, or [Ctrl]+[X] to quit settings and return to the original screen.

We used the Yahoo! “Tech” category RSS earlier, but it couldn’t keep up because it gets updated every few minutes. As a result, it disappeared from the RSS before we could Tweet it. For websites that get updated frequently like news, we may want to shorten the interval. Let’s check the overall interval before setting up the cron.
If the content gets updated only a few times a day, it worked fine with an hour interval auto-execute.

Auto Post When Raspberry Pi Launches!

We’re done setting up the auto post! Or maybe not. Before we move on, it seems there is a way to trigger auto-execute when “Raspberry Pi launches” with cron, not just time. So we gave it a shot.
Let’s get right to the point. When dealing with “time” like this, it won’t work properly. We took a closer look to find out why. Below is the overview!

First, the program we used for the test is the following:

/var/www/news/tweet_start.php

Execute command
php /var/www/news/tweet_start.php

We made it so that it Tweets the time as well to avoid double posts. Also, we left room for 5 tries since it executes right after Raspberry Pi launches and there may not be any Internet connection at first. In our test, it usually posted on the 1st – 2nd try.
Now, let’s execute this program when Raspberry Pi launches.
Information on how to set up cron upon launch was found in the link on Wikipedia in the English manual page.

Crontab : Scheduling Tasks – math-linux.com
It is summarized at the bottom of this page. It seems with cron, you can either set up the date and time, or specify the strings as shown below.
There are also special strings of characters:

String Action
@reboot execution at boot
@yearly execution once a year, “0 0 1 1 *”
@annually execution once a year, “0 0 1 1 *”
@monthly execution onnce a month, “0 0 1 * *”
@weekly execution once a week, “0 0 * * 0”
@daily execution once a day, “0 0 * * *”
@midnight execution once a day, “0 0 * * *”
@hourly execution once an hour, “0 * * * *”

“@reboot” in the 1st row of the chart says, “execution at boost”. There are other strings to execute “once” per unit. Aside from “@reboot”, you can use numbers to express it, so you probably won’t use it too much. Ok then, let’s test it out!
Similar to before, enter “crontab-e” from the command line to bring up the cron settings screen. Then, register the command using “@reboot”.

@reboot php /var/www/news/tweet_start.php

Save the settings and reboot Raspberry Pi! Check your Twitter from a different computer when you do this.

raspberrypi25_05

Figure 5

If your post looks like this, you did it!
I wish I could say that, but it actually doesn’t work.

raspberrypi25_06

Figure 6

Here’s why. If you look closely, you will notice that the timestamp of the Tweet text is way off from the actual date and time. This is clearer when you shut down and reboot after a while. What happened was, the time when the machine shut down got posted. In my case, I booted Raspberry Pi on Monday after the holidays and noticed this. It turns out this happens because Raspberry Pi doesn’t have an internal battery. When the power goes out, the internal clock of Raspberry Pi goes too, so after rebooting, it moves based on the time when the system shut down until the clock gets reset.
I always had my LAN cable plugged in, so I didn’t notice at all. But when I shut off the network and booted it, the clock did not get reset and the system kept using the old time. From this, it seems that the “@reboot” of cron cannot keep up with the reset time. Also, in the RSS auto post tool, the posts are made based on the current time, so this will not work for us. To solve this, we can reboot the ntp, or use the ntpdate package. We gave it a try, but sometimes it was a few minutes off and sometimes it couldn’t execute in time for the “Raspberry Pi launch”. It was quite difficult to get it right.
Regardless, auto-execute upon launch is not mandatory for this tool, so I force my Raspberry Pi to “Sleep for 60 seconds, then start the process” in order to make everything work. It works without a problem with Wi-Fi, so I was able to mount the functions I wanted successfully!
By the way, if you use something called “RTC (real-time clock module”, you can prevent the clock inside Raspberry Pi from stopping.

Real-time clock
A real-time clock (RTC) is a computer clock (most often in the form of an integrated circuit) that keeps track of the current time.
Although the term often refers to the devices in personal computers, servers and embedded systems, RTCs are present in almost any electronic device which needs to keep accurate time.

Summary

We completed our Raspberry Pi Twitter Bot!
As far as auto-execute goes, there is room for improvements. But it works fine, so let’s settle for now! After all, trial and error is what makes electronic kit so much fun!

Next, we will work with a smartphone again! It’s been a while.
Before, we used the “MPD” app to listen to music. Next, we will use the “VNC” app so that you can touch the Raspberry Pi desktop screen directly!
Once you can do this, you might be able to forget about dealing with display, keyboard and mouse! They can get pretty annoying at times, right?