Don’t you wish you could do that neat “who are you following that’s not following you back” Twitter trick in Perl? Today I decided to learn a little more about the Twitter API by rolling a solution out on my own.

The idea was recently demonstrated in Ruby and was neatly done in 15 lines or less(you might have noticed something familiar about the post title?). It used a Ruby gem which contained the Twitter functions, so we had to roll out some Perls of our own(pun intended).

We pretty much get the same result in under 15 lines, except Perl 5 doesn’t offer the syntactic sugar of subtracting sets of objects from 2 arrays in just one statement. In Perl, the statement @friends – @followers will be evaluated in scalar context, which results in the number of items in @followers being subtracted from the number of items existent in @friends, not in the subtraction of the objects contained in @followers from @friends.

Another difference was in the choice of the Twitter web services function. The Ruby version fetched a list of numeric Twitter ID’s and then dispatched one remote service call per ID to fetch the rest of the user’s data. This method limits the script to 100 users queried per hour, as was explained on the post about the Ruby version.

I chose to fetch data for 100 users at a time using a heavier web service. This way you can verify sets of up to 10,000 users (friends + followers) per hour, which covers a pretty broad range of accounts(you know, I’ve got a whopping 30 followers).

We don’t want to waste all this data after it’s returned by the web service. So we can use an arrayref of Zen::Twitter::User objects to maintain local state for each User returned.

The examples, as well as the Perl modules, are available in the Zen::Twitter tarball.

Our first example checks which friends do not follow us back on Twitter:

use lib './lib';
use Zen::Twitter;
use Zen::Twitter::Authentication;

my $cred_args = {
    filename => $ENV{HOME} . '/zen_twitter_tools/credentials.xml'
};
my $credentials = Zen::Twitter::Authentication::credentials($cred_args);

my $args = {
    credentials => $credentials
};

my $friends_not_followers = Zen::Twitter::friends_not_followers($args);

foreach my $user ( @{$friends_not_followers} ) {
 print "http://twitter.com/" . $user->screen_name() . "\n";
}

We use this to answer the opposite question – Which followers have we not befriended yet?

use lib './lib';
use Zen::Twitter;
use Zen::Twitter::Authentication;

my $cred_args = {
    filename => $ENV{HOME} . '/zen_twitter_tools/credentials.xml'
};
my $credentials = Zen::Twitter::Authentication::credentials($cred_args);

my $args = {
    credentials => $credentials
};

my $followers_not_friends = Zen::Twitter::followers_not_friends($args);

foreach my $user ( @{$followers_not_friends} ) {
 print "http://twitter.com/" . $user->screen_name() . "\n";
}

Being the Unix fans that we are we print as little output as possible. In our case, we print the user’s Twitter profile url, which is just “http://twitter.com/” . $user->screen_name() . If you look closely, that $user (instance of Zen::Twitter::User) object being iterated on the foreach loop is loaded with juicy details about the user(see the Zen::Twitter documentation). We only used one instance method to obtain the user’s screen name but as you can see it’s possible to print a boatload of data if you need to.

2009-05-18 – Version 0.10

This is release 0.10 of Zen Twitter Tools and the Zen::Twitter library, which currently covers the following:

• Search
• Timeline
• Status
• User
• Direct Message
• Friendship
• Social Graph
• Account
• Favorite
• Notification
• Block
• Help

You can read more about the above function groups at the Twitter API documentation site.

The following scripts are currently included on Zen Twitter Tools v 0.10:

Account Management
account_rate_limit_status.pl – Check your current API limits and how many credits you have left.
account_update_delivery_device.pl – Update delivery device for Twitter notifications.
account_update_profile_color.pl – Update your profile colors.
account_update_profile.pl – Update your profile data.
account_verify_credentials.pl – Test the data on credentials.xml by logging in from the command line.

Block Management
block_blocking.pl – List users you are currently blocking.
block_create.pl – Block a user
block_destroy.pl – Unblock a user
block_exists.pl – Check whether you are currently blocking a user.
block_ids.pl – List IDs of all users you are blocking.

Direct Messages
direct_message_destroy.pl – Delete a Direct Message
direct_message_send.pl – Send a Direct Message
direct_messages.pl – List Direct Messages inbox
direct_messages_sent.pl – List Sent Direct Messages

Favorites
favorite_create.pl – Add a status to your Favorites
favorite_destroy.pl – Remove a status from your Favorites
favorites.pl – List your Favorites

Followers and Friends
followers_not_friends.pl – Lists which followers you have not yet befriended.
friendship_create.pl – Follow a user
friendship_destroy.pl – Unfollow a user
friendship_exists.pl – Check whether user_a follows user_b
friends_not_followers.pl – Lists which friends do not yet follow you back.
friends_timeline.pl – Your friends’ 20 latest updates
mentions.pl – The 20 latest mentions of @you
public_timeline.pl – The Twitter-wide latest 20 updates, updated every 60 seconds.

Notification (Mobile)
notification_follow.pl – Receive mobile notifications for a user.
notification_leave.pl – Stop receiving mobile notifications for a user.

Search
search.pl – CLI interfact to search.twitter.com. Searches Tweets for keywords. Example: search.pl ‘twitter tools’

Social Graph
socialgraph.pl – Simple friend/follower count.

Status
status_destroy.pl – Delete one of your status messages.
status_show.pl – Show the status of a user.
status_update.pl – Update your status.

Help
test.pl – Tests a simple exchange with the Twitter server to check connectivity.

Users
user_timeline.pl – Retrieve a USER’s timeline(Same data you see on http://twitter.com/USER’s page)
user.pl – Reports the current status of the user identified by screen_name

Sample Scripts
Sample scripts are provided with each module so you can implement/customize your own tools. The scripts can also be used to directly manipulate your account. All functions requiring authentication need a properly configured credentials.xml file. Setup is very simple, you should not have any problems using the scripts and library.

Not Implemented on v 0.10
The following functions were not implemented on 0.10 :
- Search trends (Reason: did not implement the JSON-only requests yet.)
- Profile image uploads(2 functions: background image and user image).
- Account signoff function. (Not applicable on a stateless CLI environment.)

Download: zen-twitter-tools-0.10.tar.gz

Note: I’ll be working on improving the documentation for the library. For now I’ve quickly annotated each function but did not provide standard CPAN-quality POD’s for all modules.

2009-05-14

I found some Twitter modules on CPAN but the Twitter API seemed fun, it’s well documented and getting more popular by the day, so I felt like having a go at it in Perl.

Bug reports, suggestions and constructive criticism are always welcome.

So far we have some base classes, authentication, HTTP and XML modules in place. The only demos available at this time are the Friend/Follower monitoring scripts.

The Zen::Twitter::User Object
Using Class::Accessor we automate the creation of accessor methods for each field returned in the Twitter user data structure. See below for the vast ammount of user information you can get from a single Zen::Twitter::User object.

Zen::Twitter
The Zen::Twitter module is the general front-end employing direct Twitter functions to create added functionality. For instance, friends_not_followers() and it’s counterpart followers_not_friends() are implemented in it, using 2 direct Twitter functions from Zen::Twitter::Friends to obtain the respective lists and then diff them from each other.

Zen::UserAgent
This is currently a single-function stub really, all it does is create a single point from where our UserAgents are created. We might need to add functionality, so right now it might seem like overkill but it just might come in handy to have all our HTTP User Agent stuff tucked in one namespace.

Zen::Twitter::Authentication
Hides the details of finding and parsing credentials.xml and gives us a Credentials object to pass around wherever authentication is required.

Zen::Twitter::Credentials
A simple Credentials interface. Currently offers username() and password() methods only. Created it so we wouldn’t be passing a hashref around accessing its internals without a proper interface.

Zen::Twitter::Friends
Maybe this one should have been called Zen::Twitter::Users, keeping the names sync’d with Twitter’s. It contains 2 direct Twitter querying functions: friends() and followers(), both return an arrayref of Zen::Twitter::User objects for the user identified by the Credentials passed in.

Zen::Twitter::HTTP
Queries the Twitter XML services and returns Perl data structures parsed by XML::Simple. A Zen::Twitter::User can be directly instantiated from each user tag on the XML returned by the functions described here.

Download
zen-twitter-tools-0.01.tar.gz

That’s all folks! I hope you can use some of this code to create your own Twitter apps(I placed it all under the BSD License, just in case).

2009-05-17 – Version 0.08

- Implemented Block methods, with sample scripts.
Twitter API Sections now Available in Zen::Twitter :

• Search(without ‘trends’)
• Timeline
• Status
• User
• Direct Message
• Friendship
• Social Graph
• Favorite
• Notification
• Block
• Help

Next up in TO DO list: Account Methods

Download: zen-twitter-tools-0.08.tar.gz

Version 0.07

- Implemented Friendship methods, with samples.
- Implemented Social Graph methods. Included sample scripts.
- Implemented Favorite methods, with sample scripts.
- Implemented Notification methods, with sample scripts.

Twitter API Sections now Available in Zen::Twitter :

• Search(without ‘trends’)
• Timeline
• Status
• User
• Direct Message
• Friendship
• Social Graph
• Favorite
• Notification
• Help

Next up in TO DO list: Block Methods

Download: zen-twitter-tools-0.07.tar.gz

Version 0.04

- Implements the 4 Direct Message methods: direct_messages(), sent(), new() and destroy() which allow you to create and send direct messages as well as view the lists of sent and received messages.

Twitter API Sections now Available in Zen::Twitter : Search(without ‘trends’), Timeline, Status, User and Direct Message Methods.
Next up in TO DO list: Friendship Methods

Download: zen-twitter-tools-0.04.tar.gz

Version 0.03

- Implements all 3 Status methods: show(), update() and destroy() which allow you to view any status messages(authentication needed for protected updates), update your own status and destroy your own unwanted updates. Includes 3 example scripts implementing each.

- Implements the single test() Help method. Sample script allows you to “ping” the Twitter servers and test connectivity.

Twitter API Sections now Available in Zen::Twitter : Search(without ‘trends’), Timeline, Status and User Methods.
Next up in TO DO list: Direct Messaging.

Download: zen-twitter-tools-0.03.tar.gz

Version 0.02

- Implements all 4 Timeline Methods : Display public timeline, friends’ timeline, a specific user’s timeline and mentions of you.
- Zen:Twitter::User object is employed on all Timeline results where a user is mentioned
- 4 scripts added to demonstrate the Timeline methods

- Implements Search (No ‘trends’ functionality yet.)
- search.pl script added to demonstrate search functionality

- Filtering/sorting HTTP parameters unimplemented by the library are passed on to Twitter.

Download: zen-twitter-tools-0.02.tar.gz

As I opened Twitter today I noted a same “friend”(someone I “follow” on Twitter) had taken over my timeline. It wasn’t a person, it’s a company I follow.

Today one of their professional sunday Twitterers decided to auto-post links to about 100 product checkout pages in hopes someone would be gladly Twittering around on a sunday, stumble past one of their $7500 product links and complete a purchase.

Think about it: what are the odds of this kind of Twitter spamming converting into the sale of a MacBook or an expensive trip somewhere?

If this company had a million followers(it doesn’t), perhaps a few dozen would even click through to a 140 character message in all caps, like “MACBOOK SPECIAL: U$ XXXX.XX COUPON YYY-ZZ CLKC DISCNT TWITTER SPECIAL TODAY ONLY”

I’d be willing to bet on the contrary, they’re losing sales by doing this, and they’re lowering the value of their brand at the same time. I bet the next time I see their logo on Twitter I’ll probably remember this episode(and I’m not the kind that holds grudges), so will anyone else who happened to be in the line of their Twitter machine-gun fire this session.

How many followers did this company give up spamming hundreds of Twitter timelines? I don’t know, but I can tell you they lost at least one.

Auto-micro-blogging is just as lame as “old school” auto-blogging
Another inconvenience on Twitter is that they’re currently not fighting auto-bloggers. So, after I unfollowed the above company, I noticed a certain “friend” had, in turn, taken over my timeline. But it wasn’t there just a minute ago! At some point this person decided it’d be a great idea to scrape Google News and post every single article until they drained their 100-api-calls-per-hour Twitter API limit. I can assure you that if someone ever wanted to read the latest 100 Google News articles, they would have spelled it news.google.com.

Changes on v. 0.12

Namespace move to Twitter::

The Zen::Twitter library has been moved to the Twitter:: namespace. To avoid creating yet another root namespace(Zen::) we’ll use an existing root NS from CPAN, namely Twitter::

The base Zen::Twitter module now becomes Twitter::ZenTwitter. This module implements the friends_not_followers and its counterpart followers_not_friends functions. The Zen Twitter Tools scripts have all been modified to work with the new namespaces and should work just fine.

Tools and Library are now separate releases

Zen Twitter Tools has been broken in two: the library has been packaged into CPAN-standard form and the scripts are now zen-twitter-tools 0.12

This has the advantage of allowing the module to be listed in CPAN in the future, decoupling the library from the scripts(previously coupled by the include directory path being set to “./lib/”).

Installation

Download the libraries and tools below.
- Unpack Twitter-0.12.tar.gz

For a systemwide install this needs to be run as root:
- cd Twitter
- perl Makefile.PL
- make
- make install

Zen Twitter Tools:
- Unpack zen-twitter-tools-0.12.tar.gz
- cd zen-twitter-tools
- Edit credentials.xml and add your secret Twitter user/pass
- Now you can use Twitter from your command line!

Download

Library: Twitter-0.12.tar.gz
Tools: zen-twitter-tools-0.12.tar.gz

After a short email dialogue with Marc Mims of Net::Twitter, I’ve decided to remove Twitter::ZenTwitter from CPAN. He never requested that I do it, though I could understand from our dialogue that leaving my modules there would confuse things when he clearly looked forward to make Net::Twitter the default Perl interface to that service. This is entirely my opinion, none of this was even brought up in the emails. This short post is a quick explanation as to why the modules vanished from CPAN and from my directory there.

Zen::Twitter started as a script to find out which friends did not follow me back on Twitter. Not that I care, I have about 30 followers today and I know them all personally, so I never needed to know who did or didn’t follow me back…but it was meant as a followup to someone who posted a Ruby script to do the same some time ago.

Then I added functionality to find out which followers you had not befriended yet(the opposite direction). From there, to implementing nearly the full Twitter API it was a short hop – mainly because I had already implemented the User class, the HTTP request class and so on.

In the meantime, I had not even had a look at Net::Twitter. So…. After I released the Twitter::* modules and sent them to CPAN, Marc contacted me and asked if there was anything wrong with Net::Twitter. I said, of course, no, there wasn’t.

So then he asked why I hadn’t used Net::Twitter and chose instead to release my own modules. I felt bad that I had done all of this without giving it much thought or researching things further.

If nobody had said anything I’d probably have carried on without even knowing the other modules were there. I suppose CPAN is meant to accommodate everyone’s ideas so really we could have both coexist, but for something as trivial as Twitter I think I was just taking space.

Added that Doug Williams of Twitter didn’t think my libraries should be listed on http://apiwiki.twitter.com/Libraries I can just conclude this whole Twitter programming thing wasn’t meant for me. Let’s move on. For anyone interested in hacking around with Twitter and Perl, I’m leaving these modules as they are frozen in time.