I love Spring – create your own RSS feed with java

This blog is about a small private project, the topic fits to the company blog. The key message of this blog: I love Spring.

And I love to read news in my Google Reader. I want to put as much informations into this channel. I like it, because it is asynchronous and I can pull informations, if I have some free time. Unfortunately not every information is provided via a RSS or Atom-Feed. So I was looking into a lot of converter-solutions which fits my needs. Some examples: FreeMyFeed, mmmmail.com or a solution for a twitter-to-rss-conversion. But nothing was really suitable. Then I thought: HEY, you are a programmer ! Don’t think about it, do it!!

I have tons of ideas, what I want to read in my Google Reader, endless possibilities. Just two examples:

• I am a geocacher and I want to get notified if a new special geocache is published in my neighborhood. On geocaching.com you can set up e-mail-notifications for new caches. But you cannot choose, that you only want to get notifications for T5 (climbing)-caches. And I don’t want mails, I want to read it in my Google Reader.
• I want to read my twitter timeline in my google reader. Twitter allows you to subscribe to feeds from a single user. But some users protect their tweets and you have to be authenticated to read them. That’s not possible with Google Reader. And I want to read all Tweets in a single Feed, not one for each person I follow. And I want to filter some feeds (“I just became the mayor of…”) and I want to see linked pictures directly in the feed, without clicking them. And and and… A bunch of very special requirements.

So I started a small project. Here is my protocol of one week with some nice Spring-evenings 🙂

• Evening 1 – Providing some example data as feed
  Set up project with Maven, initial configuration of Spring MVC including a Controller and a view based on AbstractAtomFeedView – A little bit request mapping, creating some mock data in my controller and – YES! – after starting my tomcat, chrome asks me to subscribe to my first own feed with google reader. Sure – without any sense, it’s localhost 🙂

  1@Override
  2protected void buildFeedMetadata(Map<String, Object> model, Feed feed, HttpServletRequest request) {
  3…
  4@Override
  5protected List<Entry> buildFeedEntries(Map<String, Object> model, HttpServletRequest request, HttpServletResponse response) throws Exception {

• Evening 2 – Remove mock data and display E-Mails from my google inbox
  Connecting to my google-inbox with Spring Integration ‘s ImapMailReceiver was very simple and very effective. I just pull out mails from my “toRSS”-Label which I configured in my Google Inbox. Together with some filter rules to mark for example mails from geocaching.com with my “toRSS”-Label, this is everything I need.

  1Message[] messages = mailReceiver.receive();

• Evening 3 – Get displayable text from javax.mail.Message and Filter mails
  Getting displayable text from javax.mail.Message is not very nice (MultiPart, BodyPart, whatever…). Do you know a good tool or framwork for doing that? Nevertheless, after getting the content I worked on filtering my mails, based on my very individual rules (see my needs). For example I had to open each link to a new geocache to see, if it is a climbing-cache. I modified this filter rules very often. Being my own Product Owner is very relaxing 🙂
• Evening 4 – Persist it!
  Spring Integration’s ImapMailReseiver reads a new mail in your inbox only a single time. And this makes sense to me. I just want to collect data for my feeds from many sources and separate this from the aspect to provide the data as a feed (view). So I had to store it into a database. Time to setup MongoDB with Spring Data. Great! Again – very very easy. I used a basic CrudRepository of Spring Data for Mongo DB . Setup and storing the first values was done in 30 minutes. Again a little bit refactoring, some interface, some model classes (RssContent, RssContentProvider) and I was able to receive mails, store the extracted content in Mongo and deliver it later as a RssFeed.

  1public interface RssContentRepository extends CrudRepository<RssContent, String> {
  2    List<RssContent> findByUserAndFeed(String user, String feed);

• Evening 5 – Schedule Content-Provider, Twitter
  As I mentioned in step 4, collecting the data should be separated and not be done, if a http request comes in. So I had to setup a scheduling for my ContentProvider. And, what else, very easy. Autowire all ContentProvider in my SchedulerService, create a method, mark it with Spring’s @Scheduled , and call all provider in this method.

  1@Scheduled(fixedRate = 1000 * 60 * 60)
  2public void executeAll() {
  3    for (ContentProvider contentProvider : contentProviderList) {
  4        List<RssContent> content = contentProvider.getContent();
  5        contentRepository.save(content);
  6        ...

  After that I added my Twitter-ContentProvider. I am using Spring Social ‘s Twitter -Api to receive all tweets from my timeline. Again, being my product owner was fun – I filtered some twitter-noise, tried to analyze the links to display pictures directly in my feed… Very funny!

  1List<Tweet> timeline = twitter.timelineOperations().getHomeTimeline(1, 200, lastTweetId, 0);

• Evening 6 – Replace “localhost”
  Unfortunately Google Reader is not running on my localhost. So I had to deploy my application to an accessible public place. Sure, Spring has a solution – Cloud Foundry. I was using the Springsource Toolsuite , so deploying to Cloud Foundry is as easy as deploying to a local tomcat. MongoDB is provided as a service. I only had to create a Spring configuration profile “cloud”, which is using another mongo-db-factory. After that i just added @Profile(“cloud”) to my SchedulerService, because I don’t want this permanent data collection during my development.
• Evening 7 – Refactoring
  At this point, my project only delivered a single feed. Time to take a closer look on my Spring MVC-path mapping. The following three lines and some additional data in my mongo-db provided by the ContentProvider are supporting multiple feeds (and also multiple user, perhaps a future feature 😉 )

  1@RequestMapping(value = "/user/{user}/feed/{feed}", method = RequestMethod.GET)
  2public ModelAndView getFeedContent(@PathVariable String user, @PathVariable String feed) {
  3    Iterable<RssContent> content = contentRepository.findByUserAndFeed(user, feed);
  4    ...

What I want to say with this blog: Spring is great and it can help you in almost all directions of software development. You can always find good documentations, examples or you can get help in the big community. All steps from above were created in less then a few hours. I don’t have to do a deep-dive into all these base technologies to create the first results. But, like with all frameworks – if you want to be a professional developer and want to create stable and scalable solutions you definitely have to know the base technologies in order to understand how your application is working and for example behaves under heavy load.