Built a Miniature Painting News Aggregator

Posted: March 10, 2012 in Op Ed
Tags: , , , , , , , , ,

Although I clearly should be studying or doing something else more productive, I spent a lot of the last week building a miniature painting news aggregator. I didn’t want to build a miniature painting news aggregator. When I want to look at painted miniatures I have a few sites I go to. I subscribe to quite a few blog feeds. I frequent the odd forum. But one site I used to waste time on was From the Warp, particularly the From the Warp sidebars which contained the From the Warp Blogger Group. Miniature Painting News

It was a good way to waste time. It was also a good way to find other blogs to follow. There are other blogging networks targeting gamers or miniature painters, but I thought this one was the best. It certainly was the best curated. It was the one I chose to join when I split my miniature painting exploits off from Muskblog. With no progress seeming to be happening I inquired with Ron what was up. I did some research. I looked at Yahoo Pipes, but the tool I thought could do the job was SimplePie.

I’d previously suggested to Ron to move From the Warp to WordPress even WordPress.com as Blogger/blogspot seems to be limiting what he wants to do with his blog. I still don’t know when the From the Warp Blog Network will be up and functioning. I tried to help. In the mean time I think I’ll pay more attention to the TGN Blog Network and I decided to see what I could build with SimplePie using the feeds I personally follow plus a few more.

Not all feeds are created equal. There are technical problems even bugs in SimplePie. Ideally they’ll get fixed and new release will solve all my problems. In the mean time I decided to let other people poke around and see if they have any suggestions, maybe they are PHP wizards.

As I knew someone would ask, I took a screen shot to try and explain how it works. Most everything is pulled from RSS feeds. Flickr, TGN, even Dakka have working RSS feeds that SimplePie can work with. The rest of the information is pulled from two text files containing a list of feeds. If the From the Warp Network comes back online I’ll try to work it in, but given all the trouble I had with feeds from blogspot, I have my doubts that software will handle 1000 plus feeds effectively and gracefully.

The data in the red boxes to the left comes from one text file. The data on the lower right comes from another text file. The text file that controls the data on the left is going to be smaller and much more exclusive. There is no policy. I didn’t want to create yet another blog network, I just wanted a place to go to look at cool stuff. The most recent post appears at the top and one random feed is featured in the bottom box. I create a thumbnail from the first image in the post. This is perhaps the most intensive and problematic feature of my news aggregator. I’d prefer people just uploaded their finished miniatures to Flickr or a distant second Dakka Dakka. Cool Mini Or Not has no feeds for their gallery and has become more and more of an online store, although there are a lot of nice minis their site doesn’t appeal that much to me. I’m interested in knowing how something was done. Ditto Bell of Lost Souls which has too many writers and just a way too busy site design now. I just want to look at the minis not fight with their web design or told to go fuck myself.

If people want to know how to get their blog included, I suggest they join the TGN Blog Network. If they want to know how they get their website added to one of the two text files I created I suggest they create awesome content. If you consistently create awesome content I or someone else will notice and then I may add your feed assuming you have one that functions. The text file that controls the content on the left I intend to be more exclusive, difficult to get on to, easy to get taken off of. The other text file I’m more willing to just throw a website on, but I’m big on quality content. I can handle the odd picture of your new baby, but if you go on too many tirades or post too much stuff that should end up Facebook or more likely in a private message I’ll be unsubscribing from your feed, removing you from my sidebar and definitely deleting you from the text file that controls the miniature painting news aggregator.

I hope the news aggregator is useful to someone other than me, it took a long time to make. I also hope it isn’t too expensive to maintain so I can keep it lean and mean. Some people would like it to run faster and there are some optimizations that may be possible, but cutting the content in half may make it run twice as fast but is that really desirable? Finding the right balance and the coolest content is the key. If you know of an RSS (or Atom) feed I should include let me know via email or in a comment. Remember I’m a quality over quantity guy and I want painted miniatures or painting advice primarily not your opinion of why Swooping Hawks suck.

Advertisements
Comments
  1. muskie says:

    I’ve tweaked it some more and found a workaround for my problem with SimplePie. The end result is more feeds are being pulled, more are being considered for random inclusion on the bottom left. The bottom right is working better and merges the most recent item from all the blogs that I included in the text files. The top left remains exclusive choosing just the most recent post from amongst a few feeds.

    I also added some social media buttons so you can share the featured feed item with your friends on Facebook or your followers on Twitter.

    I keep looking for good feeds and helping SimplePie find the information. More optimization is still needed and sites like Tumblr and Pinterest may yet have useful feeds, but damned if I can find them. A lot of content is shared on Facebook or in forums but it isn’t easy to access like it is on Flickr or through well functioning feeds like those provided by WordPress.com

  2. […] Comments muskie on Built a Miniature Painting New…Built a Miniature Pa… on There is no ‘F’ or…muskie on Finished Five OOP […]

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s