Although I clearly should be studying or doing something else more productive, I spent a lot of the last week building a miniature painting news aggregator. I didn’t want to build a miniature painting news aggregator. When I want to look at painted miniatures I have a few sites I go to. I subscribe to quite a few blog feeds. I frequent the odd forum. But one site I used to waste time on was From the Warp, particularly the From the Warp sidebars which contained the From the Warp Blogger Group.
It was a good way to waste time. It was also a good way to find other blogs to follow. There are other blogging networks targeting gamers or miniature painters, but I thought this one was the best. It certainly was the best curated. It was the one I chose to join when I split my miniature painting exploits off from Muskblog. With no progress seeming to be happening I inquired with Ron what was up. I did some research. I looked at Yahoo Pipes, but the tool I thought could do the job was SimplePie.
I’d previously suggested to Ron to move From the Warp to WordPress even WordPress.com as Blogger/blogspot seems to be limiting what he wants to do with his blog. I still don’t know when the From the Warp Blog Network will be up and functioning. I tried to help. In the mean time I think I’ll pay more attention to the TGN Blog Network and I decided to see what I could build with SimplePie using the feeds I personally follow plus a few more.
Not all feeds are created equal. There are technical problems even bugs in SimplePie. Ideally they’ll get fixed and new release will solve all my problems. In the mean time I decided to let other people poke around and see if they have any suggestions, maybe they are PHP wizards.
As I knew someone would ask, I took a screen shot to try and explain how it works. Most everything is pulled from RSS feeds. Flickr, TGN, even Dakka have working RSS feeds that SimplePie can work with. The rest of the information is pulled from two text files containing a list of feeds. If the From the Warp Network comes back online I’ll try to work it in, but given all the trouble I had with feeds from blogspot, I have my doubts that software will handle 1000 plus feeds effectively and gracefully.
The data in the red boxes to the left comes from one text file. The data on the lower right comes from another text file. The text file that controls the data on the left is going to be smaller and much more exclusive. There is no policy. I didn’t want to create yet another blog network, I just wanted a place to go to look at cool stuff. The most recent post appears at the top and one random feed is featured in the bottom box. I create a thumbnail from the first image in the post. This is perhaps the most intensive and problematic feature of my news aggregator. I’d prefer people just uploaded their finished miniatures to Flickr or a distant second Dakka Dakka. Cool Mini Or Not has no feeds for their gallery and has become more and more of an online store, although there are a lot of nice minis their site doesn’t appeal that much to me. I’m interested in knowing how something was done. Ditto Bell of Lost Souls which has too many writers and just a way too busy site design now. I just want to look at the minis not fight with their web design or told to go fuck myself.
If people want to know how to get their blog included, I suggest they join the TGN Blog Network. If they want to know how they get their website added to one of the two text files I created I suggest they create awesome content. If you consistently create awesome content I or someone else will notice and then I may add your feed assuming you have one that functions. The text file that controls the content on the left I intend to be more exclusive, difficult to get on to, easy to get taken off of. The other text file I’m more willing to just throw a website on, but I’m big on quality content. I can handle the odd picture of your new baby, but if you go on too many tirades or post too much stuff that should end up Facebook or more likely in a private message I’ll be unsubscribing from your feed, removing you from my sidebar and definitely deleting you from the text file that controls the miniature painting news aggregator.
I hope the news aggregator is useful to someone other than me, it took a long time to make. I also hope it isn’t too expensive to maintain so I can keep it lean and mean. Some people would like it to run faster and there are some optimizations that may be possible, but cutting the content in half may make it run twice as fast but is that really desirable? Finding the right balance and the coolest content is the key. If you know of an RSS (or Atom) feed I should include let me know via email or in a comment. Remember I’m a quality over quantity guy and I want painted miniatures or painting advice primarily not your opinion of why Swooping Hawks suck.