{"id":544,"date":"2011-03-15T08:37:16","date_gmt":"2011-03-15T14:37:16","guid":{"rendered":"http:\/\/www.techory.com\/blog\/hacking-rss-filtering-processing-obscene-amounts-of-information\/"},"modified":"2011-03-17T08:06:56","modified_gmt":"2011-03-17T14:06:56","slug":"hacking-rss-filtering-processing-obscene-amounts-of-information","status":"publish","type":"post","link":"http:\/\/techory.com\/sxsw\/?p=544","title":{"rendered":"Hacking RSS: Filtering &amp; Processing Obscene Amounts of Information"},"content":{"rendered":"<p><strong>Presenter<br \/><\/strong><span class=\"pres_name\"><a href=\"http:\/\/fastwonderblog.com\/\">Dawn Foster<\/a>,<\/span> <span class=\"pres_title\">MeeGo Community Mgr<\/span> <span class=\"pres_company\">Intel<br \/><a href=\"http:\/\/fastwonderblog.com\/2011\/03\/11\/hacking-rss-filtering-processing-obscene-amounts-of-information-at-sxsw\/\">Presentation Slides and Videos<\/a><\/span><\/p>\n<p>295 Exabytes of data in 2007, amount doubles every 3 years, 4 months. Over 600+ Exabytes now. You want to find the needle in all of this data. <\/p>\n<p>RSS Alone is a start. You can follow the sources you want, but&#8230;<\/p>\n<ul>\n<li>Do you care about everything in each feed?<\/li>\n<li>What about feeds you aren&#8217;t subscribed to?<\/li>\n<li>Can you keep up with what you have?<\/li>\n<\/ul>\n<p><strong>Prioritize Your Reader (Google Reader)<\/strong><\/p>\n<ul>\n<li>Put thins you care about at the top (yahoo pipes, things you really really like)<\/li>\n<li>Categorize<\/li>\n<li>Don&#8217;t try to read everything. Get to what you can.<\/li>\n<\/ul>\n<p><strong>Outsource and Crowd-source New Sources<\/strong><\/p>\n<ul>\n<li><a href=\"http:\/\/tweetedtimes.com\/\">Tweeted Times<\/a><\/li>\n<li><a href=\"http:\/\/digg.com\">Digg<\/a><\/li>\n<li><a href=\"http:\/\/reddit.com\">Reddit<\/a><\/li>\n<li><a href=\"http:\/\/techmeme.com\">Techmeme<\/a><\/li>\n<li><a href=\"http:\/\/stumbleupon.com\">Stumbleupon<\/a><\/li>\n<li><a href=\"http:\/\/news.google.com\">Google News<\/a><\/li>\n<\/ul>\n<p><strong>The Real Magic is in Filtering RSS<\/strong><\/p>\n<p>In Google reader, a yahoo pipe of analyst research blogs mentioning Online Community, a yahooo pipe of analyst research blogs mentioning Meego.<br \/>You need to filter out thing you don&#8217;t care about.<br \/>Another yahoo pipe pulls in favorite blogs using PostRank to find only the ones with a lot of comments or social mentions.<\/p>\n<p><strong>RSS Filtering Tools<\/strong><\/p>\n<ul>\n<li><a href=\"http:\/\/pipes.yahoo.com\/\">Yahoo Pipes<\/a><br \/>You can filter any data found in any field of the RSS feed.<\/li>\n<li>FeedRinse<\/li>\n<li>FeedDemon<\/li>\n<li>Code your own<\/li>\n<\/ul>\n<p><a href=\"http:\/\/www.postrank.com\/\">PostRank<\/a><\/p>\n<ul>\n<li>Takes the best posts in a feed<\/li>\n<li>Ranks it on engagement (links\/sharing\/comments\/etc.)<\/li>\n<li>You can get the output as an RSS feed<\/li>\n<li>Feed includes postrank number in a field which you can filter against.<\/li>\n<\/ul>\n<p><a href=\"http:\/\/backtweets.com\/\">BackTweets<\/a><\/p>\n<ul>\n<li>Data about links on Twitter<\/li>\n<li>Finds links regardless of shortening service<\/li>\n<li>No RSS Feeds (no longer available)<\/li>\n<li>But&#8230; You can use the API = Yahoo Pipes to build one!<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>PresenterDawn Foster, MeeGo Community Mgr IntelPresentation Slides and Videos 295 Exabytes of data in 2007, amount doubles every 3 years, 4 months. Over 600+ Exabytes now. You want to find the needle in all of this data. RSS Alone is a start. You can follow the sources you want, but&#8230; Do you care about everything [&#8230;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[30],"tags":[],"_links":{"self":[{"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/posts\/544"}],"collection":[{"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=544"}],"version-history":[{"count":2,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/posts\/544\/revisions"}],"predecessor-version":[{"id":562,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=\/wp\/v2\/posts\/544\/revisions\/562"}],"wp:attachment":[{"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=544"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=544"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/techory.com\/sxsw\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=544"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}