Bitcoin Forum

= RSS feeds for topics and users' posts =

I would like to announce a new web service that allows you to watch topics and users' activities using an RSS reader.

== Quick guide ==

To subscribe to a specific topic, use following URL in RSS reader: http://100btc.pl/bitcointalk/rss/threads/TID, where TID is an ID of the topic. The ID can be found in url bar in the browser. For example, an URL for the topic "ASICMINER: Entering the Future of ASIC Mining by Inventing It" is https://bitcointalk.org/index.php?topic=99497.0 and its ID is 99497 (please note that ID of a topic always starts with a digit after "topic=" and ends with a digit before a dot). The RSS feed for this topic can be dowloaded from: http://100btc.pl/bitcointalk/rss/threads/99497.

Similary, to subscribe to posts from specific user, use following URL: http://100btc.pl/bitcointalk/rss/users/UID, where UID is an ID of the user. For example, profile of user 'friedcat' is available at https://bitcointalk.org/index.php?action=profile;u=49840 and his ID is 49840. The RSS feed for his posts is available at http://100btc.pl/bitcointalk/rss/users/49840.

== How the service works ==

Known feeds are updated every 10 minutes by cron job and are stored in site cache, so they can be served without interaction with bitcointalk.org server.

When a new topic is retrieved from the site for the first time, the service needs to check the topic existence and its title. To do that it sends a single request to the bitcointalk.org server. If everything is fine, empty feed is generated and is returned to the requestor. After that within a minute the newly generated feed is filled with a content and stored in cache.

==

This is an early version of the service. I am aware of a few things that should be done better. I will fix them in near future but first I would like to know what you think. Any comments and suggestions are welcome.

Just like donations (: 164iKDnK6Ri8HAGbtGMdhhs59FG1f4Vovz.

V.

Is this a cron job per feed?

Or is one cron job for all of them at the same time and we will see the forums grind to a halt every ten minutes in the future?

This is single cron job. It is run every minute. It checks which channels have not been updated in the last 10 minutes and regenerates them. The script sends queries sequentially, never more than one at a time. It is a kind of a simple balancer.  It can be further improved in order not to update more than one or two channels at a time.

V.

Any guidelines on polling rates?

I have configured my reader to poll the feeds every 10 minutes. This means that the delay in the delivery of messages to my inbox does not exceed 20 minutes.

All the feeds are stored on the server as a static files and are served directly by web server, so performance issues are unlikely. If the service will gain in popularity, the distribution of the traffic across several servers will not be problematic.

V.

Oh fuck yeah! I'll be stalkin' like a boss, now!  8)

Any chance you would release the source code on github?

At the moment? No.
In the future? Maybe.

Everything depends on the number of users. If it will grow, I will have the motivation to work on this project and it is quite possible that I will release the code. At the moment it is only a prototype, which requires too many design decisions to develop it in a larger group of people.

It's been over a week since announcement of the service and it is right time to post statistics.

The total number of downloads per day:


Number of bytes downloaded per day:


Most popular threads:

Most popular users:

If you have any ideas on how to improve the service, please post them in this thread.

Thanks for using the service.

V.

Feature request: allow having the posts from multiple users in one feed. I don't want to have a separate feed for 20 people. I just want a "interesting stuff on BCTalk" feed.

Nice work overall.

Sorry for late response. Let me understand your needs correctly. Do you know which users you want to observe? If so you should use some kind of news aggregator like Digg Reader or Feedly (these are cloud services) or TinyRSS (you can host it yourself) or Thunderbird (and I belive that, there are tons of extensions for Firefox and Chrome that might be usefull for you as well). You will get single list of news from bitcointalk.org and you will be able to share it with other users.

Or maybe you don not know what to read and you want advice? In that case I can not help you because I do not know what you are interested in. Perhaps other users have their own reading lists and can share.

V.

Overall high utility service

Thanks

My issue is that I already have an RSS reader (The Old Reader), and I have lots of other RSS feeds in it. So my "view all feeds" feed contains both bitcointalk posts and lots of other stuff that I may not be in the mood for. What I'd like to do is have the bitcoin talk feeds appear in one "feed"/stream, so that I can just click that feed and see what's new on bitcointalk today. But currently I have to read the individual feed for each user.

Are you saying that other RSS readers allow you to create custom feeds composed of multiple 'base' feeds?

Yes. For example Feedly has a nice feature called folders. You can put all your BCT feeds in one folder and read them as a single stream of news.

V.

The service is currently shut down due to following error:

2013-12-02 07:14:04.269287 Can't open page https://bitcointalk.org/index.php?action=profile;u=142067;sa=showPosts: Connection refused.

I do not know why this is happening. Perhaps administrators have blocked some IP addresses on the firewall in response to recent attacks and accidentally they have killed an innocent. I want to talk with administrators and clarify the case, but unfortunately I do not know who is right person to this talk. Can you help me?

V.

Thanks. Done. Waiting for response.

V.

We are online again.

V.

This is great. I've always wanted something like this.

It took me a little more than a month to find some time and check why RSS feeds for the topics stopped working. It turned out that the threads template has been changed, so the parser stopped recognize the posts.

At the moment everything seems to be working.

Sorry for the long response time, but this is a project that I'm doing in my spare time.

V.

Looks like this might have happened again - not seeing some topic threads updated for more than 10 days.

Is this still being supported? I haven't seen RSS updates from this tool in quite some time.

Is there any way of generating RSS feeds?

The system is back. There is a problem with supervisor. I will investigate the problem later this week. Send me private message next time you notice any further problems.

V.

PS. I will consider opening the source code.

Great service  ;D

Is there a way to watch an entire board, so that you get an update every time there is a new thread?
For example, a rss feed that sends you new threads created on the meta board.

Nice to see that RSS Feed service, I'm going to try it tomorrow  ;)

But first I want to ask something to developer:

CAUTION: It's just a comment, not a criticism.
Can you please explain why you prefer to not release the code for example on github? You want to monetize it on the near future or something?

IMAO, people interested in bitcoin appreciate open-source projects and I think that if you release the code on github and you open a BTC address for donations, people that use it is going to donate some satoshis for you  :D

That's just my opinion  ;)

Well, this is an old post. 2013! Seems like just last week...

lol  :D didn't see it! haha
I wanted to try it tomorrow but I didn't noticed that's a 2013 post.

As well, @Vysogota Last Active: November 18, 2014, 09:06:07 AM
So… I think that we're not going to receive any response  :D