Bitcoin Forum
December 14, 2017, 02:19:31 PM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: New web service: RSS feeds for topics and users' posts  (Read 205403 times)
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
October 10, 2013, 05:12:35 PM
 #1

= RSS feeds for topics and users' posts =

I would like to announce a new web service that allows you to watch topics and users' activities using an RSS reader.

== Quick guide ==

To subscribe to a specific topic, use following URL in RSS reader: http://100btc.pl/bitcointalk/rss/threads/TID, where TID is an ID of the topic. The ID can be found in url bar in the browser. For example, an URL for the topic "ASICMINER: Entering the Future of ASIC Mining by Inventing It" is https://bitcointalk.org/index.php?topic=99497.0 and its ID is 99497 (please note that ID of a topic always starts with a digit after "topic=" and ends with a digit before a dot). The RSS feed for this topic can be dowloaded from: http://100btc.pl/bitcointalk/rss/threads/99497.

Similary, to subscribe to posts from specific user, use following URL: http://100btc.pl/bitcointalk/rss/users/UID, where UID is an ID of the user. For example, profile of user 'friedcat' is available at https://bitcointalk.org/index.php?action=profile;u=49840 and his ID is 49840. The RSS feed for his posts is available at http://100btc.pl/bitcointalk/rss/users/49840.

== How the service works ==

Known feeds are updated every 10 minutes by cron job and are stored in site cache, so they can be served without interaction with bitcointalk.org server.

When a new topic is retrieved from the site for the first time, the service needs to check the topic existence and its title. To do that it sends a single request to the bitcointalk.org server. If everything is fine, empty feed is generated and is returned to the requestor. After that within a minute the newly generated feed is filled with a content and stored in cache.

==

This is an early version of the service. I am aware of a few things that should be done better. I will fix them in near future but first I would like to know what you think. Any comments and suggestions are welcome.

Just like donations (: 164iKDnK6Ri8HAGbtGMdhhs59FG1f4Vovz.

V.
1513261171
Hero Member
*
Offline Offline

Posts: 1513261171

View Profile Personal Message (Offline)

Ignore
1513261171
Reply with quote  #2

1513261171
Report to moderator
1513261171
Hero Member
*
Offline Offline

Posts: 1513261171

View Profile Personal Message (Offline)

Ignore
1513261171
Reply with quote  #2

1513261171
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1513261171
Hero Member
*
Offline Offline

Posts: 1513261171

View Profile Personal Message (Offline)

Ignore
1513261171
Reply with quote  #2

1513261171
Report to moderator
1513261171
Hero Member
*
Offline Offline

Posts: 1513261171

View Profile Personal Message (Offline)

Ignore
1513261171
Reply with quote  #2

1513261171
Report to moderator
1513261171
Hero Member
*
Offline Offline

Posts: 1513261171

View Profile Personal Message (Offline)

Ignore
1513261171
Reply with quote  #2

1513261171
Report to moderator
greyhawk
Hero Member
*****
Offline Offline

Activity: 924


View Profile
October 10, 2013, 06:15:03 PM
 #2

Is this a cron job per feed?

Or is one cron job for all of them at the same time and we will see the forums grind to a halt every ten minutes in the future?
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
October 10, 2013, 06:41:32 PM
 #3

This is single cron job. It is run every minute. It checks which channels have not been updated in the last 10 minutes and regenerates them. The script sends queries sequentially, never more than one at a time. It is a kind of a simple balancer.  It can be further improved in order not to update more than one or two channels at a time.

V.
nimda
Hero Member
*****
Offline Offline

Activity: 784


0xFB0D8D1534241423


View Profile
October 10, 2013, 10:47:02 PM
 #4

Any guidelines on polling rates?

I recommend asking me for a signature from my GPG key before doing a trade. I will NEVER deny such a request.
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
October 11, 2013, 07:11:00 AM
 #5

Any guidelines on polling rates?

I have configured my reader to poll the feeds every 10 minutes. This means that the delay in the delivery of messages to my inbox does not exceed 20 minutes.

All the feeds are stored on the server as a static files and are served directly by web server, so performance issues are unlikely. If the service will gain in popularity, the distribution of the traffic across several servers will not be problematic.

V.
Kluge
Donator
Legendary
*
Offline Offline

Activity: 1218


Michael, send me some coins before I hitman you


View Profile
October 11, 2013, 07:18:22 AM
 #6

Oh fuck yeah! I'll be stalkin' like a boss, now!  Cool

Don't mix your coins someone said isn't legal
gweedo
Legendary
*
Offline Offline

Activity: 1246


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW
October 11, 2013, 07:23:13 AM
 #7

Any chance you would release the source code on github?

Want to earn 2500 SATOSHIS per hour? Come Chat and Chill in https://goseemybits.com/lobby
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
October 11, 2013, 08:11:39 AM
 #8

Any chance you would release the source code on github?

At the moment? No.
In the future? Maybe.

Everything depends on the number of users. If it will grow, I will have the motivation to work on this project and it is quite possible that I will release the code. At the moment it is only a prototype, which requires too many design decisions to develop it in a larger group of people.

Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
October 21, 2013, 05:51:01 AM
 #9

It's been over a week since announcement of the service and it is right time to post statistics.

The total number of downloads per day:

Code:
2013-10-12 438
2013-10-13 403
2013-10-14 475
2013-10-15 673
2013-10-16 788
2013-10-17 813
2013-10-18 839
2013-10-19 759
2013-10-20 810

Number of bytes downloaded per day:

Code:
2013-10-12 14319652
2013-10-13 13646242
2013-10-14 15697403
2013-10-15 23024745
2013-10-16 25667567
2013-10-17 25903330
2013-10-18 28232714
2013-10-19 24446883
2013-10-20 26504304

Most popular threads:
Code:
1 1072 /bitcointalk/rss/threads/99497 ASICMINER: Entering the Future of ASIC Mining by Inventing It
2 449 /bitcointalk/rss/threads/235763 ASICMINER Speculation Thread
3 320 /bitcointalk/rss/threads/289730 [IPVO] [Multiple Exchanges] Neo & Bee - LMB Holdings
4 317 /bitcointalk/rss/threads/294735 Bitburner Fury - Hashrate Protection
5 298 /bitcointalk/rss/threads/309918 [ANN] Bi•Fury | 5+ GH/s USB Miner [FASTEST USB MINER IN THE WORLD]
6 297 /bitcointalk/rss/threads/179769 [Work in progess] Burnins Avalon Chip to mining board service
7 272 /bitcointalk/rss/threads/266828 [BTC-TC] BTC Growth: Capital Growth via Hedge Fund-Style Investing
8 260 /bitcointalk/rss/threads/86854 [NastyFans.org] NASTY MINING | NASTY POOL
9 232 /bitcointalk/rss/threads/280570  [BTC-TC] CryptX introduces the PETA-MINE - 18,000 CHIPS IN SEPTEMBER/UPDATE
10 116 /bitcointalk/rss/threads/310783 [Crypto-stocks] PETA-MINE dividend 7,3% paid
11 104 /bitcointalk/rss/threads/297503 [ActiveMining] The Official Active Mining Discussion Thread [Self-Moderated]

Most popular users:
Code:
1 1178 /bitcointalk/rss/users/49840 friedcat
2 272 /bitcointalk/rss/users/84153 Felipeo
3 270 /bitcointalk/rss/users/142607 cryptx
4 269 /bitcointalk/rss/users/70823 burnin
5 266 /bitcointalk/rss/users/38146 someone42
6 259 /bitcointalk/rss/users/127826 cryptocyprus
7 258 /bitcointalk/rss/users/40149 Deprived
8 257 /bitcointalk/rss/users/132160 DrGregMulhauser
9 92 /bitcointalk/rss/users/1707 kslaughter
10 88 /bitcointalk/rss/users/64492 havelock

If you have any ideas on how to improve the service, please post them in this thread.

Thanks for using the service.

V.
go1111111
Full Member
***
Offline Offline

Activity: 182


View Profile
November 23, 2013, 02:17:01 AM
 #10

Feature request: allow having the posts from multiple users in one feed. I don't want to have a separate feed for 20 people. I just want a "interesting stuff on BCTalk" feed.

Nice work overall.

Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
November 26, 2013, 06:50:29 AM
 #11

Feature request: allow having the posts from multiple users in one feed. I don't want to have a separate feed for 20 people. I just want a "interesting stuff on BCTalk" feed.

Nice work overall.


Sorry for late response. Let me understand your needs correctly. Do you know which users you want to observe? If so you should use some kind of news aggregator like Digg Reader or Feedly (these are cloud services) or TinyRSS (you can host it yourself) or Thunderbird (and I belive that, there are tons of extensions for Firefox and Chrome that might be usefull for you as well). You will get single list of news from bitcointalk.org and you will be able to share it with other users.

Or maybe you don not know what to read and you want advice? In that case I can not help you because I do not know what you are interested in. Perhaps other users have their own reading lists and can share.

V.
maurya78
Hero Member
*****
Offline Offline

Activity: 490


View Profile
November 26, 2013, 10:24:48 AM
 #12

Overall high utility service

Thanks

go1111111
Full Member
***
Offline Offline

Activity: 182


View Profile
November 27, 2013, 08:36:44 PM
 #13

Sorry for late response. Let me understand your needs correctly. Do you know which users you want to observe? If so you should use some kind of news aggregator like Digg Reader or Feedly (these are cloud services) or TinyRSS (you can host it yourself) or Thunderbird (and I belive that, there are tons of extensions for Firefox and Chrome that might be usefull for you as well). You will get single list of news from bitcointalk.org and you will be able to share it with other users.

My issue is that I already have an RSS reader (The Old Reader), and I have lots of other RSS feeds in it. So my "view all feeds" feed contains both bitcointalk posts and lots of other stuff that I may not be in the mood for. What I'd like to do is have the bitcoin talk feeds appear in one "feed"/stream, so that I can just click that feed and see what's new on bitcointalk today. But currently I have to read the individual feed for each user.

Are you saying that other RSS readers allow you to create custom feeds composed of multiple 'base' feeds?

Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
November 28, 2013, 06:45:22 AM
 #14

Are you saying that other RSS readers allow you to create custom feeds composed of multiple 'base' feeds?


Yes. For example Feedly has a nice feature called folders. You can put all your BCT feeds in one folder and read them as a single stream of news.

V.
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
December 02, 2013, 06:45:30 AM
 #15

The service is currently shut down due to following error:

2013-12-02 07:14:04.269287 Can't open page https://bitcointalk.org/index.php?action=profile;u=142067;sa=showPosts: Connection refused.

I do not know why this is happening. Perhaps administrators have blocked some IP addresses on the firewall in response to recent attacks and accidentally they have killed an innocent. I want to talk with administrators and clarify the case, but unfortunately I do not know who is right person to this talk. Can you help me?

V.
gweedo
Legendary
*
Offline Offline

Activity: 1246


Java, PHP, HTML/CSS Programmer for Hire!


View Profile WWW
December 02, 2013, 07:09:00 AM
 #16

The service is currently shut down due to following error:

2013-12-02 07:14:04.269287 Can't open page https://bitcointalk.org/index.php?action=profile;u=142067;sa=showPosts: Connection refused.

I do not know why this is happening. Perhaps administrators have blocked some IP addresses on the firewall in response to recent attacks and accidentally they have killed an innocent. I want to talk with administrators and clarify the case, but unfortunately I do not know who is right person to this talk. Can you help me?

V.

PM theymos https://bitcointalk.org/index.php?action=profile;u=35

Want to earn 2500 SATOSHIS per hour? Come Chat and Chill in https://goseemybits.com/lobby
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
December 02, 2013, 08:13:50 AM
 #17

The service is currently shut down due to following error:

2013-12-02 07:14:04.269287 Can't open page https://bitcointalk.org/index.php?action=profile;u=142067;sa=showPosts: Connection refused.

I do not know why this is happening. Perhaps administrators have blocked some IP addresses on the firewall in response to recent attacks and accidentally they have killed an innocent. I want to talk with administrators and clarify the case, but unfortunately I do not know who is right person to this talk. Can you help me?

V.

PM theymos https://bitcointalk.org/index.php?action=profile;u=35

Thanks. Done. Waiting for response.

V.
Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
December 03, 2013, 07:54:44 AM
 #18

We are online again.

V.
Voodah
Full Member
***
Offline Offline

Activity: 238



View Profile
January 03, 2014, 03:14:55 AM
 #19

This is great. I've always wanted something like this.



Vysogota
Newbie
*
Offline Offline

Activity: 16


View Profile
February 26, 2014, 08:28:38 AM
 #20

It took me a little more than a month to find some time and check why RSS feeds for the topics stopped working. It turned out that the threads template has been changed, so the parser stopped recognize the posts.

At the moment everything seems to be working.

Sorry for the long response time, but this is a project that I'm doing in my spare time.

V.
Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!