Bitcoin Forum

Why there is a Clone of bitcointalk.org in the web?

https://bitcointa.lk/
It contains copies of all posts from Bitcointalk.org.

It's the NSA's backup lol.

Hi, i coded Bitcointa.lk. Announcement thread: https://bitcointalk.org/index.php?topic=515932.0 (https://bitcointalk.org/index.php?topic=515932.0)
I always hated how Bitcointalk.org was administrated (or how it was not administrated at all), so i woke up one day and decided to do something.

Short features list:
-Real time synch of every message
-Posts removed on Bitcointalk.org are marked as removed on Bitcointa.lk, but you can still read them; you can also see a history of deleted messages (http://imgur.com/a/9F1qU)
-You can use it to post on Bitcointalk.org (i personally do it); it would become your gateway to access Bitcointalk.org

And naturally, i did it for free and i didn't spent a million dollar in the process.
Posted from Bitcointa.lk - #aROgnmf7qUAOcdNM

Uhm ok ...

I will stay here @bitcointalk...
For me it is very well administrated ...

To each his own :)
However, you know my point of view ;)

https://pbs.twimg.com/media/Bjb6W8ICQAA1lGh.png
Posted from Bitcointa.lk - #ObNwFoodr01feWHc

Its a backup of all posts + its beautiful!

For me it is not very good that you can read deleted posts ... some of them were dangerous  - scam, phishing, etc.

Well its really useful for self moderated threads. You can see exactly what people posted.

well from this point of view maybe yes.
... if it would be moderated and he would remove scam posts ... and leave only deleted posts from self moderated threads it can be even more useful. Isn't?

Anyway I really think Bitcointalk.org is no.1 and I never was @ better forum... And just in my opinion is that making a clone that "looks more fancy" and contains deleted post (what is not IMO as good as you know already) is nothing special and it might be even from some points not really nice from a point of view of users that deleted their post, because for example - they did posted their e-mails or something, that they recon they do not want to keep @ forum for some reason.

No one is saying you should use it... I love the current layout of bitcointalk, but some dont. Those who dont now atleast have an option.

Deleted content is only visible to registered (linked) users. So there is no "i accidentally posted my email address" problem.
Posted from Bitcointa.lk - #pJeGZFI18BfrDwuA

Good job. I think the bitcointalk forum could do a much better job at attracting new bitcoin users. Now they come here and see a default simple machines forum with a few tweaks. It could be much more inviting and professional. Bitcointalk also has lots of funds to accomplish this from all of the donations that are now worth a small fortune.

I just wanted to say thank you to theymos for having decided to ban our crawler twice during the last 12 hours.
Posted from Bitcointa.lk - #yUXGFN4tVExKSjIn

How frequently does it poll? Possible it was auto-blocked as DoS.

It depends on the number of posts, however a total of < 50k requests/day over 5 IPs.
All together, around one per 2 seconds.

If it was auto blocked as DoS it's one thing, if he just decided to block the competition while this forum is left dying with unloading images, server errors everywhere and a 10 years old board, well, i expected that, but i still have no words.
Posted from Bitcointa.lk - #0zLZOUaxU7Sr2W6A

I don't think Theymos banned it, its a rather useful site, the moderation staff are using it to catch the people that are bump spamming, and then deleting their posts to get around the 24 hour bump limit.

I'm surprised to hear that it's used even from the staff. :)
I'm sorry for having complained, but I'm quite sure it was a manual ban.

I will just use it as an excuse to make the crawler smarter and able to detect when banned... It was in the todo anyway.
Posted from Bitcointa.lk - #xRyIEg5gjSsOYBPf

Your crawler is probably taxing on the server.  How often is it running, and how many pages does it load at once?

Haha, sneaky bastards.

It has been running just fine for a month now, and the server handled it perfectly until lately (last 3 days?), when the forum started lagging, but that's not related to a change on my side.

Given that i own a website that has in the order of millions (5-10) hits per day (of non cached and dynamic content, not of gifs), i would be happy to give a suggestion or two on the infrastructure behind bitcointalk.org.
Posted from Bitcointa.lk - #RhbaqkRpcd9z6Mk3

What IP?

IPs are only banned for over-use if they request pages more frequently than once per second. Faster requests are not allowed.

So now we have two bitcointalks :)

Great ... :P
Anyway if you all think this is ok - good.
But some people might be confused while visiting bitcointa.lk.

Thank you for passing by, theymos.
Have you changed the behaviour of the DoS automatic banner recently? Because it's the only rationale solution i can think off, if those bans are not manual. The crawler worked just fine for a month until yesterday.

The 5 IPs i'm using right now are 54.217.199.213, 46.137.37.197, 54.195.167.127, 54.195.152.218, 54.73.190.33.

Let me simply know if you want me to kill this project and the crawler.
Posted from Bitcointa.lk - #nicm5KCTPCqeXZOn

Those IPs are not blocked currently. But your other abusive IPs were blocked. Just your quotefast requests (which are only part of what your crawler does) were occurring at an average frequency of 7.6 requests per second in the most recent access logging period. Your requests constituted 3.4% of all forum requests in this period. This is entirely unacceptable and of course resulted in those IPs being banned.

The maximum allowed bot request frequency is 1 request per second. Those IPs are now accessing pages at an average of 2.5 requests per second combined. If you continue exceeding the allowed request limit, we will continue banning your IPs.

The 7.6 peak was due to the backlog i had on edited content, that i implemented yesterday. If you check the traffic one week ago you should see an average way less than 1 request per second. (and i'm sort of sorry for this peak).

This forum has less than 30k posts per day, so those numbers doesn't add up on my side, something must be wrong (especially since that i removed the sync of modified content after this problem). I will write a way to see how many requests i actually do and what % fails/etc. The problem could be peak hours, when there is more than 1 post per second... But i doubt that.
Posted from Bitcointa.lk - #yCkCrM3Xkyk4sagw