[Abandoned] I created the Bitcointalk Stats Live Counter

[Husna QA]

-snip- I don't know, maybe something like that has already been offered here, I haven't read the whole topic, but it was possible to add 2 lines in this calculator: -snip-

I tried to compare the BlackHatCoiner version of the Rank Calculator with Iasenko's version. I use my registration date data in this forum; there is a difference in the two versions' date. I don't know exactly which of the two versions is more accurate because I don't log the history as I move up the rankings at each level.


https://btctalk.live/rankcalculator/get.php?date=February%2012,%202018


https://docs.google.com/spreadsheets/d/1khSA28Ab6miwyknp3JvStcL0Yu05SIbGBDZDKcOLO6s/edit#gid=0
@BlackHatCoiner, by the way, I got this


https://btctalk.live/free-bitcoin/

[BlackHatCoiner]

@Husna QA, mine's calculator is surely not accurate. Once I became Sr. Member it was 2 days late. I think that in order to fix this problem and make it 100% accurate I'll have to import the activity unix times on my database and then start calculating each rank based on the list. I can't figure out of a smarter way.

[SFR10]

I think that in order to fix this problem and make it 100% accurate I'll have to import the activity unix times on my database and then start calculating each rank based on the list. I can't figure out of a smarter way.

You'll need one more thing apart from that if you're after 100% accuracy:

• Somehow go over the data for each profile ^[scrape] and exclude periods in which someone has been inactive, then do the automated calculation.
  
  • Perhaps having a limit/interval of some sort in between profile searches, wouldn't put too much stress on the server.
    _{- I'm not a back-end coder but in theory, this should work.}

[BlackHatCoiner]

I added this cool feature that sums all the UTXOs it receives. Then, it refreshes the number every 3 seconds to replace it with the new result. Try it!
These are lots of bitcoins and I got them in less than 1 minute. Mempool, the wild west...



I recently found out that blockchain.info provides an API that contains web sockets completely free. I was going to make a thread about that, but it seems that web sockets are available to the public for free.

[Vod]

I recently found out that blockchain.info provides an API that contains web sockets completely free.

If you want to download larger amounts of data, you'll find it slow or... not free. 

[SFR10]

I added this cool feature that sums all the UTXOs it receives. Then, it refreshes the number every 3 seconds to replace it with the new result. Try it!

A cool feature indeed but you should probably consider moving it to a separate/unique page instead ^{[like all of the links at the bottom of the page]} since it's not connected to other stats from this forum

[BlackHatCoiner]

Spam alert! These are the stats of the last 3 days. Lots of farming accounts may be on their way.

2020-12-18 : New posts: 8499, new topics: 183, new members: 1435
2020-12-19 : New posts: 9224, new topics: 191, new members: 2906
2020-12-20 : New posts: 11429, new topics: 180, new members: 9060

A cool feature indeed but you should probably consider moving it to a separate/unique page instead ^{[like all of the links at the bottom of the page]} since it's not connected to other stats from this forum

It's losing its point if I move it on a different directory. I moved it on the bottom, I think it's the best option I had.

[hosseinimr93]

The information displayed for dates January 16 and January 17 is wrong.
It seems to be a bug.

[BlackHatCoiner]

The information displayed for dates January 16 and January 17 is wrong.
It seems to be a bug.

Yes. That's true. I went into the database directly to see why that and saw this:



A brief explanation.

Generally, I don't save the new posts, topics and members everyday. I just save the total posts, topics and members of every single day and once you want to get the info of one day I show you:

the_day_after - your_day

Example for 2021-01-21's members:
members = 3052229 (which are the total members of 01/22) - 3046556 (which are the total members of 01/21) = 5673

The day after is not written on the list until 01/23/2021 - 00:00 UTC.


Now, I don't understand why it wrote zero for everything in 2021-01-17. It must be a web hosting error? Or mine? I don't know. Unfortunately, since 01/16/2021 is affected by 01/17/2021, I'll have to remove those two dates. Nonetheless, I have seen this once again, but it was correct:

2020-12-24 : New posts: 9259, new topics: 180, new members: 2377
2020-12-25 : New posts: 8673, new topics: -10456, new members: 2692
2020-12-26 : New posts: 9160, new topics: 109, new members: 2535

The "new topics" seems bugged but it's not. An admin/mod just deleted over 10,000 topics that day.

[BlackHatCoiner]

This site is now abandoned due to the usage of CloudFlare in the statistics page.

[LoyceV]

This site is now abandoned due to the usage of CloudFlare in the statistics page.

That's too bad Maybe it was too resource-intensive?
I've been downloading empty pages for the past 2 months too.

[BlackHatCoiner]

Maybe it was too resource-intensive?

No, it doesn't have to do with that. Possibly Cloudflare protects bitcointalk from getting embedded.

[NotATether]

Maybe it was too resource-intensive?

No, it doesn't have to do with that. Possibly Cloudflare protects bitcointalk from getting embedded.

It most likely thinks your script is a bot (which it is TBH) and then presents it a captcha that it doesn't have a chance at solving, at first load. People mainly use it to verify all the visitors are human.

As another example, if the entire forum started using cloudflare to load posts and stuff, then my post scraping script would probably break as well.

[LoyceV]

It most likely thinks your script is a bot (which it is TBH) and then presents it a captcha that it doesn't have a chance at solving, at first load.

Nope, the Statistics Center shows Cloudflare's "Checking your browser"-page on my normal browser too (when I click it for the first time). It's the same thing that happens when you click "All" to see more than one page in a topic, that's what made me think it has to do with resource usage.

[BlackHatCoiner]

As another example, if the entire forum started using cloudflare to load posts and stuff, then my post scraping script would probably break as well.

Yes, but they usually do this for “heavy” pages. A page of a thread isn't going to request much, but if you choose to get all the replies of the thread (by adding ;all next to it in the url) a cloudflare waiting screen will show up.

I really wonder, though, is the statistics page that heavy and why did they change their minds this year? Did it suddenly become heavier or they realized it was already heavy?