[Request] Stats

[cabalism13]

I'm just curious on how lively the forum is?

Request for this Stats / or maybe someone can give me a link where I can see a real-time update of Stats:

Ex:
Real-Time Users Online: https://bitcointalk.org/index.php?action=stats
Number of Users Online within 24 Hours:
Number of Users Online within the past Week:
Number of Users Online within the past Month:
Number of Users Online on 2018:
Number of Users Online on 2019:
Current Unique Number of Users Online on 2020:

Also I am curious about the traffic being get/generated by the Forum.
Are we still on the Top Ranks on The Most Visited Websites in the World?


Searched from the Past and these are the things I get:
https://bitcointalk.org/index.php?topic=4948780.0
https://bitcointalk.org/index.php?topic=5263950.0

And as I've tried the SimilarWeb with comparison to other crypto websites, I was given this ( I don't know if this was accurate - So I'm asking for someone who can give me an accurate statistics on this or where can I found one )

Also would like to request (I think this is a hard one) the total logs per Country (listed on our Local Sections) - SimilarWeb doesn't allow me for I don't have Premium Account

[1714945467]

1714945467

[1714945467]

1714945467

[1714945467]

1714945467

[1714945467]

1714945467

[tranthidung]

Statistics Bitcointalk.org ^[1] which used the statistics on BPIP.org.

With support from LoyceV and data on loyce.club, I observed following daily statistics:

• Total Members
• Total Posts
• Total Topics
• Average online per day

The observed period is a bit more than one month so I have not yet published it. Because of your request, I will show a time series plot for you later today (not have connection to dataset now).

Of course, all of statistics do not satisfy your request completely.  

According to ^[1], there is a 3.6% drop in total active profiles between two months (July compares to June).

Code:

. di (92373-95788)/95788*100
-3.5651647

[LoyceV]

Number of Users Online within 24 Hours:
Number of Users Online within the past Week:

I don't keep track of online users, only users who posted. See Active users, top posters and most active topics in the past 1h, 24h and 7d. It's not real-time though.

[tranthidung]

Here you go:

All average statistics have been on declining trends.

Forum statistics (on daily basis)

• Observed period: 01/7/2020 - 05/8/2020.
• new members, new posts, new topics.
• a_rpd: Average registration per day
• a_opd: Average online per day
• a_ppd: Average posts per day
• n_mem: Daily new members
• n_posts: Daily new posts
• n_topics: Daily new topics

Time series plots

Code:

     +--------------------------------------------------------------------+
     |      date   n_mem   n_topics   n_posts    a_rpd    a_opd     a_ppd |
     |--------------------------------------------------------------------|
  1. | 01jul2020       .          .         .   368.34   155.51   6762.03 |
  2. | 02jul2020     333        184      5551   368.25   155.47   6760.29 |
  3. | 03jul2020     393        164      5523   368.15   155.43   6758.55 |
  4. | 04jul2020     421        178      5532   368.06   155.39   6756.81 |
  5. | 05jul2020     335        135      5091   367.96   155.35   6755.07 |
     |--------------------------------------------------------------------|
  6. | 06jul2020     316         85      5585   367.87   155.31   6753.33 |
  7. | 07jul2020     501        148      5300   367.77   155.27   6751.59 |
  8. | 08jul2020     391        172      6368   367.68   155.23   6749.85 |
  9. | 09jul2020     361        202      6724   367.58   155.19   6748.12 |
 10. | 10jul2020     418        133      6448   367.49   155.15   6746.38 |
     |--------------------------------------------------------------------|
 11. | 11jul2020     417         82      6455   367.39   155.11   6744.65 |
 12. | 12jul2020     361        123      6025    367.3   155.07   6742.91 |
 13. | 13jul2020     277        111      6377    367.2   155.03   6741.18 |
 14. | 14jul2020     369        166      6615   367.11   154.99   6739.45 |
 15. | 15jul2020     404        154      7369   367.02   154.95   6737.72 |
     |--------------------------------------------------------------------|
 16. | 16jul2020     437        158      7049   366.92   154.91   6735.99 |
 17. | 17jul2020     398        176      6929   366.83   154.87   6734.26 |
 18. | 18jul2020     398        136      6500   366.73   154.83   6732.53 |
 19. | 19jul2020     351         96      6228   366.64   154.79    6730.8 |
 20. | 20jul2020     471        151      7534   366.55   154.75   6729.07 |
     |--------------------------------------------------------------------|
 21. | 21jul2020     384        153      6698   366.45   154.71   6727.35 |
 22. | 22jul2020     399        155      6870   366.36   154.67   6725.62 |
 23. | 23jul2020     376        176      6511   366.26   154.63    6723.9 |
 24. | 24jul2020     345        100      6567   366.17   154.59   6722.17 |
 25. | 25jul2020     372        166      6532   366.08   154.55   6720.45 |
     |--------------------------------------------------------------------|
 26. | 26jul2020     312        130      6262   365.98   154.51   6718.73 |
 27. | 27jul2020     286        145      6948   365.89   154.47   6717.01 |
 28. | 28jul2020     388         59      6536   365.79   154.39   6715.29 |
 29. | 29jul2020     382        175      6773    365.7   154.39   6713.57 |
 30. | 30jul2020     391        171      6403   365.61   154.35   6711.85 |
     |--------------------------------------------------------------------|
 31. | 31jul2020     402        196      6175   365.51   154.31   6710.14 |
 32. | 01aug2020     401         69      6810   365.42   154.27   6708.42 |
 33. | 02aug2020     320        135      5925   365.33   154.23   6706.71 |
 34. | 03aug2020     325        132      6523   365.23    154.2   6704.99 |
 35. | 04aug2020     355        118      6000   365.14   154.16   6703.28 |
     |--------------------------------------------------------------------|
 36. | 05aug2020     422        141      6316   365.05   154.12   6701.57 |
     +--------------------------------------------------------------------+

With multipliers (figures at the end are multipliers)

[Harlot]

I remembered a topic I had read before about the "user stats" being disabled due to the forum having performance issues with it so I think the same case will happen to the forum if they bring out this real-time data as it will bring more load to the website. Maybe having an external site like what LoyceV have is the solution here if you are interested in pulling out this data from Bitcointalk. 

[NeuroticFish]

Number of Users Online within 24 Hours:
Number of Users Online within the past Week:
Number of Users Online within the past Month:

I don't know how good/updated this is, since I've only noticed it not long ago in @BlackHatCoiner personal text and I didn't actually need it, however it may worth a look:
https://btctalk.live and https://btctalk.live/chartonline/

[Casdinyard]

Also would like to request (I think this is a hard one) the total logs per Country (listed on our Local Sections) - SimilarWeb doesn't allow me for I don't have Premium Account

I've been working on this for almost a month but first I'm trying to scrape users based from different countries by their local board posts, by simply going to a local board and scrape every comments in every thread and in every page (this is really hard). I hope I could patent this idea so my work and time spent studying scraping wouldn't be wasted. After that, I think it would be easy if I had stats with the links of local users and simply scrape when they are active (but would take me another month studying another prog language I guess).

I've also planning to make a thread saying I'm working on it but some might steal the idea.

[cabalism13]

Here you go:

New Members? or is it just the name? Been looking for the Online User Stats, total Logins if you may. 😁

[tranthidung]

I've been working on this for almost a month but first I'm trying to scrape users based from different countries by their local board posts, by simply going to a local board and scrape every comments in every thread and in every page (this is really hard). I hope I could patent this idea so my work and time spent studying scraping wouldn't be wasted. After that, I think it would be easy if I had stats with the links of local users and simply scrape when they are active (but would take me another month studying another prog language I guess).

I appreciated your time to learn and your plan to build up a project. Honestly, I don't know how to scrape website's data.

Quality is better than quantity.

Figures on active or inactive members do not reflect the real situation on the forum, such as the average quality of posts or threads created by members recently. Imagine that an active local board can be detected by higher number of posts in latest few weeks compare to the other local boards. But such figures can not make sure the quality of that active local board that is probably a spam place. More active more spam endemic inside.

A wider view, the forum used to be more actively in the past, late of 2017 and early of 2018, but now we have a very better forum (less spam, higher quality).

[LoyceV]

by simply going to a local board and scrape every comments in every thread and in every page (this is really hard). I hope I could patent this idea

Actually, it's quite easy Getting all topics on a board is something I did in January last year, and once you have all topicIDs, you can scrape all topics (which takes some time).

[Casdinyard]

A wider view, the forum used to be more actively in the past, late of 2017 and early of 2018, but now we have a very better forum (less spam, higher quality).

I think the merit implementation causes that. Back then the forum was bombarded with a lot of newbie like there's no day nor an hour that there wasn't an account made in the forum. But then now, making a lot of accounts would be useless as spam posters could easily be detected by other active members, and as merits are required to gain reputation as well as it shows a proof that you're a good member of the community.

by simply going to a local board and scrape every comments in every thread and in every page (this is really hard). I hope I could patent this idea

Actually, it's quite easy Getting all topics on a board is something I did in January last year, and once you have all topicIDs, you can scrape all topics (which takes some time).

Would you help me? I've been coding and trying to figure out ways to scrape all comments in all threads yet I keep encountering bugs with my code (python). I hope to make a thread with such data soon and might use the code to of course scrap all local boards. But I'm also trying some other apps such as octoparse but had issues with the chronological order of tasks I want to gain..

[LoyceV]

Would you help me? I've been coding and trying to figure out ways to scrape all comments in all threads yet I keep encountering bugs with my code (python).

I did scrape all posts already (but didn't keep track of the board), so it would be a waste of forum bandwidth to do it again. I know nothing about Python though.
What are you trying to accomplish?

[Casdinyard]

Would you help me? I've been coding and trying to figure out ways to scrape all comments in all threads yet I keep encountering bugs with my code (python).

I did scrape all posts already (but didn't keep track of the board), so it would be a waste of forum bandwidth to do it again. I know nothing about Python though.
What are you trying to accomplish?

Imagine all threads in Pilipinas section (which I'll try to test as I am from PH), I'll try to scrape each and every comments in each threads posted into it, but all I will extract is all the usernames and their links, meaning that I could enlist all users that are filipino here in the forum as they posted once in a thread located in our local board. Then Ill try to do the same on other local boards and compare the stats to how many users are from this and that country and which country has the most users here. Though it wouldn't be a 100% accurate, but it would still be a data to consider.

[LoyceV]

Though it wouldn't be a 100% accurate

You'll find me in many local boards

[The Sceptical Chymist]

I'd like to see this graph overlaid with a bitcoin price chart--I'm not a statistics geek and am not really interested in member activity, posts, registration, and so on, but it's interesting to me that it looks like activity and registration of new accounts booms and busts along with how bitcoin is performing.  That's probably obvious to a lot of members, but I don't think I've ever seen any statistical correlation between btc price and bitcointalk variables.

[tranthidung]

I'd like to see this graph overlaid with a bitcoin price chart--I'm not a statistics geek and am not really interested in member activity, posts, registration, and so on, but it's interesting to me that it looks like activity and registration of new accounts booms and busts along with how bitcoin is performing.  That's probably obvious to a lot of members, but I don't think I've ever seen any statistical correlation between btc price and bitcointalk variables.

You can see more details at my another thread: Assumed monthly statistics on registered accounts of bitcointalk.org (2009-2019)

Plots are not overlaid because I don't remember where I stored data for those plots hence I can not re-draw them now. From two plots, you can see unconfirmed correlation between forum members' activities and BTC price. Remember there are bias factors: merit system's kick-off and the demotion on Jr. member -- they are likely random shocks but effects are systematic. To support this assumption, you can see plots for forum ads' statistics (at end of this post), there are visual correlations sure.

That one is an overlaid plot but for 2019 only.

Regarding to forum members' activities, there is Time Series on monthly statistics of forum (new users, new topics, new posts)

Next few weeks, I will spend a little time each day to see how I can do with all collective data, to have such same plots in Impression counts for ads over years

[Halab]

Request for this Stats / or maybe someone can give me a link where I can see a real-time update of Stats:

I'm a bit late and I'm not really going to answer your request, but I just remembered that there is this page that lists online members : https://bitcointalk.org/SSI.php?ssi_function=whosOnline

From this page, a "statistician" can generate the stats you want (but he won't be able to go back in time).
The only drawback : you can't see the members who have hidden their status.