Data regarding the forum

[xfactor13]

I do not know if this is the right board to ask this (kindly assist me and I will move this one to other board if Meta is not appropriate)

I see several posts regarding the data of the forum (one most common is about merits i.e. distribution, comparison, etc).
I just want to ask if the data regarding the forum, let us say Merit distribution data for example, can it be extracted by anyone or it is just available to certain users?

If it is available for everyone to see, can you please teach me how?

[1713549511]

1713549511

[Findingnemo]

It is not available for everyone,some people were making efforts to extract these datas by their own maybe they were techies.And also Bpip helping for them in finding those datas.

[Peacemaker1994]

Well I don't know why it's so important to you. 

https://bitcointalk.org/index.php?topic=3082289.0

You can go through this post and understand how more about merit and smerit distribution. But mind you if you out just for merit you might loose focus

[Piggy]

You can find the offical raw data in here: https://bitcointalk.org/merit.txt.xz but it doesn't contain everything since the begin of the merit system. In any case you can get an idea.

[Upgrade00]

This is definitely the right board as it is an inquiry about the forum.

BPIP contains complete information about the forum and statistics as well.
And you can use it to track a users' merit history well beyond the 120 days limit.
Link: https://bpip.org

A manual search of the forum MEMBERS can be used to get more information.

[cryptovigi]

If you like stats check this thread:

https://bitcointalk.org/index.php?topic=4428616.msg39521119#msg39521119

[DdmrDdmr]

<…>

@Piggy has shown you the link to the core file to it all, although the referenced file (released on fridays) contains 120 days of TXs, and to go beyond you need to use files such as those created by @LoyceV.

Merit data is available in different formats and through different external tools (I’m assuming you want to look beyond the 120 days of sMerit TXs that can be seen un the user profiles). The choice of which one depends on which one suits your needs best (i.e. consult somebody’s merit history, get a general idea of how merit is being distributed and where, etc.).

There are tools around such as the following:
BPIP project (Vod)
Full merit transaction history for any user - now with GRAPHS! (LoyceV)
Swiss army knife Tool to query Merit data online - Full History (Piggy)
Tool to run online SQL queries over Full Merit Data (Piggy)
Merit Dashboard (DdmrDdmr)

Take your pick…

[Initscri]

Some users may also be using custom scraping tools/bots to scrape specific content from BitcoinTalk.

There's many guides online for how to build scrapers:
For example:
- https://codeburst.io/scraper-b82146396249
- https://github.com/FriendsOfPHP/Goutte
etc

Just be certain if you do build one to add code to prevent a situation where you're accidentally producing a minor DOS on the site. Put delays in the code, etc.

This is more or less how I believe BPIP works (correct me if I'm wrong through, they may have access to further data, or may be using datasets such as the one Piggy mentioned)