Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 05:42:34 AM Last edit: July 08, 2018, 08:13:35 AM by Piggy Merited by TMAN (10), Foxpup (6), suchmoon (5), qwk (5), malevolent (5), dbshck (5), LoyceV (4), DdmrDdmr (1), bill gator (1), bobq (1), criz2fer (1), theyoungmillionaire (1) |
|
Hello, I collected the data regarding all the Merits given since it was introduced (updated till 6th of April) an put it into a graph which can be navigated, this can give a pretty good idea how and where the Merits are getting distributed across the whole forum. Starting from the "Bitcoin Forum" section, you can drill down till the thread where the Merits have been given and at the same time see the number of merit collected by each section/subsection of the forum. It took some time to put it together, but i think it was worth ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Enjoy! https://albertoit.github.io/Visualization/index.htmlBonus pic: How to use it:- Mouse over on any parth of the graph will show the name of the section and the Merits which has been given under that section. - Click on each section to zoom into that section, each subsection/threads will be shown directly in the immediate outer ring strating from the center. - Click on the center to go back to the previous section.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3416
Merit: 17272
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 06:26:49 AM |
|
Well done! It clearly shows the Local sections (often said to not receive merit) do receive a lot of merit. What surprises me, is that the Altcoin boards receive almost twice as much as the Bitcoin boards! It took some time to put it together, but i think it was worth ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Would this be something you can keep up to date once in a while? Update: I collected the data regarding all the Merits given since it was introduced (updated till 6th of April) Only after I've merited your post, I realized I disagree to your data: Total: 125281 How did you get to 52,778 in total?
|
|
|
|
edwardceng
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 266
Merit: 50
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 06:59:44 AM |
|
Starting from the "Bitcoin Forum" section, you can drill down till the thread where the Merits have been given and at the same time see the number of merit collected by each section/subsection of the forum.
what methods are used to obtain data from each local board? do you calculate the Merit manually from each local moderator? I'm curious because I have ever counted Merits on my local board using the manual method with the sMerit count of each moderator.
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 07:53:50 AM Last edit: April 13, 2018, 09:31:34 AM by Piggy |
|
Well done! It clearly shows the Local sections (often said to not receive merit) do receive a lot of merit. What surprises me, is that the Altcoin boards receive almost twice as much as the Bitcoin boards! It took some time to put it together, but i think it was worth ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Would this be something you can keep up to date once in a while? Update: I collected the data regarding all the Merits given since it was introduced (updated till 6th of April) Only after I've merited your post, I realized I disagree to your data: Total: 125281 How did you get to 52,778 in total? Many of the threads/comments that received merits were actually deleted or not accessible so some of that data is not actually appearing in the graph. What you see there is there is what you can actually verify/see at the moment in the forum.The data was collected using the raw file https://bitcointalk.org/merit.txt.xzIm planning to keep it updated.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3416
Merit: 17272
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 10:10:48 AM |
|
Many of the threads/comments that received merits were actually deleted or not accessible so some of that data is not actually appearing in the graph. That does make sense, but it would mean almost 58% of all merits were sent to posts that have now been deleted. I've just (manually) checked a sample of posts with high amounts of merit from theymos' data in human readable format (17 MB), and from that I'd say less than 10% went to deleted posts. If an entire thread is deleted, obviously you can't see the board anymore. But did you included the board for deleted posts where the thread still exists? Im planning to keep it updated. Very nice!
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 10:36:44 AM |
|
Many of the threads/comments that received merits were actually deleted or not accessible so some of that data is not actually appearing in the graph. That does make sense, but it would mean almost 58% of all merits were sent to posts that have now been deleted. I've just (manually) checked a sample of posts with high amounts of merit from theymos' data in human readable format (17 MB), and from that I'd say less than 10% went to deleted posts. If an entire thread is deleted, obviously you can't see the board anymore. But did you included the board for deleted posts where the thread still exists? Im planning to keep it updated. Very nice! The rule i used was to discard comments from the counting regardless if the thread was still there. It is strange indeed that so many comments with merits were actually removed, but i had specifically put a flag for it so im pretty confident that what is missing is for that reason. I will pick few cases and check actually by hand if there is more to it that i missed. In any case i believe the graph has enought data to give a good representation of where the merits are going.
|
|
|
|
DdmrDdmr
Legendary
Offline
Activity: 2422
Merit: 10978
There are lies, damned lies and statistics. MTwain
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 10:38:17 AM |
|
Hello,
I collected the data regarding all the Merits given since it was introduced (updated till 6th of April) an put it into a graph which can be navigated, this can give a pretty good idea how and where the Merits are getting distributed across the whole forum. <...>
I think there's something wrong with the data: Up until the 23/03/2018, my calculations were as follows (much in line with with Zentdex's "Where the merit pours" distribution). This is what I calculated: ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2F6CTC7ec.jpg&t=665&c=hVZV_JX9mApEDg) In your Data for example, the Local Board adds up to 19.003, which is half of my calculation (and I calculated it 2 weeks before so there should be more, not less in your graph...).
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:14:26 AM |
|
From what i can see we are talking abou threads missing as if you have a link to a comment missing the thread still shows up. Is when the thread is missing that you get nothing.
I had previously double checked the number i see in the graph for a given thread with what i see going in a thread manually and they seems to be the same.
I should run it again and check the total numbers against the threads i found to be missing and see if it match up with the total merits(125281 in the raw file i used).
|
|
|
|
athanz88
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:16:59 AM |
|
Wow, this is next level. I made data about merit too before for my own topic but this is way out of my level, it is an awesome job. I hope you can continue to update this merits data.
Regarding to the difference between the number of merit from other thread, i guess there must be some archive to the posts that got meritted by people, so we can know the quality of the posts and increasing the accuracy of data gathering like this piggy does, and it also helps to spot merit cheaters.
|
|
|
|
DdmrDdmr
Legendary
Offline
Activity: 2422
Merit: 10978
There are lies, damned lies and statistics. MTwain
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:28:10 AM |
|
From what i can see we are talking abou threads missing as if you have a link to a comment missing the thread still shows up. Is when the thread is missing that you get nothing.
I had previously double checked the number i see in the graph for a given thread with what i see going in a thread manually and they seems to be the same.
I should run it again and check the total numbers against the threads i found to be missing and see if it match up with the total merits(125281 in the raw file i used).
Deleted posts with a Merit Tx involved are in the 4% to 6% area (of sMerit with no matching thread), depending on the dataset used in the past. That would give you a max around 7.500 sMerits not asociable to a forum section with the current dataset, but the remaining should be accounted for (unless there's been a massive deletion of threads these past two weeks, which I doubt it). What is your count of merit awarded to deleted threads ?
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3416
Merit: 17272
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:38:13 AM |
|
The rule i used was to discard comments from the counting regardless if the thread was still there. Did you pause for one second between scraping posts? If you scrape too fast, you'll get empty pages, which could explain you missed about half the merited posts.
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:45:01 AM |
|
From what i can see we are talking abou threads missing as if you have a link to a comment missing the thread still shows up. Is when the thread is missing that you get nothing.
I had previously double checked the number i see in the graph for a given thread with what i see going in a thread manually and they seems to be the same.
I should run it again and check the total numbers against the threads i found to be missing and see if it match up with the total merits(125281 in the raw file i used).
Deleted posts with a Merit Tx involved are in the 4% to 6% area (of sMerit with no matching thread), depending on the dataset used in the past. That would give you a max around 7.500 sMerits not asociable to a forum section with the current dataset, but the remaining should be accounted for (unless there's been a massive deletion of threads these past two weeks, which I doubt it). What is your count of merit awarded to deleted threads ? My bad,I actually found the problem every amount of merits given for a comment is just counted as 1, instead of the real amount of merits given. ![Embarrassed](https://bitcointalk.org/Smileys/default/embarrassed.gif) Ill be updating the graph soon
|
|
|
|
DdmrDdmr
Legendary
Offline
Activity: 2422
Merit: 10978
There are lies, damned lies and statistics. MTwain
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 13, 2018, 11:48:27 AM |
|
My bad,I actually found the problem every amount of merits given for a comment is just counted as 1, instead of the real amount of merits given. ![Embarrassed](https://bitcointalk.org/Smileys/default/embarrassed.gif) Ill be updating the graph soon Good. Sum for Count and ready to roll...
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 14, 2018, 10:28:53 AM |
|
Updated till 13th of April, now the total is correct Have fun ![Wink](https://bitcointalk.org/Smileys/default/wink.gif)
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 23, 2018, 06:53:41 AM |
|
Updated to 20th of April Total amount of merits so far was 135823 what is missing from the graph are merits belonging to deleted threads!
Total amount of merits browsable in the graph is 129923
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 27, 2018, 05:10:03 PM |
|
Updated to 27th of April Total amount of merits so far was 140.662 what is missing from the graph are merits belonging to deleted threads!
Total amount of merits browsable in the graph is 134.713
|
|
|
|
bobq
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 27, 2018, 06:53:27 PM |
|
Updated to 27th of April Total amount of merits so far was 140.662 what is missing from the graph are merits belonging to deleted threads!
Total amount of merits browsable in the graph is 134.713 Congratulations for having done such a good job. But in the end I didn't understand which is the percentage of merited posts which have been deleted (most likely because of the deletion of the entire thread). In this case, by the way, does the person keep the merit of a deleted post?
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 27, 2018, 07:02:39 PM |
|
Updated to 27th of April Total amount of merits so far was 140.662 what is missing from the graph are merits belonging to deleted threads!
Total amount of merits browsable in the graph is 134.713 Congratulations for having done such a good job. But in the end I didn't understand which is the percentage of merited posts which have been deleted (most likely because of the deletion of the entire thread). In this case, by the way, does the person keep the merit of a deleted post? Thank you. Deleted threads containing merits account for ~6k merits, about ~4.2% of the total. The people that got the merits in those thread get to keep them so it doesn't affect anybody, is just not possible for us to see where and which post was merited.
|
|
|
|
Piggy (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
April 30, 2018, 11:07:21 AM |
|
Just added the % and a new category containing the merits belonging to deleted threads, so you can have the all figure ![](https://ip.bitcointalk.org/?u=https%3A%2F%2Fi.imgur.com%2F22sFHqq.png&t=665&c=F3LQuaaYiMmcTg)
|
|
|
|
kokoska
Member
![*](https://bitcointalk.org/Themes/custom1/images/star.gif)
Offline
Activity: 714
Merit: 17
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 03, 2018, 07:59:56 PM |
|
That's cool! ![Shocked](https://bitcointalk.org/Smileys/default/shocked.gif) But where is the English distribution sMerit? I see only the location of the whole world, but there is no basic English distribution sMerit.
|
|
|
|
|