Bitcointalk Merit Dashboard

[DdmrDdmr]

<...>

Yes, I remember the recent discussion on the matter in your quote. The Dashboard lets you see Merit Distribution by subsection (i.e. local language or second level within Bitcoin, Alt. Cryptocurrencies, Economy and Other). Going a step further and doing it by topic is really too much overtime for me right now (I’m doing all this in my spare time aside from a regular fulltime job).

You would need to:
1. Study all the thread titles/OPs that you want to take into consideration (you could assume that all posts within the thread are related by content to the OP to simplify). The title does not always say it all, so you need to go into the OP itself. Then you need to classify the thread and related to 1 (or n) topic/s.

2. Perform the above for all threads created or bumped in a certain timeframe (for example, since the Merit System kick-off, or last month). There are thousands of threads to consider, although if the focus were to be merit related, you could cut down quite a bit by performing the classification only on those merited threads.

3. And then cross the above with merits per post/thread.

It would certainly be interesting, but very long to perform for me right now if done properly.


A quick approximation is to look through the most merited threads (aggregate of posts in the thread) on the Merit Dashboard (see tab called “Rankings”) filter the Date Range (i.e. August) and/or subsection, and perform a tentative approximation from the list displayed. For example, if you filter Meta for the Month of August we get:

Seeking Reporter Badge Images  -> 100 Merits in August.
Community generates suggestions to … -> 97.
Merit? This got to be a joke -> 94 Merits in August.
Analysis – Correlation Matrix -> 77 Merits in August.
How recognized are you in the bitcointalk world? -> 69 Merits in August.
[guide] Bitcointalk account security -> 62 Merits in August.
<etc.>

You could create a tentative stat for Meta and other sections by manually classifying each thread (assign each thread one main core topic), and then adding all the Merit by Topic.  Doing so for Meta may not be difficult (Merit, Account, Ideas, Plagiarism, Reporting, etc.), but trying to derive a proper topic on Bitcoin discussion or Ann threads may be rather painful, especially if you are not familiar with the content on the subsection.

Edit: I do encourage you nevertheless to give it a go, trying the dashboard approach I mentioned. Try it out on some subsections with which you are familiar, so that the context is known to you. Derived topics should be no more than 15 or 20 max. and can be categorized into groups to reduce the number. If you give it a go and find something interesting, open an OP to show us your findings.
The only fine-print needed is how topic assumption is derived by from the OP on each thread (not all merits in practice are necessary related to the OP of the thread, but it’s a workable assumption).
As a last note, it’s probably best in this case to limit data to recent months, in order to avoid heavy topic bias by early heavy meriting.

[TheBeardedBaby]

How can I select all in the list and is it possible to export the results in some format?
This is in Received merit, I see that I can arrange them only when I select them.


BTW after trying to sort them I got an error:

[vphasitha01]

How can I select all in the list and is it possible to export the results in some format?

I don't think there is a way to export the results except downloading the pdf of the selected data. But send a Prayer Message to "DdmrDdmr" and he will surely send you the data in a google spreadsheet

BTW after trying to sort them I got an error:

There is an automatic trigger for timeout of a session, unfortunately, I didn't remember the period.

Seems like @iasenko might going to changed the path from Spam Buster to Merit Analyzer ( Just kidding mate... )

[DdmrDdmr]

<...>

Yes, Tableau is a good product, but it still has some issues that have been requested by its users some time ago, but that still have not seen the light. Amongst them are two of the issues you are addressing:

-   Tableau Public does not allow export to flat files or excel files (or even to copy the data to the clipboard) unless the Dashboard is “Open”. I can open the Dashboard to this feature, but the drawback is that then not only is the data exportable (which I would really like), but so is the Dashboard itself. So basically, as absurd as it sounds, in order to allow a minimal export I need to allow for the whole source code Dashboard to be exportable too (which I’m not too keen on really).

-> If you need a specific dataset, I can share that for you on a google spreadsheet as @vphasitha01 mentioned.

-   Tableau is not great at sorting rolled-up numbers. Basically, for some internal design decision, if there is a dimension (the username) and then a rolled-up number (aggregate of merits), the latter is only ordered as a subset of the dimension entry. That is, it orders by name, and then by merits within the name (if there were more than one data entry). So when trying to order by Merits, it does not have the desired effect.
This has also been requested as a change to Tableau, but it’s still a no go.
I could put Merits as the first column, and then it would sort by the value rather than by the name, but I thought name would be a better choice.

-   Tableau Public (the free hosted open to public version I’m using) has a default timeout somewhere in the line of 4 minutes, so an inactivity period of around 4 minutes requires you to reapply your selections. Sometimes, sorting or such can cause that kind of internal error which should be resolved by refreshing the page again (and reapplying the filters). The default timeout cannot be altered by me.

Let me know if you need a specific data selection.

I'll update de Dashboard today with the most recent update.

P.D. The Merit Network on the other hand allows you to export all the contained database (File->Download (csv format)) , so perhaps that could help, but the data is rather raw there (i.e. at TX level) .
Edit: On the Merit Network , you can also copy data from the Tabs via the clipboard (i.e. go to the Summary Received tab, filter there, sort and export).

[Alone055]

Nice work!

I can see in the "Weekly Merit Evolution" section that the amount sent every week is gradually decreasing. Is that because we are getting low on quality posts or that just indicates that Merit abusers are running out of 'em as the time is passing? As theymos said, that they would eventually run out of sMerits if they try to cheat the system and keep sending them between each other.

I think you’re confusing Satoshi with theymos..

Oops! I'm not confused but I just unknowingly wrote "satoshi" instead of "theymos". God, he never gets out of my mind! 
I edited the post and changed it to theymos.

[DdmrDdmr]

<…>

I think you’re confusing Satoshi with theymos.. or you know something than many don’t ….

The reasons for a decreasing merit weekly aggregate are multiple I’d say, and not related to there being less quality posts around:

-   People have less merit from the initial airdrop, and it may be pretty low now in the big picture for those that do participate in the merit awarding procedures (many do not as we’ve commented on posts around Meta on occasions).

-   People themselves generally never really gained large amounts of merits for their posts, so the half derived dispensable sMerit amount is not too large either. That is also related to the fact that the general amount of sMerit per post and per Tx is mainly in the very low end of the scale (1 or 2 merits).

-   Merit Sources are increasingly more important to maintain the Merit System  running as a result of the above. If a few of them take a break or reduce the time they spend awarding merit, the numbers drop.

-   Many topics a reiterative. The pioneers got merit, but reiterations by other people seldom do (i.e infographics, merit abuse catching, tip for newbies, etc.).

Merit abusers will suffer the same issues (even more so). What I’d say is their golden key for them are those merit awarding “dormant” accounts, that after a time start to award their initially airdropped sMerit, being some of these accounts either hacked accounts or that decided to trade. The proportion of abused merit is probably small though in the big picture, but there is a potential usage of sMerit by some those accounts that have not merited before, which could be set to use wrongly.

[vphasitha01]

<…>

I think you’re confusing Satoshi with theymos.. or you know something than many don’t ….

Hell yeah he is confusing theymos with Satoshi

The reasons for a decreasing merit weekly aggregate are multiple I’d say, and not related to there being less quality posts around:

-   People themselves generally never really gained large amounts of merits for their posts, so the half derived dispensable sMerit amount is not too large either. That is also related to the fact that the general amount of sMerit per post and per Tx is mainly in the very low end of the scale (1 or 2 merits).

That's the main reason as per my observations. I would think if we going to encourage the posters who really contribute to the forum discussions should be awarded with more than 1 or 2 Merits in order to maintain the flow of Merits stable way. Because I don't think Merits should be a barrier for users (not the spammers) whose really deserved to be ranked up. Merits are introduced by theymos to encourage the good posters while discouraging the shitposters and spammers.

-   Merit Sources are increasingly more important to maintain the Merit System  running as a result of the above. If a few of them take a break or reduce the time they spend awarding merit, the numbers drop.

This is the second most important factor as you explained. I think there is not enough Merit sources at the present. theymos may consider this issue ASAP and nominate few more sources to balance the flow because there are good members who still willing to sacrifice their time for hunting down posters and award Merits for their posts.

Another thing that I have seen is some of the Merit sources relayed on there personal openion or principles when awarding their sMerits. It's ok if they awarding their earned Merits to some other members, but not the source Merits. I don't know is there any way to distinguish those two types of Merits for Merit sources. Merit sources shouldn't clouded their judgements with their personal openions or even their field of expertise and just look only the quality of the post. I want to know is there any way to find out unused sMerits of users? Because numbers matter.

[DdmrDdmr]

<...> I want to know is there any way to find out unused sMerits of users? Because numbers matter.<...>

Not really on an individual basis, since the exact airdropped sMerit per user has not been disclosed, and it depended on the rank + activity of each user prior to Merit System Kick-off (see further ahead).
You could know it though exactly for users that were Jr. Members, Newbies or Brand New at the time, since their initial airdrop was of 0 sMerits, and thus the calculation is simple (Received sMerit/2 – Sent sMerit). For all other Ranks you can’t know the exact minimum amount.

You can have an estimate overall as I have calculated on some occasions and now update here:

Theymos stated that the initial distributed total sMerits were around 600k (see (Re: Merit & new rank requirements).
Merit Sources generate now a maximum of 19.500 sMerits per 30 days. Of course, these don’t empty their monthly allowance, and also the quantity has increased over time. Let’s say as an hypothesis that the average distribution of Merit Sources has been of 15.000 sMerits per month.

With the above, a safe scenario would be:

+ 600.000 sMerits -> Initial Airdrop
+ 52.500 sMerits -> Merit Sources (15k/month average *7 full months as a safe hypothesis  = 105.000 -> halved -> 52,5K)
- 216.136 Total awarded sMerits (up until this morning)
------------------------------------------------------------
436.364 minimum available sMerits system wide, and that’s not even considering the system generated halved sMerits by means of those awarded sMerits from user to user.

So if the 600K is correct, which I have no doubt, there is almost double the sMerit stocked up in comparison to the total aggregate awarded sMerit. That is what I was on about before, since most of it is from the Initial Airdrop.

[DdmrDdmr]

Edit 31/08/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit: 216.136
Total Txs:       98.346
From Users:   16.462
To Users:       19.082
minDate:       2018-01-24 22:12:21  
maxDate:      2018-08-31 02:41:54

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard.  

The sMerit awarded on the last entire week (20/08/2018 ..26/08/2018) is nearly exactly the same as the previous week (only a 4 merit difference).

Also congratulations to krogothmanhattan for ranking-up this week to a Hero member of Bitcointalk! (he had the merits a long time ago, but was lacking activity).

[TheBeardedBaby]

<...>

-> If you need a specific dataset, I can share that for you on a google spreadsheet as @vphasitha01 mentioned.

Thanks, I was thinking to make a weekly list of the top merit receivers.

[DdmrDdmr]

<...>Thanks, I was thinking to make a weekly list of the top merit receivers.<…>

Ok, there are two options without having to create manual weekly lists on my behalf. Let’s see if they are enough:

Option 1- Merit Dashboard:

On the Merit Dashboard, go to the Tab called Rankings and adjust the Date filter to the desired date range. The easiest way is to click on the date shown on the left of the Date Filter slider and chose the date from the calendar. You should also change the time by double clicking the time, and set it to 00:00:00 (date can be entered in the same way).
In the image, I’ve set it to the 21/08/2018 00:00:00. The Merited User Ranking will show you the top merited for the week you selected.

The downside as we’ve talked about, is that you cannot export the list and will need to click on each user and chose "Go to Merited User Profile" to jump to his profile.


Option 2 - Merit Network

Use the  User Merit Network (also launchable from within the Dashboard). Go to the Tab called "Total Received" and add a Filter by converted_date. In the example, I’ve filtered by 21-08-2018 onwards (click on the filter cell to select date from calendar). No need to set the upper limit if you want data to include all TXs from the selected date onwards. Also no need to set time here.
The shown list is simple. I’ve just modified it to include the ToUserId which I did not have there before. "Count" is the number of TXs and "SUM amount" the number of received sMerits (which is sortable).

What may be interesting here is that you can export the list in a sense by selecting the items from the list, and copying them (clipboard) to Excel, and then build the URL there if needed based on the UserId. Copying takes a little getting used to, buy I’ve managed to do it correctly a few times in a row with no issue.

[TheBeardedBaby]

@DdmrDdmr,
Great, I'll check both options next week, I'll see what is the best.
Thanks for the help:)

[DdmrDdmr]

@DdmrDdmr,
Great, I'll check both options next week, I'll see what is the best.
Thanks for the help:)

Ok great. Just to mention that I've just added the user profile URL to "Option 2" to make it easier.

[DdmrDdmr]

Edit 07/09/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit:  219.617
Total Txs:      100.125
From Users:   16.587
To Users:       19.268
minDate:       2018-01-24 22:12:21   
maxDate:      2018-09-07 02:47:55

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard. 

The sMerit awarded on the last entire week (27/08/2018 ..02/09/2018) is down 15,33% the previous week!. This is a new minimun record, with just 3.186 distributed this week (and barely 11,41% of what was distributed during the first week of the Merit System).

[DdmrDdmr]

Edit 14/09/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit:  223.051
Total Txs:      101.844
From Users:   16.706
To Users:       19.453
minDate:       2018-01-24 22:12:21   
maxDate:      2018-09-14 01:55:43

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard. 

The sMerit awarded on the last entire week (03/09/2018 ..09/09/2018) is up 10,99% from the previous week, recovering a bit from the previous week's 15,33% drop.

[DdmrDdmr]

Edit 21/09/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit:  231.540
Total Txs:      106.752
From Users:   17.290
To Users:       21.235
minDate:       2018-01-24 22:12:21   
maxDate:     2018-09-21 02:47:57

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard. 

Specifics on this week's merit variation vs last week can be seen here: sMerit participants – A quick follow-up after the changes – Good so far.

Copper Member ranks have been converted to regular ranks.

I've still got a fews changes pending to do over this weekend, such us include Newbie -> Jr. Member in the ranked-up tab.

[Jet Cash]

Please could you split the "newbies" into true newbies and the spambies - the deranked ones. There seems to have been a flurry of activity and whingeing by the namby spambies, and it might be interesting to see who has abused their merit awarding ability by encouraging them.

[DdmrDdmr]

Please could you split the "newbies" into true newbies and the spambies - the deranked ones. There seems to have been a flurry of activity and whingeing by the namby spambies, and it might be interesting to see who has abused their merit awarding ability by encouraging them.

I only work on merited profiles to limit data scraping. So far I've come up with the following:

Code:

Current Rank          Probable Initial Rank         nUsers
Jr. Member            Hero Member                   1
Jr. Member            Member                        9
Jr. Member            New Era Newbie <=60           296
Jr. Member            New Era Newbie <=120          458
Jr. Member            New Era Newbie <=240          571
Jr. Member            New Era Newbie <=480          138
Jr. Member            Old Era Newbie <=60           1718
Jr. Member            Old Era Newbie <=120          1585
Jr. Member            Old Era Newbie <=240          1482
Jr. Member            Old Era Newbie <=480          303

Where:

- New Era: first merit gained >= 17/09/2018
- Old Era: first merit gained <   17/09/2018
- The "<=" number is the current activity (as of last Friday).

So for example, there are 571 Jr. Members, with >120..<=240 activity, that have gained their first sMerit since last monday.

Also:

Current Rank         Probable Initial Rank     nUsers
Newbie                New Era Newbie <30   145
Newbie                Old Era Newbie <30   2.092

Notes:
- There is 1 de-ranked Hero and 9 Members to Jr. Member (also shown on the Merit Dashboard) -> Deleted posts.
- An additional batch has jumped strait to Member rank.
- We cannot tell apart which activity is prior to change of rules, and which is after the change (so I'm going for full activity, will change to post count which is more meaningful than activity).
- There's quite a lot of Old Era Jr. Members that have not logged-in for a couple of months. Perhaps they gave up ...
- Still got to work on this to get it on to the Merit Dashboard -> Ranked-Up Tab (tomorrow or Monday I figure).

[DdmrDdmr]

Edit 28/09/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit:  236.823
Total Txs:      109.771
From Users:   17.578
To Users:       21.971
minDate:       2018-01-24 22:12:21   
maxDate:      2018-09-28 02:42:56

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard. 

Last of the V8s has up-ranked to Legendary.

[DdmrDdmr]

Edit 05/10/2018:

Updated Dashboard to reflect the most recent sMerit available data:

Total sMerit:  241.323
Total Txs:      112.275
From Users:    17.784
To Users:        22.543
minDate:        2018-01-24 22:12:21   
maxDate:       2018-10-05 02:51:53

The dashboard gives you access to anyone’s complete merit history in the TX tab, surpassing the 120 day limit.
Link:  BitcoinTalk Merit Dashboard. 

Aggregate awarded sMerit is now 4.508, back to June 2018 levels, being half of what it was on the week after the change of rules.