Tracking the difference of merit circulations with Default Trust Changes

[tranthidung]

TRACKING THE DIFFERENCE OF MERIT CIRCULATIONS WITH DEFAULT TRUST CHANGES

ON 09/01/2019 (IN MEANS AND MEDIANS)

- Update will be published in the newest post (weekly update), not in the OP;

Colors:
- Green: Increasing change;
- Red: Decreasing change.


ABSTRACT
Two weeks later:
(1) There is no outliers within two weeks after Default Trust changes. Only one day (09/01/2019) comes closer the the outlier-identifying threshold.
(2) Median and mean of the intraday merits twoo weeks after Default Trust Changes are higher than the figures of before period at 31.8% and 23.2%, respectively.
- Median: + 37.1%
- Mean: + 24.2%


Objectives
(1) To measure the percentage difference changes of median and mean of intraday merits between the before and after periods (the cut-off is 09/01/2019);
(2) To observe the the difference changes over time;
(3) To identify when the impacts of Default Trust Changes tails off.


As you all known that theymos decided to make DefaultTrust changes (still in pilot phase, not official).
The change of Default Trust has obviously resulted in dramatical increasing wave of merit circulations in the forum.
The following time series plot clearly shows it:

- For dataset of intraday merits within 2018, please get it there
- For dataset of the year 2019, please get it there


1) Outliers identified
- None or only one day (on 09/01/2019)

In my another topic, potential outliers for truncated dataset identified (from 19/2/2018 to 21/1/2019):

Potential outliers are days that have intraday merits beyond 124 or 1164.

For all days (till now) in the year 2019, there is no day beyond the outlier-identifying threshold.
Only one day should be taken into consideration, it is 09/01/2019 with 1161. The day has only three merits lower than the threshold, so we can call it as an outlier or not (it depends on your viewpoint)!

2) Statistics:
Statistics will be present in means +/- standard deviations; medians (interquartile ranges); mins and maxs.

You can imagine what I will do with the following dummy table to track changes.

Raw statistics are here:

Code:

. tabstat before090119 wkslater_2, s(n mean sd p50 p25 p75 min max) format(%9.1f) c(s)

    variable |         N      mean        sd       p50       p25       p75       min       max
-------------+--------------------------------------------------------------------------------
before090119 |     324.0     677.0     263.0     616.5     510.5     766.5     312.0    2463.0
  wkslater_2 |      14.0     840.4     191.4     845.5     658.0     987.0     611.0    1161.0

3) Changes over weeks
Now, let's see changes of means and medians of intraday merits over weeks after the Default Trust changes.
I compared statistics (means, medians) between the before period (for days from 19/02/2018) and different after period (from 09/01/2019 till the end of each later week).

For instance, wkslater_2 presents for the statistics of two weeks after the DefaultTrust Changes.

After two weeks, the median and mean of two-week later period are higher than figures of the before period at + 37.1% and + 24.2%, respectively.


Notes:
Update tables to 6 weeks later.

[S_Therapist]

What is the use of this statistics? Sorry if I have missed something.

[KingZee]

What is the use of this statistics? Sorry if I have missed something.

I kinda agree. Merit and trust are probably the least correlated things on the forums.

1. When you merit someone, most people do it because of what they read, not because of the person's trust. This is especially true because Trust is not displayed in the majority of the forums.

2. Trust already existed, the only change you would see on user trust ratings would be minimal compared to the first iterations. So if people merited trusted people more, or vice versa, (which is assuming there is a correlation) you're already not going to see anything change, because only a few people were removed and others added to DT.

[tranthidung]

What is the use of this statistics? Sorry if I have missed something.

Objective:
- To see how long significant changes of rules (Trust, Merit, Ranking, for example) will affect on the merit circulations in the forum.

As you can see, the changes in ranking aims at Junior Members led to demotion, but its effects on merit circulation was not too long.
Spikes appeared on the time series plot in September, but several days later, it blurred.

Edited:
I updated the OP with objectives, you can get them there:

Objectives
(1) To measure the percentage difference changes of median and mean of intraday merits between the before and after periods (the cut-off is 09/01/2019);
(2) To observe the the difference changes over time;
(3) To identify when the impacts of Default Trust Changes tails off.

[madnessteat]

Good work. I think in a week or two the hype around the system trust will be lost and will decrease the circulation of merit.

[Bitcoin Smith]

How you guys pulling all those data from forum? Is there any tools used here or its been done manually.

[tranthidung]

You can get my converted dataset there: Time Series Analysis on Distributed Merits in the forum (daily, weekly, monthly)

How you guys pulling all those data from forum? Is there any tools used here or its been done manually.

I used data dumps from LoyceV, so you can also get datasets there:
LoyceV's Merit data analysis (full data since Jan. 24, 2018; not just 120 days)
Tool to run online SQL queries over Full Merit Data (by Piggy) is a helpful topic.

[suchmoon]

~3 weeks ago theymos changed merit source allocations and added new sources. The change also reset all sources to their full capacity. I think that - and not the trust system changes - was the primary reason for this merit activity increase.

https://bitcointalk.org/index.php?topic=5093301.msg49073989#msg49073989

[tranthidung]

I known about both those changes, additional 3 new merit sources, and re-allocation of merits for active merit sources.
No one known that whether active sources really use more sMerits than before after the reallocation.

~3 weeks ago theymos changed merit source allocations and added new sources. The change also reset all sources to their full capacity. I think that - and not the trust system changes - was the primary reason for this merit activity increase.

https://bitcointalk.org/index.php?topic=5093301.msg49073989#msg49073989

One guy collected statistics on total merits circulated in some topics on 13th Jan. 2019, only four days after the Default Trust changes started.
I am not sure how far it moved after that day, but let's see a quick glance at total merits on those topics on that day: 493 merits in total.
There were probably more merits in days later.
https://bitcointalk.org/index.php?topic=2818350.msg49205675#msg49205675
The Default Trust Changes is a very hot topic, and this one probably made great changes in the way merits circulation in the forum in short-term.

More details are there:
https://bitcointalk.org/index.php?topic=5069140.msg49476226#msg49476226

[suchmoon]

No one known that whether active sources really use more sMerits than before after the reallocation.

They most likely do because (1) the increase seems to have been targeted at those sources specifically who tend to spend most of their allocation and (2) he also reset the sources to their max capacity.

E.g. an active source who had a 100 sMerit allocation and had spent it all by Jan 5, got probably 150-200 brand new sMerits on that day.

Also check this: https://bitcointalk.org/index.php?topic=5093301.msg49065513#msg49065513

I noticed an increase within hours after the merit source change, days before DT changes.

[tranthidung]

I got your points!

They most likely do because (1) the increase seems to have been targeted at those sources specifically who tend to spend most of their allocation and (2) he also reset the sources to their max capacity.

E.g. an active source who had a 100 sMerit allocation and had spent it all by Jan 5, got probably 150-200 brand new sMerits on that day.

Also check this: https://bitcointalk.org/index.php?topic=5093301.msg49065513#msg49065513

I noticed an increase within hours after the merit source change, days before DT changes.

However, it is just a theory.
If those merit sources gave away their sMerits with higher rate than usual as well as at faster speed, they probably dried up their allocated sMerits.
Of course, those sMerits sent would be circulated around the forum and contributed to the rise of total Merit circulation.

As you can see in the quote below, the difference between medians of before and after period (the cut-off day is 09/01/2019) is nearly 32 percent.
Therefore, I think that it will become clearer if we know the percentage of increased sMerits for all merit sources after the adjustment from theymos.
If the figure is somewhere around 30%, it explain almost all what we want to see.

< ... >
After two weeks, the median and mean of two-week later period are higher than figures of the before period at 31.8% and 23.2%, respectively.

Edited:
I checked the merit sources stats:

Code:

There are 123 merit sources with a total merit generation of up to 20735 sMerit per 30 days

If the reason is from the reallocation of total sMerits allocated to merit sources, the figure before the adjustment should be:
20735/ 1.3 = 15950.
15950 should be the total sMerits per 30 days before the change.
Any ideas or real statistics with the figure 15950?

[suchmoon]

Any ideas or real statistics with the figure 15950?

For the 30 days prior to Jan 5 it was below 15000, according to Piggy's tool. For the 30 days prior to Jan 9 it was already above 16000, i.e. more than 1000 extra merits in just 4 days. That translates to >50% increase over 30-day average. Anecdotal evidence also suggests that active sources got increases between 30-100%.


Edited 2020-11-30 to fix a broken image


Edited 2020-11-30 to fix a broken image

[Harlot]

Based on what suchmoon has provided I think its safe to say that the refresh on the sMerits for merit sources is the reason why there was an increase in merits given during the DT changes, it just coincides with the changes in the DT system. Yes it must also be a factor being if you want toe be on the DT list you must have a minimum number of merits as well as your voters but I think the biggest factor is the sMerit refresh that theymos has done.

[tranthidung]

It is what I expected, very close to my estimation based on the difference of medians of intra-days merits between periods before and after.
nearly 30% increases of both sMerits generated for merit sources, and of  median of total intraweek sMerits sent in the last two weeks.
So, you maybe right that the rises of merit circulations last two weeks come from the re-allocation sMerits to merit sources and their sMerits-give-away activities.
Anyway, let's see the merit circulation next few weeks to confirm the assumption.

For the 30 days prior to Jan 9 it was already above 16000

What I really wanted is to track the merit circulation over time (before and after periods) to see how long the level of interesting the Default Trust Changes last.
As we known that sMerits are not scarce, quality posts are.
Consequently, I have a feeling that when a new serious changes (or ask for opinion raised by theymos). Forum users, especially 'Etlite' class tend to become more active, discuss more. They, obviously, have enough skills, knowledge, experiences, and ideas to make highly quality posts. As side effects, their posts often got massive Merits in short period after changes implemented.

< ... >

[tranthidung]

Update:
Three weeks later after Default Trust Changed.

- Median: + 15.9%
- Mean: + 15.5%

Statistics:
- Before:
Mean +/- sd: 677 +/- 263
Median (Interquartile range - IQR): 617 (511 - 767)
- Three weeks later (09/1/2019)
Mean +/- sd: 782 +/- 180
Median (IQR): 715 (643 - 880)

Code:

. tabstat before090119 wkslater_2 wkslater_3, s(n mean sd p50 p25 p75 min max) format(%9.1f) c(s)

    variable |         N      mean        sd       p50       p25       p75       min       max
-------------+--------------------------------------------------------------------------------
before090119 |     324.0     677.0     263.0     616.5     510.5     766.5     312.0    2463.0
  wkslater_2 |      14.0     840.4     191.4     845.5     658.0     987.0     611.0    1161.0
  wkslater_3 |      21.0     781.9     179.5     715.0     643.0     880.0     587.0    1161.0
----------------------------------------------------------------------------------------------

The previous assumption is likely wrong:

It is what I expected, very close to my estimation based on the difference of medians of intra-days merits between periods before and after.
nearly 30% increases of both sMerits generated for merit sources, and of  median of total intraweek sMerits sent in the last two weeks.
So, you maybe right that the rises of merit circulations last two weeks come from the re-allocation sMerits to merit sources and their sMerits-give-away activities.

Notes:
In the raw statistical results above,
- mean: mean;
- median: p50

[KingZee]

Based on what suchmoon has provided I think its safe to say that the refresh on the sMerits for merit sources is the reason why there was an increase in merits given during the DT changes, it just coincides with the changes in the DT system. Yes it must also be a factor being if you want toe be on the DT list you must have a minimum number of merits as well as your voters but I think the biggest factor is the sMerit refresh that theymos has done.

It's obvious. Imo this merit change isn't caused at all by trust system changes, those two events are completely unrelated. I don't know why OP is searching for this illusory correlation, but whatever.

[tranthidung]

It's obvious. Imo this merit change isn't caused at all by trust system changes, those two events are completely unrelated. I don't know why OP is searching for this illusory correlation, but whatever.

Effects have two types:
- Instant effects;
- Delayed effects.
Default Trust change is one of the most breaking changes in the forum over its long history.
I don't say that Trust and Merit system correlated each other.
I just observe the circulation of merits in the forum for following objectives:
- To see whether the Default Trust change affected the way forum users used their sMerits in both short term and long term.
- To see (if there are effects) how long the effects of Default Trust change's side effects on merit circulations last.

Additionally, any judgement need to have real data. You can simply say "Ah, Default Trust change has not correlated with Merit system and merit circulations. Let's end the discussion. I believe they are not correlated each other [without data]".

If you don't see the topic interesting, just put it into your Ignore list. 

[tranthidung]

Moreover, even when Default Trust Change has not affected the merit circulations within recent weeks, it might play its role as potential bias factor.
No one knows, but I think we should trace both the two for several weeks to see the correlation or bias impacts from Default Trust change on merit circulations.

[tranthidung]

3 weeks later update:

Basic statistics:

Percent of changes:

Sorry for the inconvenience here, because I just found that in the analysis for two weeks later, I made mistakes.
I forgot to drop un-used days, that are outside the time range of two weeks later of the Default Trust Change.
Updated both two tables with correct statistics already, but it is just small change.

Code:

* For Mean percent change two weeks later
. di (841-677)*100/677
24.22452
* For median percent change two weeks later
. di (846-617)*100/617
37.115073

[tranthidung]

4 weeks later update:
Median: +15.6%
Mean: +11.1%

ABSTRACT

(1) Both median and mean of four-week-later period are higher then the before period, with cut-off day is 09/01/2019, at 15.6% and 11.1%, respectively.
(2) 50 percent of days in the four weeks later period have intraday merits in the range from 614 to 879 (the interquartile range), whilst the figures of the before period are 511 to 767.



Box plots:
Outliers displayed with red circles

Outliers, non-displayed

Basic statistics:

Code:

. tabstat before090119 wkslater_2 wkslater_3 wkslater_4, s(n mean sd p50 p25 p75 min max) format(%9.1f) c(s)

    variable |         N      mean        sd       p50       p25       p75       min       max
-------------+--------------------------------------------------------------------------------
before090119 |     324.0     677.0     263.0     616.5     510.5     766.5     312.0    2463.0
  wkslater_2 |      14.0     840.4     191.4     845.5     658.0     987.0     611.0    1161.0
  wkslater_3 |      21.0     781.9     179.5     715.0     643.0     880.0     587.0    1161.0
  wkslater_4 |      28.0     752.0     183.4     713.0     613.5     879.0     450.0    1161.0
----------------------------------------------------------------------------------------------

Percent changes:

Code:

* For means
. di (752-677)*100/677
11.078287

* For medians
. di (713-617)*100/617
15.559157