[Experimental] Bitcointalk quality score

[*Ace*]

Hello everyone, I am opening this thread to present a small script that I created experimentally, hoping that everyone will like it.

This board often discusses the quality of posts, but it is a somewhat peculiar topic since there is no precise metric to determine with certainty whether a post is of pure quality or not.
Therefore, I have created this tool that indicates how much quality users have or lack, and how many quality posts they have created during their time on the forum.

The tool uses data from the Tryninja website (thanks as always) bitlist.co and from the forum itself.

The data analysed are posts, Merit and profile age (data from the forum) merit 120 days and posts 120 days (data from bitlist.co).

The formula used is as follows:

Code:

Q_hist = M_tot / max(P_tot, 1)

Q_120 = M_120 / P_120      if P_120 > 0
Q_120 = 0                   otherwise

Age_factor = min( log(Age_days + 1) / log(3650), 1 )

Activity_factor = min( P_120 / 50, 1 )

Score = 0.5 * Q_hist * Age_factor 
      + 0.3 * Q_120 * Activity_factor 
      + 0.2 * Q_hist

Quality_score = 0.6 * Q_hist + 0.4 * Q_120    if P_120 > 0
Quality_score = Q_hist                         otherwise

Code:

Variables
P_tot
Total number of posts written by the user.
M_tot
Total number of merits received.
P_120
Number of posts written in the last 120 days.
M_120
Number of merits received in the last 120 days.
Age_days
Account age in days, calculated from the registration date.

Core Ratios
Historical Quality Ratio
Represents long-term posting quality and reputation.
Q_hist = M_tot / max(P_tot, 1)
This ratio grows slowly, cannot be inflated quickly, and rewards consistent contribution over time.

Recent Quality Ratio (120 days)
Measures current posting quality.
Q_120 = (P_120 > 0) ? (M_120 / P_120) : 0
If the user is inactive in the last 120 days, recent quality is considered zero, avoiding artificial inflation through inactivity.

Stability and Activity Factors
Age Factor (logarithmic)
Controls how much account age influences the score.
Age_factor = min( log(Age_days + 1) / log(3650), 1 )
The effect increases smoothly and reaches its maximum after roughly 10 years.
This prevents very old accounts from having unlimited advantage.

Advanced Score (internal weighting)
This score combines history, recent behavior, age, and activity in a balanced way:
Score = 0.5 × Q_hist × Age_factor
0.3 × Q_120 × Activity_factor
0.2 × Q_hist
This value is mainly used to study behavior and tune the system.

Final Quality Score (displayed)
The main score shown to users is:
Quality_score = (P_120 > 0) ? (0.6 × Q_hist + 0.4 × Q_120) : Q_hist
Active users are evaluated using both historical and recent quality.
Inactive users retain their historical quality without penalty.
Activity Status
Activity is shown separately and does not affect quality.
If P_120 = 0 and Age_days > 180 → Dormant
Otherwise → Active
This allows labels such as “Dormant – Elite”, which correctly describes historically important but inactive accounts.

To be fair, I must state that I received assistance from le chat mistral (ai), to whom I had proposed my previous formula, which was (merit_120d * Merit)/post but had serious flaws. He assisted me in drafting this other formula, which appears to be more stable.

Here are some screenshots of the tool










The score still has ranges to be determined! I have put in some random ones so we can talk about them.

<0.20 Not good
<0.40 low quality
<0.60 Normal
<1.20 Good user
<2.00 High quality
>2.00 Elite

Thank you in advance to those who will participate in this discussion. I hope you find it useful.

[CONVOAI]

How can I use this tool? I want to check which criteria my profile falls under.

[EarnOnVictor]

First, I commend your effort in putting this together, kudos to you!

But I tell you, post, and only post can be the true decider of whether a user is a quality poster. Once merits are considered, you missed it.

As merit consideration could be true of the profile of some users, especially fillippone that you used as an example, and many others, it can't be true for many other accounts if you start checking the merits attracted with the actual quality of posts. The same could be detected (in the reverse) for some accounts that deserve more merits but are disenfranchised one way or another, but notwithstanding, have better quality posts than some with higher merits.

It's only a program that could automatically scan users' posts/profile itself "for quality" that can unbiasedly say about the quality and appropriately rate users.

[SamReomo]

I think there's something wrong in your tool. fillippone's post quality is only 1.27 and high quality, that alone says the tool might not be working properly. He's one of the most reputed members of the forum and when it comes to quality then such a user should get more than elite when it comes to quality of posts.

[*Ace*]

How can I use this tool? I want to check which criteria my profile falls under.

I haven't released the tool yet, it's still under construction and I've also specified that it's experimental. If it proves useful, I'll make it available to everyone.

First, I commend your effort in putting this together, kudos to you!

But I tell you, post, and only post can be the true decider of whether a user is a quality poster. Once merits are considered, you missed it.

As merit consideration could be true of the profile of some users, especially fillippone that you used as an example, and many others, it can't be true for many other accounts if you start checking the merits attracted with the actual quality of posts. The same could be detected (in the reverse) for some accounts that deserve more merits but are disenfranchised one way or another, but notwithstanding, have better quality posts than some with higher merits.

It's only a program that could automatically scan users' posts/profile itself "for quality" that can unbiasedly about the quality and appropriately rate users.

To do what you say requires enormous skills, skills that I unfortunately do not have!
The only thing I can do is analyse everyone's data and compile metrics based on that data! This does not mean that a user with a low score has necessarily produced poor-quality posts, but simply that they have not received the Merits to say that what they have written is of high quality.

The tool does not analyse the text of the post, but only the number of Merits and the number of posts to compile a score.

I think there's something wrong in your tool. fillippone's post quality is only 1.27 and high quality, that alone says the tool might not be working properly. He's one of the most reputed members of the forum and when it comes to quality then such a user should get more than elite when it comes to quality of posts.

As specified, the ranges are currently purely random, and this is one of the problems with the formula.
If a user has an average of more than one Merit per post, this already indicates that they must be an elite member of the forum.
Since it is not easy to say what range users should be labelled with, it would be helpful for me to receive advice from all of you.

[Doan9269]

As long as our criterial for judging for quality post will never be the same, so never expect that we use a particular criterial in judging for that, this is all about individual abilities and how we could see a post being a quality one or not, except there is a particular criterial that should be used in this, maybe merits in the last 120 days or something related, but i can bet you, it may cause for serious argument when some user receives ratings they believed does not suit their worth.

[babo]

Hello everyone, I am opening this thread to present a small script that I created experimentally, hoping that everyone will like it.

~~~~CUT~~~~

Thank you in advance to those who will participate in this discussion. I hope you find it useful.

It can be nice, but obviously you have to calibrate it differently
Not all of us get a lot of credit, especially those who write on the local board and little on the international one
It should not be seen as a parameter for judging, at this point in my opinion the @vod tool gives something better without judging too much and without blaming the quality of the posts
This way you can understand if a poster is a spammer or not

The idea is nice but in my opinion it needs to be calibrated better.

[letteredhub]

Snip

To do what you say requires enormous skills, skills that I unfortunately do not have!
The only thing I can do is analyse everyone's data and compile metrics based on that data! This does not mean that a user with a low score has necessarily produced poor-quality posts, but simply that they have not received the Merits to say that what they have written is of high quality.

The tool does not analyse the text of the post, but only the number of Merits and the number of posts to compile a score.

But that's exactly what it means if we're to follow your random score ranging which I would be quoting below

<0.20 Not good
<0.40 low quality
<0.60 Normal
<1.20 Good user
<2.00 High quality
>2.00 Elite

The fact still holds that in the midst of those profiles which would be considered high quality base on merits received there are those who by texts may not qualify to be within the range given.  The issue of merit and quality posts would continue to be an unfinished debate in the forum because sometimes, not always, merits awarded to certain posts ain't as a result of quality but preference which still can't be questioned.

[NotATether]

The only way to truly get a good quality score classifier is if you present users with a post and have them click a button whether it is spam or not. Then you can actually train a machine learning model on the post HTML and the responses, and use that to classify future posts.

A simple equation will NOT cut it.

[Nathrixxx]

This idea is not that bad for consideration, but where can we apply justice so as not to overrate and underrate some users? Because of all this, we are simply looking around for what has already been justified by the merit system. To get a quality poster, check on the merit history of that user.

But maybe we should rather look into whether the merit system still serves its intended purpose or has been hijacked by those abusing it, hence I don't think there's a need for something like this again when we have the merit system already in place, and a user activity can be seen through their profile last seen, and post history.

[LoyceV]

I'm too lazy to do the math myself now, but for assessment, can you give me the quality score for orenz0, Xyloo, btc-marty and myself?

[hd49728]

It is interesting with newbies but with legendary members, I mean true legends in the forum, not simly legendary members in rank like me, don't care about any quality score.

They know who they are, what they have done and contributed over many years in this forum, and they know that other legends in the forum community know them as well. They know that their topics and posts can be helpful and enlightful for newbies in the forum, that means they know they have already created valuable contributions in this forum.

No quality scores can assess their contributions exactly and even very low quality scores won't destroy what they have contributed here.

[Adbitco]

This is good concept bro, but somehow it is now looking like Bpip ranking system, where it would rank the most activity, Most Trusted, Most Recognized and the Most Merited users. Personally I have been thinking in which area can we actually apply this scores of a thing, because as we know managers can't just start using it, I mean sig campaign managers if they want to use it to justify a member before accepting them into their campaign, then merits criteria would be useless or not needed anymore, except for just ranking up.

[*Ace*]

I'm too lazy to do the math myself now, but for assessment, can you give me the quality score for orenz0, Xyloo, btc-marty and myself?

You're not lazy, but I already know where you're going with this! These are extreme cases, but to be fair, I'll show them to you anyway. 

 

[LoyceV]

I already know where you're going with this! These are extreme cases

Extreme cases are good for testing, that's why I picked them. Maybe you can create a formula that increases based on cumulative contributions, so a user with 1 post and 10 Merit doesn't get a higher score than a user with 1000 posts and 1000 Merit.

[acroman08]

This kind of reminds me of the tool(I forgot the name of that tool) that was a bit popular around 2015-2016(I think), where users check the value of their account. Just to be clear, I am not saying they are the same thing, it just kind of reminds me of that.

Anyway, this is an interesting project, I'd love to try it out when it is done.

[fillippone]

I do like the experiment, and I think we can get somewhere with testing (even extreme cases).
Of course, this is something that is a first guidance on each user’s contribution on the forum, that can have diverse forms.

[Vod]

I will participate as well *Ace*. 

[*Ace*]

I already know where you're going with this! These are extreme cases

Extreme cases are good for testing, that's why I picked them. Maybe you can create a formula that increases based on cumulative contributions, so a user with 1 post and 10 Merit doesn't get a higher score than a user with 1000 posts and 1000 Merit.

You are absolutely correct. I will consider modifying the formula, perhaps introducing a penalty system if the profile is too young. I will think about it tonight.

I do like the experiment, and I think we can get somewhere with testing (even extreme cases).
Of course, this is something that is a first guidance on each user’s contribution on the forum, that can have diverse forms.

If it goes well, this experiment could provide valuable feedback on user quality! But it should not be misinterpreted: the tool does not judge what we write, only the data available.

I will participate as well *Ace*. 

Help is welcome, thank you, Vod! What would you like to participate in?

[NeuroticFish]

The idea is interesting, but imho the formula itself needs more work. Maybe merit made more important.

It could also be an idea to also use overall earned merit and overall posts too somewhere. (Of course, also take into account those who didn't post since merit was introduced).

The point is that I think that the current formula may be missing a huge number of old high quality posts (and merits), which could make the results somewhat inconsistent (given that satoshi 14+ score and LoyceV only 0.81)


On the other hand... if you go by everybody's ideas you'll go crazy. So do as you think it fits.