Bitcoin Forum
September 19, 2019, 03:20:16 AM *
News: Latest Bitcoin Core release: 0.18.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Interact & Explore the Merit Network  (Read 485 times)
Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 02, 2018, 11:54:30 AM
Last edit: July 04, 2018, 06:34:05 AM by Piggy
Merited by Foxpup (15), TMAN (15), Welsh (10), malevolent (7), Vod (5), jacee (5), ETFbitcoin (2), LoyceV (2), Lutpin (2), duesoldi (2), khufuking (2), Tytanowy Janusz (2), babo (1), sabotag3x (1), iasenko (1), vlad230 (1), 1miau (1)
 #1

This has been quite a long and challenging project given the amount of information to show, i have started to look into it since i saw the network graph threads of DdmrDdmr and sncc. My goal was to have an explorable version of the full network with interactions included, as well as my take on trying to find new merit sources.

Each node represent a user and the edges with arrows merit transactions, received or given.
I made 2 versions, the main difference is in the shape. Then for each of these versions i made two variants,one where the coloring is based on the Rank and the other is based on the Trust.








The node size is based on the Node Merit Size variable (on the User Information Panel) and reaches the maximum with a total of 500 merits given away, i did this to try to avoid the current merit sources shading emerging one. This should help visualize at sight the main givers just by looking at the size of the nodes.

Node Merit Size is calculated in the following way:
Code:
If Merit Given > 500
Node Merit Size = 500
  otherwise
Node Merit Size = Merit Given


The node and edge color simply tells the rank/trust of the node.

I removed from the network all the nodes and edges where:

Code:
Merit Received + Merit Given = 1

So anybody who sent a single merit or received a single one won’t be in it.

There are exactly 15954 Nodes and 54108 Edges in there. If there are more merit transactions from user A to user B they will be grouped together and be represented with 1 line(Edge) that goes from A to B

What is the purpose?

You can use these visualizations to help identify users which could be a merit source, since at first glance you have a very intuitive way of knowing the amount of Merit given (size of the node) and Rank/Trust (color of the node). Plus the personal network tells the merit story of the user.

How i made this?

I prepared all the data i needed and made several iteration with Gephi for the layout of the network graph. The drawing is made using the library Sigma Js which supports WebGL.

How to use it?

  • Each node you see can show the user name on mouse over and by clicking on it it will isolate the connections based on merit sent and received, both in the form of a graph and with a panel, which shows supplementary informations about the use selected as well. Moving the mouse over the names on the list will show the label on the graph as well.
  • You can drag the network by clicking on an empty space, zoom in and out with your mouse wheel or using the buttons in the bottom overlay. The magnifying lens with the square center the graph on screen.
  • On the left panel you can search for any user by name.

Links to the visualizations:

https://albertoit.github.io/Merit-Network-Visualization/
https://albertoit.github.io/Merit-Network-Visualization/?config=config_Trust.json
https://albertoit.github.io/Merit-Network-Visualization/?config=configsparse.json
https://albertoit.github.io/Merit-Network-Visualization/?config=configsparse_Trust.json

1568863216
Hero Member
*
Offline Offline

Posts: 1568863216

View Profile Personal Message (Offline)

Ignore
1568863216
Reply with quote  #2

1568863216
Report to moderator
1568863216
Hero Member
*
Offline Offline

Posts: 1568863216

View Profile Personal Message (Offline)

Ignore
1568863216
Reply with quote  #2

1568863216
Report to moderator
1568863216
Hero Member
*
Offline Offline

Posts: 1568863216

View Profile Personal Message (Offline)

Ignore
1568863216
Reply with quote  #2

1568863216
Report to moderator
"The nature of Bitcoin is such that once version 0.1 was released, the core design was set in stone for the rest of its lifetime." -- Satoshi
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
iasenko
Hero Member
*****
Online Online

Activity: 686
Merit: 1220


Running my first Full Node :)


View Profile WWW
July 02, 2018, 12:14:18 PM
 #2

Great work as usual.
My small cloud is beautiful

DdmrDdmr
Hero Member
*****
Offline Offline

Activity: 616
Merit: 2671

There are lies, damned lies and statistics. MTwain


View Profile WWW
July 02, 2018, 12:57:56 PM
Last edit: July 02, 2018, 01:08:07 PM by DdmrDdmr
 #3

As I tend to do, I debugged the graph for my username and I saw two Donators on it. One is Ognasty, who appears with a decent sized sphere (the largest, along with other usernames) even though Ognasty has historically sent me 1 sMerit and I have not sMerited this account. The size is the same as other contributors who have awarded me way more merits, so I’m not sure the sizing is working 100% correctly.

It is indeed quite a lot of work to get there. It seems that you can get something similar on to Tableau, and I looked ad doing it on my Dashboard, but having to go through Gephi to derive the coordinates is not a quick process, and seems way too slow to allow me to do it weekly without too much additional overhead. I don’t know if you are planning frequent update of your tool, but I believe is a heavy task.

Nicely done and appreciate the effort behind the scenes.

Edit: I'm also getting a graph on the Trust graphs, but I'm not in the trust.txt file. Am I missing something ?

yuzefa
Newbie
*
Offline Offline

Activity: 79
Merit: 0


View Profile
July 02, 2018, 01:09:28 PM
 #4

Really nice tools, this could be usefull for mods and admins.
I'm still thinking we need really much more Merit sources, and this could help to identify them faster.

 Wink
Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 02, 2018, 01:20:59 PM
 #5

As I tend to do, I debugged the graph for my username and I saw two Donators on it. One is Ognasty, who appears with a decent sized sphere (the largest, along with other usernames) even though Ognasty has historically sent me 1 sMerit and I have not sMerited this account. The size is the same as other contributors who have awarded me way more merits, so I’m not sure the sizing is working 100% correctly.

It is indeed quite a lot of work to get there. It seems that you can get something similar on to Tableau, and I looked ad doing it on my Dashboard, but having to go through Gephi to derive the coordinates is not a quick process, and seems way too slow to allow me to do it weekly without too much additional overhead. I don’t know if you are planning frequent update of your tool, but I believe is a heavy task.

Nicely done and appreciate the effort behind the scenes.

Edit: I'm also getting a graph on the Trust graphs, but I'm not in the trust.txt file. Am I missing something ?

I capped on purpose the size of each node base on a scale from 1 to 500 merits sent (Node Merit Size field in the User Information Panel), otherwise we would be looking at enormous nodes of the merit sources which sent more than 1k merits, taking all the scene, this way you can notice also those users who sent perhaps "just" few hundred merits and are not merit sources.

In this case Gephi produces a gfex which i can then use directly to show the graph, you need anyway to prepare and clean the data and then i had to intervene directly on the file to add other information in the way i wanted (Trust, Rank and the scale for the node i mentioned above), it takes time anyway and is something that can be produced from time to time, i agree.

For each graph there is a different gfex file and it changes the coloring of the nodes beside the shape of the graph, you can find the files in here:

https://github.com/AlbertoIT/Merit-Network-Visualization/tree/master/data

DdmrDdmr
Hero Member
*****
Offline Offline

Activity: 616
Merit: 2671

There are lies, damned lies and statistics. MTwain


View Profile WWW
July 02, 2018, 01:59:02 PM
 #6

<...>
Ok, I saw that the size of the node was capped to 500 in the OP. The size of the node is therefore overall, as a sMerit sender, not relative to the sMerit sent to me specifically, but in general Forum wide If I understood it properly now.

Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 03, 2018, 02:20:20 PM
 #7

Yes is working that way, i just added the rule i used to show how the parameter which govern the size of the node is calculated:

Code:
If Merit Given > 500
Node Merit Size = 500
  else
Node Merit Size = Merit Given

For example From the User Information Panel:

Quote
TMAN
Merit Received: 601.0

Merit Given: 1590.0

Merit Given + Received: 2191.0

Rank: Hero Member

Trust: +1

Node Merit Size: 500.0

Quote
babo
Merit Received: 43.0

Merit Given: 139.0

Merit Given + Received: 182.0

Rank: Legendary

Trust: +1

Node Merit Size: 139.0

Another detail i would like to point out.
The goal of this work is to find new merit sources, but if you look at the peripheral area of the graph you can see there are a lot of isolated graph, i'm not 100% sure, but there could be a lot of abusers there. If anybody want to investigate, would be relatively easy since there is a version of the graph where red trusted users are already marked.
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1764
Merit: 2028

Use SegWit and enjoy lower fees.


View Profile WWW
July 03, 2018, 07:29:53 PM
 #8

Really neat, even though user with slow connection like me need to wait for a while (10-60s) Smiley

Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 04, 2018, 06:36:15 AM
 #9

Really neat, even though user with slow connection like me need to wait for a while (10-60s) Smiley

It is quite a lot of information, in fact the waiting time in the begin goes mostly in the downloading of the file.

There are exactly 15954 Nodes(Users) and 54108 Edges("Merit connections") in there. If there are more merit transactions from A to B they will be grouped together and be represented with 1 line that goes from A to B, basically a "Merit connection"

athanz88
Sr. Member
****
Offline Offline

Activity: 476
Merit: 355


View Profile
July 04, 2018, 09:29:36 AM
 #10

Wow this is a nice visualization of user's merit. What i like the most about your work is, it can be used to look on how worthy a person is, based on the big names who gives them merits. I know that the weight of the merit is the same across all the member, but earning merits from big names sure give you some kind of "proud" feeling over it.

And it is easier to notice that via this visualization, rather than scrolling through a member's merit history list.
vlad230
Sr. Member
****
Offline Offline

Activity: 546
Merit: 277



View Profile
July 04, 2018, 12:00:14 PM
 #11

This looks very interesting Smiley

It would be nice if we could use a system like this to see a user's post distribution across the boards for a better view of one's activity here on BCT.

For example, we could see which users mostly post in bounty threads Wink
ETFbitcoin
Legendary
*
Offline Offline

Activity: 1764
Merit: 2028

Use SegWit and enjoy lower fees.


View Profile WWW
July 04, 2018, 08:07:22 PM
 #12

Really neat, even though user with slow connection like me need to wait for a while (10-60s) Smiley

It is quite a lot of information, in fact the waiting time in the begin goes mostly in the downloading of the file.

There are exactly 15954 Nodes(Users) and 54108 Edges("Merit connections") in there. If there are more merit transactions from A to B they will be grouped together and be represented with 1 line that goes from A to B, basically a "Merit connection"



I see, but i think it would be better if there's simple loading bar/percentage. At first, i thought your site isn't compatible with my browser or my security extension break the page.

Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 05, 2018, 07:14:12 AM
 #13

Wow this is a nice visualization of user's merit. What i like the most about your work is, it can be used to look on how worthy a person is, based on the big names who gives them merits. I know that the weight of the merit is the same across all the member, but earning merits from big names sure give you some kind of "proud" feeling over it.

And it is easier to notice that via this visualization, rather than scrolling through a member's merit history list.

The idea of this kind of this visualizations is indeed to give an immediate visual impact (and expose patterns ) through simple properties( size of the graphs, colors, size of nodes, connections ) in order to have an immediate idea of what is happening, without having to look and analyze raw numbers.

This looks very interesting Smiley

It would be nice if we could use a system like this to see a user's post distribution across the boards for a better view of one's activity here on BCT.

For example, we could see which users mostly post in bounty threads Wink

This is more useful to represent interactions between user, but something can be already done on a similar topic, check in which board a user is awarding merit mostly

https://albertoit.github.io/Merit-Explorer-SQL/

Code:
SELECT SUM(Merit),Board,SubBoard
FROM MeritData
INNER JOIN UserData on fromID=UserData.UserID
WHERE UserData.Username like "vlad230"
Group BY Board,SubBoard
ORDER BY Board,SubBoard;

in here you can find other SQL query examples if you are interested: https://bitcointalk.org/index.php?topic=4551881.0

Really neat, even though user with slow connection like me need to wait for a while (10-60s) Smiley

It is quite a lot of information, in fact the waiting time in the begin goes mostly in the downloading of the file.

There are exactly 15954 Nodes(Users) and 54108 Edges("Merit connections") in there. If there are more merit transactions from A to B they will be grouped together and be represented with 1 line that goes from A to B, basically a "Merit connection"



I see, but i think it would be better if there's simple loading bar/percentage. At first, i thought your site isn't compatible with my browser or my security extension break the page.

You are right, i have been working so long with it i stopped seeing this sort of problems, i'll make it clear somehow that is loading the data in the begin Smiley
Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
July 06, 2018, 07:52:59 AM
 #14

i now made a change which should help understand the data is actually being loaded and everything is "fine", for those who may wonder why they see an empty screen for a while before anything happen  Smiley


lequocvuongpro
Jr. Member
*
Offline Offline

Activity: 462
Merit: 2


View Profile
July 06, 2018, 08:03:36 AM
 #15

This has been quite a long and challenging project given the amount of information to show, i have started to look into it since i saw the network graph threads of DdmrDdmr and sncc. My goal was to have an explorable version of the full network with interactions included, as well as my take on trying to find new merit sources.

Each node represent a user and the edges with arrows merit transactions, received or given.
I made 2 versions, the main difference is in the shape. Then for each of these versions i made two variants,one where the coloring is based on the Rank and the other is based on the Trust.








The node size is based on the Node Merit Size variable (on the User Information Panel) and reaches the maximum with a total of 500 merits given away, i did this to try to avoid the current merit sources shading emerging one. This should help visualize at sight the main givers just by looking at the size of the nodes.

Node Merit Size is calculated in the following way:
Code:
If Merit Given > 500
Node Merit Size = 500
  otherwise
Node Merit Size = Merit Given


The node and edge color simply tells the rank/trust of the node.

I removed from the network all the nodes and edges where:

Code:
Merit Received + Merit Given = 1

So anybody who sent a single merit or received a single one won’t be in it.

There are exactly 15954 Nodes and 54108 Edges in there. If there are more merit transactions from user A to user B they will be grouped together and be represented with 1 line(Edge) that goes from A to B

What is the purpose?

You can use these visualizations to help identify users which could be a merit source, since at first glance you have a very intuitive way of knowing the amount of Merit given (size of the node) and Rank/Trust (color of the node). Plus the personal network tells the merit story of the user.

How i made this?

I prepared all the data i needed and made several iteration with Gephi for the layout of the network graph. The drawing is made using the library Sigma Js which supports WebGL.

How to use it?

  • Each node you see can show the user name on mouse over and by clicking on it it will isolate the connections based on merit sent and received, both in the form of a graph and with a panel, which shows supplementary informations about the use selected as well. Moving the mouse over the names on the list will show the label on the graph as well.
  • You can drag the network by clicking on an empty space, zoom in and out with your mouse wheel or using the buttons in the bottom overlay. The magnifying lens with the square center the graph on screen.
  • On the left panel you can search for any user by name.

Links to the visualizations:

https://albertoit.github.io/Merit-Network-Visualization/
https://albertoit.github.io/Merit-Network-Visualization/?config=config_Trust.json
https://albertoit.github.io/Merit-Network-Visualization/?config=configsparse.json
https://albertoit.github.io/Merit-Network-Visualization/?config=configsparse_Trust.json


This very great idea for Bitcointalk, Hope it's will be applied for Rule of Merit rank. Thank for your sharing. Good luck !!.
nngella
Member
**
Offline Offline

Activity: 252
Merit: 37

Colletrix - Bridging the Physical and Virtual Worl


View Profile
October 25, 2018, 02:44:28 PM
 #16

May I know the reckoning date of the data that you used? 
I just search some users and I think the data is not updated.

You can see merit abusers in this by looking for trading of merits or giving abundant merits (also to identify multiple accounts)

Piggy
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1376



View Profile WWW
October 25, 2018, 05:08:40 PM
 #17

May I know the reckoning date of the data that you used? 
I just search some users and I think the data is not updated.

You can see merit abusers in this by looking for trading of merits or giving abundant merits (also to identify multiple accounts)

This is using merit data available till 1st of July, i may update it later on.
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!