Request @theymos: Can you show userIDs instead of user names in trust.txt.xz?

[LoyceV]

The forum has some very useful weekly data dumps:
https://bitcointalk.org/merit.txt.xz shows userIDs, and is easy to analyze.
https://bitcointalk.org/trust.txt.xz shows usernames, which makes it a lot harder to process:

It's a pain to get all userIDs from this data, especially the ones with weird or even invisible characters.

Current format

Code:

theymos->HostFat
theymos->dooglus
theymos->Raize
theymos->Meni Rosenfeld
theymos->Maged
theymos->Raoul Duke
theymos->gmaxwell
theymos->OgNasty
theymos-/>CanaryInTheMine
theymos->zvs
theymos->-ck
theymos->malevolent
theymos-/>Matthew N. Wright
theymos->John (John K.)
theymos->Tomatocage
theymos->SaltySpitoon
theymos->ineededausername
theymos->DeaDTerra
theymos->BadBear
theymos-/>El Cabron
theymos->Blazr
theymos->vgo
theymos->BCB
theymos->Dabs
theymos-/>Xian01
theymos-/>lophie
theymos-/>johnny5
theymos->Benson Samuel
theymos-/>TradeFortress 🏕
theymos->shiftybugger
theymos-/>uhnonamiss
theymos->Cyrus
theymos-/>nubbins
theymos-/>FullLife
theymos->ibminer
theymos->Wardrick
theymos-/>Boelens
theymos-/>Rub3n
theymos-/>TMAN
theymos-/>TomUnderSea
theymos->dwdoc
theymos-/>blackarrow
theymos->Adriano
theymos->Taras
theymos-/>MarkAz
theymos-/>nachius
theymos->EFS
theymos-/>marcotheminer
theymos->dbshck
theymos-/>TheNewAnon135246
theymos->hilariousandco
theymos->MadZ
theymos->shorena
theymos-/>artw1982
theymos->mitzie
theymos->franckuestein
theymos-/>Jaaawsh
theymos-/>LYCAN
theymos->sapta
theymos-/>Spodermen
theymos->OmegaStarScream
theymos->xandry
theymos->rickbig41
theymos->Barcode_
theymos->Anon11073
theymos->Halab
theymos->Xal0lex
theymos->chimk
allinvain->JWU42
allinvain->warrensgun
allinvain->bbxx
allinvain->Voogru
allinvain->ninjaboon
allinvain->thanke
allinvain->Kartaiv
allinvain->kimosan
allinvain->daddyfatsax
allinvain->bitdominion
allinvain->abdullahadam
allinvain->jimmothy
allinvain->armedmilitia
allinvain->gingpark
allinvain->BitcoinBrains
allinvain->7queue
allinvain->Biodom
allinvain->grn
allinvain->mbauman89
allinvain->zyhsu815
allinvain->lekiller
allinvain->sk8erskid420
allinvain->Finksy
allinvain->MCHouston
allinvain->kilo17
allinvain->tarzan2
allinvain->lex_minutor
allinvain->thedreamer
HostFat->sirius
HostFat->theymos
HostFat->Gavin Andresen
HostFat->dooglus
HostFat-/>gmaxwell
HostFat->rb1205
HostFat->paci
HostFat-/>CanaryInTheMine
HostFat->Stemby
HostFat->ziomik
HostFat->Vod
HostFat->ercolinux
HostFat->BadBear
HostFat->diego1000
HostFat->kelpy
HostFat->GIANNAT
HostFat->bertani
HostFat->Cripto
HostFat-/>Lauda
HostFat->gdassori
HostFat->ghibly79
HostFat->brutale2
HostFat->Anon39
HostFat->davvo
HostFat->alch1mista
teff->theymos
LZ->satoshi
Elwar->theymos
satamusic->MiLkz
satamusic->bt128
satamusic->aziernest
satamusic->PokemonGoArena
btchris-/>TECSHARE
btchris-/>Vod
Bimmerhead->super3
Bimmerhead->daburone
Bimmerhead->mindragon
Bimmerhead->traderwin
Bimmerhead->stonehedge
Stephen Gornick->spndr7
Stephen Gornick->smoothie
Stephen Gornick->jeffersonairplane
Stephen Gornick->b!z
Stephen Gornick->Factotum
Stephen Gornick->Cyrus
Stephen Gornick->Humanxlemming
Stephen Gornick->emdes
Stephen Gornick->tygeade
Stephen Gornick->mrhelpful
Stephen Gornick->Southpaw0
Stephen Gornick->acs267
Stephen Gornick->lemipawa
Stephen Gornick->CLE_Maximus
Stephen Gornick->PA992
Stephen Gornick->shorena
Stephen Gornick->cakir
Stephen Gornick->ifightformerkel
Stephen Gornick->zodiac3011
Stephen Gornick->GHCoins45
Stephen Gornick->kotwica666
Stephen Gornick->Atomicat
Stephen Gornick->irfan_pak10
Stephen Gornick->enhu
Stephen Gornick->Dudeperfect
Stephen Gornick->bitbunnny
Stephen Gornick->zazarb
Stephen Gornick->Josef27
Stephen Gornick->bitcoin revo
Stephen Gornick->RussiaCoinDotInfo
Stephen Gornick->meatmeat
Stephen Gornick->Salmen
Stephen Gornick->stingers
Stephen Gornick->huynhaff
Stephen Gornick->adroitful_one
Stephen Gornick->BoXXoB
Stephen Gornick->tarzan2
Stephen Gornick->harizen
Stephen Gornick->rizkyhiw
Stephen Gornick->ahmedjamal1998
Stephen Gornick-/>The Pharmacist
Stephen Gornick->itarachiu
Stephen Gornick->Isildur (official)
Stephen Gornick->BestWebCreator
Stephen Gornick->ATguy
Stephen Gornick->XERES
Stephen Gornick->btz1
Stephen Gornick->LLec
Stephen Gornick->bL4nkcode
Stephen Gornick->Woshib
Stephen Gornick->pvaspecialist
Stephen Gornick->Gunthar
Stephen Gornick->Chevas Regal
Stephen Gornick->mistonel
Stephen Gornick->BitcoinDomainDeals
Peter Todd->Pieter Wuille
Peter Todd->gmaxwell
Peter Todd->jdillon
fabianhjr->theymos
fabianhjr->majamalu
fabianhjr->TECSHARE
midnightmagic->nanotube
midnightmagic->phantomcircuit
midnightmagic->gmaxwell
.........
.........
.........

Requested format

Code:

35t203
35t3420
35t4171
35t5776
35t6347
35t9477
35t11425
35t18321
35d18614
35t19897
35t19971
35t23092
35d24749
35t31288
35t37522
35t38894
35t40007
35t40976
35t41911
35d44233
35t51908
35t53862
35t54113
35t54791
35d58223
35d62227
35d63141
35t66510
35d67058
35t76857
35d77188
35t78147
35d78483
35d83877
35t84866
35t85316
35d92133
35d95139
35d98986
35d103602
35t105603
35d105804
35t112568
35t125747
35d128716
35d138601
35t140584
35d147773
35t153634
35d153656
35t164822
35t174480
35t181801
35d186695
35t210680
35t225121
35d308108
35d344701
35t347141
35d366423
35t375981
35t382413
35t553678
35t779106
35t969327
35t1053119
35t1068464
35t1202061
198t4755
198t15858
198t39971
198t55841
198t64005
198t68062
198t100703
198t104297
198t114347
198t117393
198t120900
198t137510
198t144088
198t153176
198t155567
198t156757
198t223922
198t306434
198t312916
198t313952
198t319876
198t347350
198t347946
198t371885
198t387443
198t433148
198t460182
198t463163
203t4
203t35
203t224
203t3420
203d11425
203t14716
203t15648
203d18614
203t18953
203t27427
203t30747
203t31410
203t41911
203t57872
203t71769
203t73619
203t76206
203t76216
203d101872
203t120714
203t120821
203t151278
203t165089
203t231686
203t262971
418t35
462t3
554t35
682t408677
682t546570
682t780585
682t894370
1171d15728
1171d30747
1496t73830
1496t98593
1496t131102
1496t173106
1496t360508
2228t205
2228t25960
2228t51082
2228t67815
2228t77813
2228t78147
2228t84386
2228t109754
2228t128443
2228t128986
2228t131037
2228t136850
2228t152856
2228t162395
2228t165138
2228t181801
2228t205338
2228t206417
2228t221888
2228t250879
2228t303259
2228t339589
2228t350580
2228t350984
2228t358787
2228t360402
2228t369212
2228t369756
2228t372249
2228t374494
2228t381490
2228t386072
2228t393142
2228t405674
2228t407213
2228t407841
2228t433148
2228t458393
2228t461467
2228t480991
2228d487418
2228t524125
2228t545598
2228t550758
2228t555675
2228t568782
2228t676638
2228t717279
2228t765632
2228t798632
2228t807047
2228t817682
2228t844504
2228t903987
2228t946940
2546t2786
2546t11425
2546t100781
2671t35
2671t14792
2671t15728
2759t1268
2759t4534
2759t11425
.........
.........
.........

("t" stands for "trusts", "d" stands for "distrusts")

This would make is much easier to create things like LoyceV's Trust list viewer, which I think makes it easier to view a user's custom trust list and can help users to create their own.

[jackg]

I think both is probably better than just using one...

Theyre both quite hefty to have to keep using forum queries to work out the usernames. I think it's done like that for a reason but I wouldn't personally be against having both, I'd be against changing it to just be a userid entirely however...

[LoyceMobile]

I think both is probably better than just using one...

That would be even better indeed.

[pugman]

Rather why not change the whole format of the file?

Maybe a proper constructed table like this maybe? .

|	User_ID	|	User	|	Trusts	|	Distrusts	|
|	35	|	theymos	|	xxx yyy zzz	|	aaa bbb ccc	|
|	03	|	satoshi	|	xxx yyy zzz	|	aaa bbb ccc	|

[suchmoon]

Rather why not change the whole format of the file?

It needs to be easy to parse. JSON, XML, CSV...

I'm not a big fan of LoyceV's format without separators, it would require pre-processing or custom parsing to load into tools that I use.

[jackg]

Rather why not change the whole format of the file?

It needs to be easy to parse. JSON, XML, CSV...

I'm not a big fan of LoyceV's format without separators, it would require pre-processing or custom parsing to load into tools that I use.

Pugman might be onto something though.

A csv file of
userid, username, userid, username,
Like what we have now but having both details in the one file...

[pugman]

Rather why not change the whole format of the file?

It needs to be easy to parse. JSON, XML, CSV...

I'm not a big fan of LoyceV's format without separators, it would require pre-processing or custom parsing to load into tools that I use.

Can we not just have an area dedicated to this whole file, as in when we go to the trust page, we could see the table content of who trusts/distrusts who?

Won't it make things easier? Im a dumbhead, ignore me if I am too stupid to understand things.

[LoyceV]

I'm not a big fan of LoyceV's format without separators

I used "t" and "d" as separators, but anything else is fine too.

A csv file of
userid, username, userid, username

It's an improvement, but since usernames can have "weird characters", it's still not that easy to separate.

Can we not just have an area dedicated to this whole file

That is the reason why I ask this Theymos' data dumps make it possible for users to create their own data analysis, which means less work for Admin.
My recent example, and the reason for creating this topic:

LoyceV's Trust list viewer

[Coin-1]

My recent example, and the reason for creating this topic:

LoyceV's Trust list viewer

Your tool is useful. Since I regularly update TOP-200 the most trusted members DT1, DT2, DT3, I'm also voting for the requested suggestion. In this case, it will be much easier to parse the dump of the Marketplace Trust Network for machine processing.

But maybe some people will want to download the file "trust.txt.xz" and manually check the relationship in a human readable format, so there is no need to change the structure of this file. In my opinion, the Marketplace Trust Network should be published in two files: "trust.txt.xz" and, for example, "trust_id.txt.xz".

[Quickseller]

Your proposal is good as the current system makes it difficult to keep track of changes in the trust network when VIPs and mods change their display name.

On the other hand, only providing UIDs makes it very difficult for anyone to analyze the information without automated tools.

Maybe a good solution would be to map the name to the UID if the name appears on trust.txt.xz

[mikeywith]

this is pretty easy to parse  using Excel.




you just need the 2 rows for username and id (M,N)
copy-paste the data frum trust.txt to (I,J)
use the function below on any 2 columns you want and it will parse the results for you

Code:

=IF(ISERROR(INDEX($N$3:$N$100,MATCH(I3,$M$3:$M$100,0))),"",INDEX($N$3:$N$100,MATCH(I3,$M$3:$M$100,0)))

*you need to replace I with J for the trustee column.

of course i assume you know how to put these in a single column if needed and how to add whatever separators you want.



if you having a trouble with it, just send me a an excel or text file with all usrename and ids, i'l upload an excel file where you just need to import the trust.txt and everything else will be automated.

[LoyceV]

just send me a an excel or text file with all usrename and ids

Getting the userIDs with the usernames is the problem.

[mikeywith]

just send me a an excel or text file with all usrename and ids

Getting the userIDs with the usernames is the problem.

it's not, it's very simple, using piggy's tool > https://albertoit.github.io/Merit-Explorer-SQL/  ( last update according to piggy is last friday 1 feb 2019)

use this query

Code:

select username,userid from  userdata 
order by userid 

if you wish to see the last userid first in case so that you will only copy the new users that you don't have , use this query

Code:

select username,userid from  userdata 
order by userid desc

hit execute.

copy all users and ids, paste them in an excel file( or wherever you want them to be) and you will get 2 rows , userid and username

to speed things up , i copied everything (ctrl + A) pasted it in a text file and simply removed everything above the tables. and i got something like this

Code:

UserId	UserName
3	satoshi
4	sirius
11	madhatter
13	SmokeTooMuch
24	dwdollar
26	NewLibertyStandard
29	giik
30	BitcoinFX
33	m0mchil
35	theymos
40	xc

you can download the .txt file with complete list here > https://file.io/HIZ47N

last user id  = 2528400


i'll upload an excel file a bit later which will parse the format to this

Code:

35t203
35t3420
35t4171

you will still have to do a bit of manual work every time you want to update it.

1- copy paste new users from piggys tools using "desc" order.( done once at every update you make)
2- copy paste the trust.txt data to the excel file.

not a perfect approach but something you can work with.

[LoyceV]

you can download the .txt file with complete list here > https://file.io/HIZ47N

I checked 2 of the usernames that gave me a hard time:
https://bitcointalk.org/index.php?action=profile;u=131122
https://bitcointalk.org/index.php?action=profile;u=249929
They're not in there. There are only a few that give me a hard time.

1- copy paste new users from piggys tools using "desc" order.( done once at every update you make)
2- copy paste the trust.txt data to the excel file.

Thanks for the work you put in, but I'm looking for a more permanent solution with less manual work. I'm trying a different approach now, if it works I'll post a weekly "trust_id.txt.xz" (thanks Coin-1 for the name).

[mikeywith]

I checked 2 of the usernames that gave me a hard time:
https://bitcointalk.org/index.php?action=profile;u=131122
https://bitcointalk.org/index.php?action=profile;u=249929
They're not in there. There are only a few that give me a hard time.

ya it's strange, i checked those names and they are not there, i am not sure how piggy retrieve his data , does bpip has any database similar to piggy's format?

[LoyceV]

does bpip has any database similar to piggy's format?

No, I scraped the usernames one by one from BPIP.

[Piggy]

ya it's strange, i checked those names and they are not there, i am not sure how piggy retrieve his data , does bpip has any database similar to piggy's format?

The users in there are only those which ever sent or received any merit, so for trust purposes is not complete.

Peraphs is better to use this db in here https://bitcointalk.org/index.php?topic=5066192.msg47728970#msg47728970 even if it's not guaranteed either the will be all the id available.

[DdmrDdmr]

<…> Peraphs is better to use this db in here https://bitcointalk.org/index.php?topic=5066192.msg47728970#msg47728970 even if it's not guaranteed either the will be all the id available.

That’s kind of what I do. I start try to retrieve the profile id from the merited profiles I download, and default to your December 2018 DB for those unmerited.
 
This month I got something like 15 to 20 profiles I had to retrieve manually (using the search profile option), since they were created from December 2018 onwards. Retrieving a few manually is fine, but if many new profiles enter the scene, then retrieving the info for drawing the trust relations becomes a bit cumbersome. User Ids in the trust.txt file would improve retrieval options (better as a separate file as has been suggested on this thread).

[LoyceV]

I checked 2 of the usernames that gave me a hard time:
https://bitcointalk.org/index.php?action=profile;u=131122
https://bitcointalk.org/index.php?action=profile;u=249929

I think I pulled it off. I've converted trust.txt into trust.userIDs.txt. As far as I can tell, I covered all usernames it without any mistakes.

My request still stands, as it makes things much easier, but for now it works for me. I'll publish weekly updates through LoyceV's Trust list viewer.

[LoyceV]

Bump:

Bump: since more and more users change their username, I'd really like to have this request granted!