Bitcoin Forum
March 19, 2024, 03:25:28 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2]  All
  Print  
Author Topic: Additional data dumps?  (Read 915 times)
TheBeardedBaby
Legendary
*
Offline Offline

Activity: 2184
Merit: 3134


₿uy / $ell


View Profile
February 08, 2019, 08:37:25 AM
 #21

It's not a necrobump.
Can we have the modlog and seclog dumps instead everyone to scrape the data from the server?

The forum strives to allow free discussion of any ideas. All policies are built around this principle. This doesn't mean you can post garbage, though: posts should actually contain ideas, and these ideas should be argued reasonably.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1710818728
Hero Member
*
Offline Offline

Posts: 1710818728

View Profile Personal Message (Offline)

Ignore
1710818728
Reply with quote  #2

1710818728
Report to moderator
1710818728
Hero Member
*
Offline Offline

Posts: 1710818728

View Profile Personal Message (Offline)

Ignore
1710818728
Reply with quote  #2

1710818728
Report to moderator
1710818728
Hero Member
*
Offline Offline

Posts: 1710818728

View Profile Personal Message (Offline)

Ignore
1710818728
Reply with quote  #2

1710818728
Report to moderator
100bitcoin
Sr. Member
****
Offline Offline

Activity: 858
Merit: 423


View Profile WWW
February 08, 2019, 12:15:11 PM
 #22

It's not a necrobump.
Can we have the modlog and seclog dumps instead everyone to scrape the data from the server?


Can one still able to scrape data from the server? I thought theymos prohibited it since bitcointalk.to started to scrape the whole forum.

TheBeardedBaby
Legendary
*
Offline Offline

Activity: 2184
Merit: 3134


₿uy / $ell


View Profile
February 08, 2019, 12:23:34 PM
 #23

Can one still able to scrape data from the server? I thought theymos prohibited it since bitcointalk.to started to scrape the whole forum.

Both LoyceV and Vod are doing it, also i've seen other users too, so I think there is no any prohibition, yet.
I think if those dumps are available for download directly from the forum, more people can benefit out of it and there will be less traffic to the server.

LoyceV
Legendary
*
Offline Offline

Activity: 3248
Merit: 16270


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 08, 2019, 12:27:18 PM
 #24

Scraping works, but the current modlog covers only a limited time. It's not possible to get a complete overview of all banned users. From various sources, I have a list of 170k banned users now, but it's far from complete.

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
TheBeardedBaby
Legendary
*
Offline Offline

Activity: 2184
Merit: 3134


₿uy / $ell


View Profile
February 08, 2019, 12:44:52 PM
Last edit: February 08, 2019, 02:03:45 PM by iasenko
 #25

Scraping works, but the current modlog covers only a limited time. It's not possible to get a complete overview of all banned users. From various sources, I have a list of 170k banned users now, but it's far from complete.

Is there any place to have the modlog in raw format available? Even for the limited time. I want to check some things Smiley

LoyceV
Legendary
*
Offline Offline

Activity: 3248
Merit: 16270


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 08, 2019, 05:02:00 PM
 #26

Is there any place to have the modlog in raw format available? Even for the limited time. I want to check some things Smiley
Do you mean older versions? I used archive.li and archive.org.

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
tranthidung
Legendary
*
Offline Offline

Activity: 2212
Merit: 3904



View Profile WWW
January 09, 2020, 03:48:01 AM
Last edit: January 09, 2020, 05:54:43 AM by tranthidung
 #27

UID -> name, merit, potential activity, posts
 post ID -> topic ID, time, UID
 topic ID -> board ID, first post ID
 board ID -> board name
The new year, so I bump it to ask for additional data dump granted by theymos.

Besides these formats above, I ask for this one (for merit data):
Code:
time amount msg user_from user_to boardid
1516831941  1 2818066.msg28853325 35 877396 24
I already collected the boardid, so if the merit data has only one additional variable for board's ID (boardid), it will eliminate the need to scrap data (from LoyceV's help) each 6 months. Although I don't know the others need such variable in data dumps or not.

For some sorts of analyses like these:

▄▄███████▄▄
▄██████████████▄
▄██████████████████▄
▄████▀▀▀▀███▀▀▀▀█████▄
▄█████████████▄█▀████▄
███████████▄███████████
██████████▄█▀███████████
██████████▀████████████
▀█████▄█▀█████████████▀
▀████▄▄▄▄███▄▄▄▄████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀
.
 MΞTAWIN  THE FIRST WEB3 CASINO   
.
.. PLAY NOW ..
tranthidung
Legendary
*
Offline Offline

Activity: 2212
Merit: 3904



View Profile WWW
January 11, 2020, 03:36:40 AM
Merited by LoyceV (1), PrimeNumber7 (1)
 #28

From the reply of admin yesterday, I think now it is a very good time to think of a consistent format for the forum's data dumps. Each dataset has different variables inside, but I think all of them should be connected with only common variable (at least one variable) - userid.

Username, no matter it is username or display name or both will result in differences when connecting different datasets dumped by the forum.

For additional data dumps, it is not the priority and I am not in a position to ask for it too much, but for current data formats, a small adjustment: from username to userid will be good.

LoyceV asked for this change too: https://bitcointalk.org/index.php?topic=5104467.msg53551686#msg53551686

▄▄███████▄▄
▄██████████████▄
▄██████████████████▄
▄████▀▀▀▀███▀▀▀▀█████▄
▄█████████████▄█▀████▄
███████████▄███████████
██████████▄█▀███████████
██████████▀████████████
▀█████▄█▀█████████████▀
▀████▄▄▄▄███▄▄▄▄████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀
.
 MΞTAWIN  THE FIRST WEB3 CASINO   
.
.. PLAY NOW ..
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!