Bitcoin Forum
December 15, 2024, 08:52:21 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Need to obtain statistics from forum (new members and new posts)  (Read 247 times)
1719EBDIHEC (OP)
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
February 07, 2019, 04:41:56 PM
 #1

Hi!

I am writing a master's thesis on bitcoin price formulation, and I need to obtain statistics from Bitcointalk.org from July 1, 2015, to December 31, 2018. I can only find data up until end of November 2017. Anyone know where to access the data I need?
Help is highly appreciated!
LoyceV
Legendary
*
Offline Offline

Activity: 3528
Merit: 17829


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 07, 2019, 04:51:53 PM
 #2

What data are you looking for? New registrations can be counted by checking profiles, and total number of posts can be counted by checking when a certain post number was made.

▄▄███████████████████▄▄
▄█████████▀█████████████▄
███████████▄▐▀▄██████████
███████▀▀███████▀▀███████
██████▀███▄▄████████████
█████████▐█████████▐█████
█████████▐█████████▐█████
██████████▀███▀███▄██████
████████████████▄▄███████
███████████▄▄▄███████████
█████████████████████████
▀█████▄▄████████████████▀
▀▀███████████████████▀▀
Peach
BTC bitcoin
Buy and Sell
Bitcoin P2P
.
.
▄▄███████▄▄
▄████████
██████▄
▄██
█████████████████▄
▄███████
██████████████▄
███████████████████████
█████████████████████████
████████████████████████
█████████████████████████
▀███████████████████████▀
▀█████████████████████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀

▀▀▀▀███▀▀▀▀
EUROPE | AFRICA
LATIN AMERICA
▄▀▀▀











▀▄▄▄


███████▄█
███████▀
██▄▄▄▄▄░▄▄▄▄▄
████████████▀
▐███████████▌
▐███████████▌
████████████▄
██████████████
███▀███▀▀███▀
.
Download on the
App Store
▀▀▀▄











▄▄▄▀
▄▀▀▀











▀▄▄▄


▄██▄
██████▄
█████████▄
████████████▄
███████████████
████████████▀
█████████▀
██████▀
▀██▀
.
GET IT ON
Google Play
▀▀▀▄











▄▄▄▀
1719EBDIHEC (OP)
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
February 07, 2019, 05:24:23 PM
 #3

Specifically, I need the daily number of new posts, and daily number of new members. (If its possible to get page views as well, that would be ideal.) I have all that data from July 1, 2015 - November 31., 2017. So I need from December 1., 2017, to December 31., 2018.

LoyceV
Legendary
*
Offline Offline

Activity: 3528
Merit: 17829


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 07, 2019, 05:44:57 PM
 #4

I need the daily number of new posts
You can get a pretty good estimate by finding a post around the same time each day, and subtracting post numbers. It's easiest to find them by date if you look in one of the many altcoin ANN (spam) threads. Kinda like I did here.

Quote
and daily number of new members.
See Open scraped data of all the users - SQL Lite DB - 2.481.270 users

▄▄███████████████████▄▄
▄█████████▀█████████████▄
███████████▄▐▀▄██████████
███████▀▀███████▀▀███████
██████▀███▄▄████████████
█████████▐█████████▐█████
█████████▐█████████▐█████
██████████▀███▀███▄██████
████████████████▄▄███████
███████████▄▄▄███████████
█████████████████████████
▀█████▄▄████████████████▀
▀▀███████████████████▀▀
Peach
BTC bitcoin
Buy and Sell
Bitcoin P2P
.
.
▄▄███████▄▄
▄████████
██████▄
▄██
█████████████████▄
▄███████
██████████████▄
███████████████████████
█████████████████████████
████████████████████████
█████████████████████████
▀███████████████████████▀
▀█████████████████████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀

▀▀▀▀███▀▀▀▀
EUROPE | AFRICA
LATIN AMERICA
▄▀▀▀











▀▄▄▄


███████▄█
███████▀
██▄▄▄▄▄░▄▄▄▄▄
████████████▀
▐███████████▌
▐███████████▌
████████████▄
██████████████
███▀███▀▀███▀
.
Download on the
App Store
▀▀▀▄











▄▄▄▀
▄▀▀▀











▀▄▄▄


▄██▄
██████▄
█████████▄
████████████▄
███████████████
████████████▀
█████████▀
██████▀
▀██▀
.
GET IT ON
Google Play
▀▀▀▄











▄▄▄▀
1719EBDIHEC (OP)
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
February 11, 2019, 03:15:00 PM
 #5

Thank you, LoyceV!
Where do I find the # for each post?
LoyceV
Legendary
*
Offline Offline

Activity: 3528
Merit: 17829


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
February 11, 2019, 03:20:28 PM
 #6

Where do I find the # for each post?
Copy the URL, your post for example:
https://bitcointalk.org/index.php?topic=5107302.msg49692785#msg49692785
The bold part is the post number.

▄▄███████████████████▄▄
▄█████████▀█████████████▄
███████████▄▐▀▄██████████
███████▀▀███████▀▀███████
██████▀███▄▄████████████
█████████▐█████████▐█████
█████████▐█████████▐█████
██████████▀███▀███▄██████
████████████████▄▄███████
███████████▄▄▄███████████
█████████████████████████
▀█████▄▄████████████████▀
▀▀███████████████████▀▀
Peach
BTC bitcoin
Buy and Sell
Bitcoin P2P
.
.
▄▄███████▄▄
▄████████
██████▄
▄██
█████████████████▄
▄███████
██████████████▄
███████████████████████
█████████████████████████
████████████████████████
█████████████████████████
▀███████████████████████▀
▀█████████████████████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀

▀▀▀▀███▀▀▀▀
EUROPE | AFRICA
LATIN AMERICA
▄▀▀▀











▀▄▄▄


███████▄█
███████▀
██▄▄▄▄▄░▄▄▄▄▄
████████████▀
▐███████████▌
▐███████████▌
████████████▄
██████████████
███▀███▀▀███▀
.
Download on the
App Store
▀▀▀▄











▄▄▄▀
▄▀▀▀











▀▄▄▄


▄██▄
██████▄
█████████▄
████████████▄
███████████████
████████████▀
█████████▀
██████▀
▀██▀
.
GET IT ON
Google Play
▀▀▀▄











▄▄▄▀
1719EBDIHEC (OP)
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
February 11, 2019, 03:44:10 PM
 #7

Ok, I must be doing something wrong:

Date           Time         Post #       # new posts
13.des   3:01 am   2561205   
14.des   3:34 am   2581018   19813
15.des   3:20 am   2588638   7620
16.des   3:27 am   2595235   6597
17.des   2:46 am   2595701   466

According to the statistics here on bitcointalk.org, there were 40668 new posts on December 13, 2017.
DdmrDdmr
Legendary
*
Offline Offline

Activity: 2534
Merit: 11079


There are lies, damned lies and statistics. MTwain


View Profile WWW
February 11, 2019, 04:18:26 PM
 #8

<...>
I can help you out quickly with the number of registered users per day: https://docs.google.com/spreadsheets/d/1tZ2a_bSEkbh0Qk-_QfX2WEhc0huRxAsg5hXfGIjI8RA/edit?usp=sharing

The period of time goes from the 17/11/2009 until the 05/12/2018.

Note that:
-   Data for the 05/12/2018 may be partial (incomplete day).

-   Data is derived from @piggy’s complete profile scrape, referenced on this thread.

-   The above referenced profile scrape is the latest I’m aware of, so there is no available external data from 05/12/2018 related to the number of accounts created per day.

-   Many accounts that are created on a daily basis are bot accounts (or just for viewing certain stuff), and actually never get any activity under their belly. See Analysis - Bitcointalk Accounts – Date of Register vs Last Active per Rank for a more comprehensive description.


Retrieving posts per day is a pain in the backside, so I can’t help you with that on a daily granular scale. You could try a per month rough vision like I’ve done in the past, checking some stats on different snapshots on archieve.is or WayBackMachine (see for example Analysis – Merit per post per Section/Subsection).

Page Views would require the same technique as above (although I’ve never been bothered with it).
1719EBDIHEC (OP)
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
February 12, 2019, 08:03:08 AM
 #9

DdmrDdmr, thank you! Very helpful  Smiley I´ll might have to stick to only new members, but anyways, thank you so much!
Alex_Sr
Hero Member
*****
Offline Offline

Activity: 840
Merit: 962


HOLD BITCOIN! Fiat - SCAM!


View Profile WWW
February 12, 2019, 08:13:02 AM
 #10

Hi!

I am writing a master's thesis on bitcoin price formulation, and I need to obtain statistics from Bitcointalk.org from July 1, 2015, to December 31, 2018. I can only find data up until end of November 2017. Anyone know where to access the data I need?
Help is highly appreciated!

Monthly statistics:

Statistics of user registrations on Bitcointalk 2017-2019


░░░░░░░▄▄▄▄▄▄
░░░░▄██████████▄
░░░██████████████
░░██████▐▌██████
█████░░░░░░░▀█████
██████▄▄░░▄▄░░██████
████████░░▀▀▄██████
████████░░▄▄▄░░█████
██████▀▀░░▀▀▀░░█████
█████░░░░░░░░█████
░░██████▐▌██████
░░░██████████████
░░░░▀██████████▀
░░░░░░░▀▀▀▀▀▀
░░░
||
|
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!