[2374859]

I have successfully modified the scraper to scrape the time of the last post of any user, enabling to use this information to explore user activity on this forum more precisely . I have scraped user profile information + time of the last post up to user id : 2374859 . The scraping took place in last days of September.

In this post I gathered the complete information of activity by " last post" and compared it to activity by " last active" , the comparison showed significant differences. I did not show the " Brand News" since they have not made any posts and their contribution to forum is almost non existent. Also fresh full stats by rank can be seen in the end of post.

As before, I used last 3 month period  and also looked as stats within this year ( from 1st of January to 24th of September)

Here are the number of users who have made posts in these 2 time frames by rank:

Including Newbies , there are total of 137122 users who have made a post in last 3 months, if we exclude Newbies , the number drops to 31211. If we take a look at how many posted within this year, with Newbies there are 313609 users and without them 46149 users.

Here is the activity by " last post"  in percent:

The percent graph shows that there is very high activity from Jr.Members, this is because of the last rank requirement change- most of inactive users where demoted to Newbie level.

Further I compared " last post " activity to " last active" activity of users , this shows us that there is a significant amount of users who have been reading the forum and haven't been participating in the discussions. It was quite surprising for me , I was not expecting so many users to be reading and not posting across all the ranks, especially the high ranks. Here is the comparison graph:

Here are the difference between number of users who have posted and those have been " last active" :

Here is the data I showed above in graphs with full users statistics , which shows that right now we have a lot more Members than Jr.Members around

If you have anything you would like to see with this current data, i would happy to hear your ideas.

   

[19113.]

Hello!
In my last post I looked at first 2 million users and their rank distribution and at user activity. This time I will show more complete distribution of user rank ad their activity in this forum. I have scraped users with their ID's up to 2312500. The complete distribution is shown below. I did not not just scrape IDs above 2 million and merge them with old data, I have completely re-scraped all the IDs for more accurate representation of situation.

I have not shown deleted skipped users in this distribution. The amount of deleted/skipped id's is 19113. Skipped ID amount depends on the errors while scraping data and in addition there is some amount deleted users, which is estimated to be around 16'000 users .

To compare 2 million users IDs to 2'312'500 ID's there is little surprise that the biggest addition to statistics are to Brand New users (230k) , Newbies (66k) and Jr (12k) .

In my last post there was a lot of discussion about the activity and how to define it. One of the best ways to define it would be by using time of the last post of any given users but this is technically challenging since it requires scraping users last post, this option right now is not supported by current scraper version. While I or anyone else haven't yet made these modifications to scraper in order to achieve this, we will have to get by with "last activity" data which is scraped from users profile. This time i decided to show also the activity of lower rank user activity. It seems to me that it is a good practice to define the activity as "last activity" in last 3 months in addition i thought it would be interesting to see activity within this year.

The total number of users who have had "last activity" within last 3 months is 425584. If we exclude Brand New's and Newbie's the number drops significantly and the number then is 82056.

Further lets look at the activity charts regarding this year.

The total number of users who have had "last activity" within this year is 985811.  This year represents approximately first 7 months of this year. If we exclude Brand New's and Newbie's the number then is 102186.

Since I just showed activity in 2 different timelines, I think most of us would like to see comparison also:

Links to some of my other posts:
 '2 million users and teir stats ' : https://bitcointalk.org/index.php?topic=4901670.msg44145551#msg44145551
'Lets go deeper in "brand new" and " newbie" statistics ' : https://bitcointalk.org/index.php?topic=4759325.msg42983972#msg42983972

[моём предыдущем посте]

Всем привет !

Я опубликовал этот анализ в мета разделе, но аудитория этого раздела ограничена, поэтому я решил запостить это еще и сюда. Если людям в этом разделе тоже будет интересно, то я буду делать больше постов здесь.
 
На момент написания этого поста на форуме зарегистрировано 2334998  пользователей, из которых я отобрал первые 2 миллиона с ID-1 до ID- 2 000 000. Двухмиллионный пользователь зарегистрировался 3 апреля 2018 года. Как многие из вас догадываются или знают, большая часть из пользователей имеют низкие ранги. Я рассматривал низкие ранги в моём предыдущем посте, в этом посте я хотел бы сделать больший упор на высших рангах, начиная с Jr. Member. Смысл этого поста в том, чтобы максимально просто показать распределение первых двух миллионов пользователей и уровень их активности.

Так как круговые диаграммы очень удобны и понятны, я использовал именно их. На них можно увидеть распределение пользователей bitcointalk с ID от 1 до 2'000'000 .

Кроме того, я изучил активность рангов Jr. Member и выше. Для того, чтобы определить активность пользователей, я пользовался  "Last Activity". Я отфильтровал даты активности по требуемым временным рамкам для пользователей с рангом Jr. Member и выше. Один график показывает распределение пользователей, которые были активны в прошлом месяце, а второй график - пользователей, которые хоть раз были активны за этот год включая прошлый месяц.

Среди выбранных рангов нашлось 61836 пользователей, которые были активны в прошлом месяце.

На этом графике мы видим, что доля пользователей, которые были активны хоть раз за этот год для большинства рангов колеблется от 70% до 80%. Доля таких пользователей среди рангов “Copper” и “Legendary” выше 80%, и это не удивительно, ведь
1) Таких пользователей относительно немного;
2) Легендарные пользователи – люди, которые больше всех интересуются криптовалютами;
3) Ранг “Copper member” был введен относительно недавно.

Если сложить всех пользователей, активных за последний год, начиная с Jr. Member и заканчивая Legendary и Copper, то мы получим 90993 пользователя. И это интересно, потому что может показаться, что на этом форуме гораздо больше активных пользователей, так как их зарегистрировано более двух миллионов.

Этот анализ был проведен на основе данных, которые были собраны в начале июля этого года. Думаю, что график “активные в прошлом месяце” не изменился бы, если бы мы взяли данные посвежее. Возможно, мы бы увидели чуть больше пользователей на графике «активные за этот год», но мне кажется, что в целом они и так неплохо отражают действительность.

Если кому-то интересно поиграться с этими данными, то вот они в более доступной форме:

P.S Если у вас есть какие-либо предложения насчет того, чтобы вы хотели бы видеть, я с радостью выслушаю все ваши идеи.

Ранг	Oк*    Апм**       %    A эг***    %
Jr.Members	60389        32035           53.05     47025         77.87
Members	27621        11772           42.62     18424         66.70
Full M.	18170        8959             49.31     12704         69.92
Sr.M.	8999          4441             49.35     6384          70.94
Hero	4140          2252             54.40     3139          75.82
Legendary	2144          1519             70.85     1892          88.25
Copper M.	1441          858               59.54     1425          98.89
Deleted	16000
Donator	109
Admin	2
Staff	26
VIP	37
Global moderator	3

*Общее кол-во   
**Активные в последний месяц         
 *** Активные за этот год

[my last post]

Hello! At the moment of writing this post, there technically are 2333176 users registered from which I have scraped 2 million users. The 2 millionth user was registered on 2018/04/03. As most of us suspect or know the biggest part of these users are with lower ranks, I have looked at lower ranks more extensively in my last post  thus in this post I wanted to focus more on Jr. Members and above. My main goal with this post is to show distribution of first 2 million users , give an insight to user activity level while keeping it simple as possible.

Since PIE charts are the most common and one of the most intuitive charts to read, here are the distribution of bitcointalk ID's from 1 to 2'000'000 .

In addition I looked at the activity of rank Jr. Member and above. This time I characterised the activity by "Last activity" in certain time period . One plot for users who's last activity was last month and one plot for those who has been last active this year including last month.

Between these ranks last activity within last month was seen for 61836 users.

At this plot we can see that most ranks last activity within this year was in range from 70 to 80% . But Legendary and Copper members are above 80%, this is not an surprise since
1) Users with these ranks are in relatively small numbers
2) Legendary users are the biggest crypto enthusiasts
3) Copper members are relatively new rank type

If we sum up all the user accounts with last activity within this year from Jr.Member to Legendary + Copper members , we get 90993 users. In my mind this is a interesting result because most of us could easily think that there are much more active users in this forum since there are 2million + registered users.

This analysis was made from data which was scraped in start of July this year. I believe that the " last activity:last month" plot wont change much if we had data which would have been scraped more recently. Possibly we would see a bit more users in the " last activity: this year" plot but overall I think that this is a good representation of the situation.

In case someone wants to play with this data, here it is in more accessible manner:

Rank	Full N    Act.last m. N       %    Act.this year N     %
Jr.Members	60389        32035           53.05     47025         77.87
Members	27621        11772           42.62     18424         66.70
Full M.	18170        8959             49.31     12704         69.92
Sr.M.	8999          4441             49.35     6384          70.94
Hero	4140          2252             54.40     3139          75.82
Legendary	2144          1519             70.85     1892          88.25
Copper M.	1441          858               59.54     1425          98.89
Deleted	16000
Donator	109
Admin	2
Staff	26
VIP	37
Global moderator	3

P.s. If you have any ideas or suggestions what you would like to see, i will be happy to hear your ideas.

[From first 2'000'000 users there are 1'443'323 "Brand New" and 417'536 "Newbie" users]

From first 2'000'000 users there are 1'443'323 "Brand New" and 417'536 "Newbie" users. Together these 2 positions amounts to 1'860'859 , this is 93% of all users !! This is huge amount and this information was seen already in my last posts*, where users started to speculate of possible reasons behind this huge new account number. Therefore I decided to take a closer look to these users. There are limited statistics which can be extracted from users profiles thus limited information what we can analyse.




Lets take a look at the biggest group of "users"or id's - "Brand New', term "users" does not seem right since almost all of them do not contribute to the forum in anyway. There are 1'443'323 "Brand New" users. The big question, why so many? My first instinct was to think that bitcointalk.org being an old ,influential domain which is highly ranked in search engines , is heavily abused for creating backlinks. I filtered those accounts which has backlinks in them and it turns out that they are not so many, but the number is notable 1,32% (19'051 Id's) (1.Graph) of " Brand New's" have them in their profiles. While going trough data, I noticed that there are profiles which has last activity " Never" , I filtered them and it turns out that they are a lot -10,3% (1.Graph) from all " Brand New's". I am not sure why these users have this kind of " Last Activity" in their profiles , my guess is that they are "nuked" by moderators .

Other stats that can be extracted from user profiles are: post count (2.Graph) and last activity (3.Graph). Most of these accounts have not made any posts , there is a small fraction that has made 1+ posts.

If we take a look at (3.Graph) ,we can see " Brand New" account activity representation, I calculated the difference between last activity and registration date , grouped  data and plotted it in respect to days.I excluded the 10,3% with last activity " Never" . We can see that about 76 percent of " Brand News"  analysed last active where less then 5 days after registration. 






I did similar analysis on " Newbie" accounts , Newbie accounts also has has some notable percentage of links in them, there are about 10 000 accounts with links in their profiles and most likely they are meant as back-links. (Graph.4) If we take a look at the post count (6.Graph), we can see that that about 32% of Newbies have made only one post and the rest have made more than 1 post. In the " Activity plot"  (5.Graph) we can see that 32,5% of all " Newbie"  accounts are last used 10 days or less after registration.


In conclusion,there is 1,16 million users which have abandoned their accounts after 10 or less days. Small fraction of all Newbie and Brand New accounts have links in their accounts , in majority of them , they are with purpose to use power of this forum to increase google positions of other web-pages , there also is notable percentage of accounts which have status " Never" in Last Activity. The biggest part of inactive accounts are still not explained, some of them could be sleeping spam accounts. In my last posts* users suggested that inactive accounts are consequences of IP bans.

*last posts:
1)https://bitcointalk.org/index.php?topic=4649770.msg41998196#msg41998196
2)https://bitcointalk.org/index.php?topic=3333773.msg34863787#msg34863787

In this post I have done position analysis on Bitcointalk users in range from id=1'000'000 to id=2'000'000. Users with this range of id's were registered from 2017/05/11 to 2018/04/03.

Without further ado, Here is the distrubution of postions of second million bitcointalk users

• Brand New   733503   73,35%
• Newbie         212864   21,28%
• Jr. Member    33395     3,34%
• Member        13682     1,36%
• Full Member  6334       0,63%
• Sr.Member     222        0,02%
• Hero               0            0%
• Legendary      0            0%

How we can see there is no Hero and Legendary users, this is not surprising since these positions requires high level of activity .

P.s. If you have any ideas on what kind of information you would like to see, let me know.

Here is the link to first part: https://bitcointalk.org/index.php?topic=3333773.msg34863787#msg34863787

[So about 90% of first 1mil users accounts are Brand new or Newbie.]

Scraped through first 1mil user IDs, end result was roughly 900K rows of users.
End result was so much smaller than IDs because my scrapper throws an exception if "Last active: Never" ('Never' isn't correct date format for a database).


Quick sorted these 900K users:

• 661K Brand New
• 190K Newbie
• 45K Jr. member
• 12K Member
• 11K Full Member
• 8K Sr. Member
• 2K Legendary
• 4K Hero member

So about 90% of first 1mil users accounts are Brand new or Newbie.

29K MMM Global spam accounts
4K BtcCoScammedMe* accounts

303K users last active this year (202K Brand new) this looks suspicious




 

I into a bounty hunting quiet while, particularly Twitter. In this period I created some tools to automate the process for me. And now wondering, if anyone would be interested in purchase my tools. probably it would be subscription based. What would be the price you willing to spend on such toolset? What other tools do you want/need?

• Automatic retweet of a specified user. in near future -  limit retweet count per day, to eliminate spam.
• Collect all the retweets, and generate a report from them. Post reports automatically
• Notifications about a new post by bounty manager in bounty thread.
• Can implement other features by request.