[TOOL] ⭐Bitcointalk BRDb score

[*Ace*]

I reactivated the scraper a few days ago, but last night I had to stop it again as I was no longer receiving data from Bitlist; therefore, the tool will not be retrieving any further data from Bitlist at present.
I am already working on creating a personal database by retrieving data directly from BitcoinTalk, but this will take a long time to synchronise with all users.
For the time being, I have no other alternatives.

[Lillominato89]

I reactivated the scraper a few days ago, but last night I had to stop it again as I was no longer receiving data from Bitlist; therefore, the tool will not be retrieving any further data from Bitlist at present.
I am already working on creating a personal database by retrieving data directly from BitcoinTalk, but this will take a long time to synchronise with all users.
For the time being, I have no other alternatives.

Come on mate, don’t give up – your tool is really useful for people like me who use the BRDb result as an indicator of the other person’s level.
If it stays static, it’s no longer any use!
I just want to encourage you to do your best, but I also realise it takes quite a bit of time.

[LoyceV]

I am already working on creating a personal database by retrieving data directly from BitcoinTalk, but this will take a long time to synchronise with all users.

I have some data What do you need?

[*Ace*]

I am already working on creating a personal database by retrieving data directly from BitcoinTalk, but this will take a long time to synchronise with all users.

I have some data What do you need?

What data do you have available?
Essentially, I need:
Total posts
Total merit points
Posts in the last 120 days
Merit points in the last 120 days
Registration date
And last login

The BRDB calculation is based on this data. I’ve started scraping the profiles, but it’s my first time doing this and it’s taking me ages

[un_rank]

I have some data What do you need?

By some, LoyceV means every available piece of data that you could possibly (and in some cases impossibly) need.

I do not think I have seen this tool before. It looks really good and of practical use to those not familiar with usernames here. I would prefer to have a website like with bpip does and not a script but this will give more real time data to those using it while on the forum.

- Jay -

[LoyceV]

What data do you have available?

[overview] LoyceV's useful data on Bitcointalk

Essentially, I need:
Total posts

I kinda count all posts, so including deleted ones. You'd be at 917 instead of 911.

Total merit points

I have, but only for earned Merit, and updated weekly: https://loyce.club/Merit/userID_sent_received.txt.

Posts in the last 120 days

I didn't keep track of this.

Merit points in the last 120 days

You could get a complete overview (updated weekly, based on theymos' weekly Merit data dump) from loyce.club/Merit/merit.all.txt.

Registration date

I don't keep track of those. You could get a estimate by skipping 10,000 profiles and only checking some of them. It gives you a range, but I don't think you care for the exact date if it's  years ago.

And last login

That's going to require frequent checking. I don't have this.

I’ve started scraping the profiles, but it’s my first time doing this and it’s taking me ages

With a maximum of 1 page load per second, you're looking at 1000+ hours just for checking all profiles.

[*Ace*]

I have some data What do you need?

By some, LoyceV means every available piece of data that you could possibly (and in some cases impossibly) need.

I do not think I have seen this tool before. It looks really good and of practical use to those not familiar with usernames here. I would prefer to have a website like with bpip does and not a script but this will give more real time data to those using it while on the forum.

- Jay -

It’s not just a userscript for the forum; there’s also a web version, just like bpip or bitlist.
For example, here’s your user profile: https://brdbscoreapi.ace-d89.workers.dev/profile/3471165
I realise that not everyone likes using a tool via a userscript.

What data do you have available?

[overview] LoyceV's useful data on Bitcointalk

Essentially, I need:
Total posts

I kinda count all posts, so including deleted ones. You'd be at 917 instead of 911.

Total merit points

I have, but only for earned Merit, and updated weekly: https://loyce.club/Merit/userID_sent_received.txt.

Posts in the last 120 days

I didn't keep track of this.

Merit points in the last 120 days

You could get a complete overview (updated weekly, based on theymos' weekly Merit data dump) from loyce.club/Merit/merit.all.txt.

Registration date

I don't keep track of those. You could get a estimate by skipping 10,000 profiles and only checking some of them. It gives you a range, but I don't think you care for the exact date if it's  years ago.

And last login

That's going to require frequent checking. I don't have this.

I’ve started scraping the profiles, but it’s my first time doing this and it’s taking me ages

With a maximum of 1 page load per second, you're looking at 1000+ hours just for checking all profiles.

Yes, it’s actually taking a long time to get the initial data. In fact, I’m not tracking all the forum UserIDs, just the ones my tool recognises so far – there are about 19,000+ of them – and it took about 26 hours to run the full check.
I’ll have a look at your data and see if it’s useful for my specific situation. Thanks, LoyceV

[*Ace*]

Right then, folks, I’ll soon be ready to migrate the BRDb scores to the new personal database.
However, this will have some consequences!
I don’t currently have the actual figures for posts and Merit over the last 120 days; I’ve only been retrieving this data since yesterday afternoon, so there will be a reduction in scores until everything is fully synchronised – this will take 120 days.😁

[*Ace*]

Update

The tool has been restarted and is now up and running again, but it is no longer directly linked to the Bitlist API. I would still like to thank Tryninja for providing the initial data.
The tool now has its own database; obviously, it is not as comprehensive as Bitlist or BPIP, but it will gradually be populated.
The data displayed on both the userscript and the website is updated in real time; only the graph will be updated from time to time – I believe the scraping cycle takes around 28 hours.
Unfortunately, for many profiles there has been a drop in posts and merits over the last 120 days, but this data will synchronise day by day. Profiles such as Fillippone’s, which have received thousands of merits, could not be retrieved via the page scrape because the worker timed out, so the realignment will take around 120 days.
I hope the tool proves useful to everyone; further updates will follow.

[dkbit98]

The tool has been restarted and is now up and running again, but it is no longer directly linked to the Bitlist API. I would still like to thank Tryninja for providing the initial data.
The tool now has its own database; obviously, it is not as comprehensive as Bitlist or BPIP, but it will gradually be populated.

Well I don't see any difference on my end, everything looks and works identical so far, and you are probably using similar data.
We dont need so much details anyway, and you can always add links for Bitlist and BPIP for more information.
This is already provided for people who are using BPIP extension.

[*Ace*]

The tool has been restarted and is now up and running again, but it is no longer directly linked to the Bitlist API. I would still like to thank Tryninja for providing the initial data.
The tool now has its own database; obviously, it is not as comprehensive as Bitlist or BPIP, but it will gradually be populated.

Well I don't see any difference on my end, everything looks and works identical so far, and you are probably using similar data.
We dont need so much details anyway, and you can always add links for Bitlist and BPIP for more information.
This is already provided for people who are using BPIP extension.

I still have some of the data I’ve collected from Bitlist over the last few months; I’ve kept it as a starting point, but I’ve now removed all references to the Bitlist API, with the exception of the Merit page – though that too will be removed as soon as my database has enough data.
It won’t be easy, and I don’t know if the Cloudflare worker can be used for this purpose for much longer.
At the bottom of the profile page there should be a reference to Bitlist, if I’m not mistaken, so this option is already available.

[Lillominato89]

I haven’t noticed any major changes in the scores either.
My graph has gone down, but over time – I reckon a lot of people don’t like me anymore xD, and neither does your tool, seeing as it’s reminding me of it!
Apart from that, it seems to me that the data is still the same; some things change and others perhaps don’t! I should pay more attention to this – perhaps not all the data displayed by the script is up to date in your database?

[*Ace*]

I haven’t noticed any major changes in the scores either.
My graph has gone down, but over time – I reckon a lot of people don’t like me anymore xD, and neither does your tool, seeing as it’s reminding me of it!
Apart from that, it seems to me that the data is still the same; some things change and others perhaps don’t! I should pay more attention to this – perhaps not all the data displayed by the script is up to date in your database?

In theory, everything should be calculated correctly now, but it’s still a work in progress.
Calculations can sometimes play tricks on you; for example, your graph has been falling over time, but the day before yesterday it dropped sharply, whereas today it’s gone back up.
I need to figure out what the worker is doing wrong or what’s going wrong with the scraper.
It’s obviously something to keep an eye on.

[DYING_S0UL]

I haven’t noticed any major changes in the scores either.
My graph has gone down, but over time – I reckon a lot of people don’t like me anymore xD, and neither does your tool, seeing as it’s reminding me of it!
Apart from that, it seems to me that the data is still the same; some things change and others perhaps don’t! I should pay more attention to this – perhaps not all the data displayed by the script is up to date in your database?

In theory, everything should be calculated correctly now, but it’s still a work in progress.
Calculations can sometimes play tricks on you; for example, your graph has been falling over time, but the day before yesterday it dropped sharply, whereas today it’s gone back up.
I need to figure out what the worker is doing wrong or what’s going wrong with the scraper.
It’s obviously something to keep an eye on.

I checked several users score and many of them (including me) had that sharp decline (like a heartbeat ). Not sure what caused that, but then again, it went back to the previous score, a flat line.
Maybe it made a temporary error calculating that lead to this?
Others (mostly) seems to have steady decline (not that big).

Anyway, anything new in the pipeline?  

[*Ace*]

I haven’t noticed any major changes in the scores either.
My graph has gone down, but over time – I reckon a lot of people don’t like me anymore xD, and neither does your tool, seeing as it’s reminding me of it!
Apart from that, it seems to me that the data is still the same; some things change and others perhaps don’t! I should pay more attention to this – perhaps not all the data displayed by the script is up to date in your database?

In theory, everything should be calculated correctly now, but it’s still a work in progress.
Calculations can sometimes play tricks on you; for example, your graph has been falling over time, but the day before yesterday it dropped sharply, whereas today it’s gone back up.
I need to figure out what the worker is doing wrong or what’s going wrong with the scraper.
It’s obviously something to keep an eye on.

I checked several users score and many of them (including me) had that sharp decline (like a heartbeat ). Not sure what caused that, but then again, it went back to the previous score, a flat line.
Maybe it made a temporary error calculating that lead to this?
Others (mostly) seems to have steady decline (not that big).

Anyway, anything new in the pipeline?  

When I migrated to the new scraper, I unfortunately forgot to include some data needed for the BRDb count.
This led to a drastic drop in numbers for all users; by the time I realised, it was already too late – almost all the UserIDs had already been scraped – but I think I’ve sorted that problem out now.