Bitcoin Forum
May 04, 2024, 12:47:54 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: JSON API for merit data  (Read 428 times)
hatshepsut93 (OP)
Legendary
*
Offline Offline

Activity: 2968
Merit: 2145



View Profile
September 02, 2019, 04:27:21 PM
 #1


I have created a public API for merit data, based on LoyceV's log of merit transactions, which is based on theymos's merit dumps. The data is updated once per week.

I didn't tune the rate limits yet, cause I don't know what kind of load can I expect, so refer to the headers if needed.

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
"The nature of Bitcoin is such that once version 0.1 was released, the core design was set in stone for the rest of its lifetime." -- Satoshi
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
1714783674
Hero Member
*
Offline Offline

Posts: 1714783674

View Profile Personal Message (Offline)

Ignore
1714783674
Reply with quote  #2

1714783674
Report to moderator
hatshepsut93 (OP)
Legendary
*
Offline Offline

Activity: 2968
Merit: 2145



View Profile
September 17, 2019, 04:58:07 PM
Merited by LoyceV (10), hugeblack (2), ABCbits (1), o_e_l_e_o (1), tranthidung (1), Husna QA (1)
 #2

Here's a userscript that utilizes my API do display earned merit of every poster just bellow their total merit score.

Code:
// ==UserScript==
// @name     Display earned merit
// @version  1.0
// @grant    none
// @include        https://bitcointalk.org/index.php?topic=*
// @run-at         document-end
// ==/UserScript==

const getMeritsFromApi = async function(ids) {
    return await fetch(`https://bct-quote-notifier.herokuapp.com/api/merit/v1/profiles/?ids=${ids.join(",")}`).then(r =>
        r.json()
    )
}

const insertEmerit = function(meritNode, eMerit) {
    try {
        let eMeritNode = document.createElement("text")
        meritNode.after(eMeritNode)
        meritNode.after(document.createElement("br"))
        eMeritNode.textContent = `eMerit: ${eMerit}`
    } catch (e) {
        console.error(e)
    }
}

const main = async function() {
    let posters = new Map([])
    for (const node of document.querySelectorAll(".poster_info > .smalltext")) {
        const meritNode = [...node.childNodes].find(e => e.textContent.match(/Merit/))

        const userId = [...node.childNodes].find(e => e.href && e.href.match(/action=profile;u=/)).href.split(";u=")[1]

        if (meritNode && userId) {
            let poster = posters.get(userId)
            if (poster) {
                poster.meritNodes.push(meritNode)
            } else {
                posters.set(userId, { meritNodes: [meritNode] })
            }
        }
    }

    let earnedMerits = await getMeritsFromApi([...posters].map(poster => poster[0]))

    earnedMerits
        .filter(e => e !== null)
        .forEach(e => {
            let poster = posters.get(e.id.toString())
            if (poster) {
                poster.earned = e.earned
            }
        })

    for (let [, poster] of posters) {
        try {
            poster.meritNodes.forEach(node => {
                insertEmerit(node, poster.earned || 0)
            })
        } catch (e) {
            console.error(e)
        }
    }
}

main()



.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
LoyceV
Legendary
*
Offline Offline

Activity: 3304
Merit: 16587


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 17, 2019, 05:38:03 PM
Merited by hatshepsut93 (1)
 #3

Are you still using my weekly merit.all.txt for this, or are you also scraping Recent merits on a regular basis to show near real-time results? Every 10 minutes will probably be enough for this.

hatshepsut93 (OP)
Legendary
*
Offline Offline

Activity: 2968
Merit: 2145



View Profile
September 17, 2019, 05:44:10 PM
 #4

Are you still using my weekly merit.all.txt for this, or are you also scraping Recent merits on a regular basis to show near real-time results? Every 10 minutes will probably be enough for this.

I'm using your merit.txt, didn't even know we finally have a better merit log. Are you going to continue to update your merit.txt?

I could change my API to scraping recent merits if there was some interest from the public, so far I got 0 requests in the last week  Embarrassed

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
DdmrDdmr
Legendary
*
Offline Offline

Activity: 2310
Merit: 10758


There are lies, damned lies and statistics. MTwain


View Profile WWW
September 17, 2019, 05:55:38 PM
 #5

<…> I could change my API to scraping recent merits if there was some interest from the public, so far I got 0 requests in the last week <…>
The issue is probably more on the usability side, than the readiness of it being up-to-date right down to the most recent TXs. The profile and TX data references the inner forum IDs, which is fine, but anyone who is willing to exploit the data still needs to retrieve/scrape the mappings of the UserIDs to names and message IDs to the message hierarchy (i.e. forum path (section, subsection, etc.), post name).
LoyceV
Legendary
*
Offline Offline

Activity: 3304
Merit: 16587


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
September 17, 2019, 06:30:15 PM
 #6

I'm using your merit.txt, didn't even know we finally have a better merit log. Are you going to continue to update your merit.txt?
As far as I know, it's been there since the introduction of Merit. But I'll keep doing my weekly updates anyway.

hatshepsut93 (OP)
Legendary
*
Offline Offline

Activity: 2968
Merit: 2145



View Profile
December 31, 2019, 02:17:32 PM
 #7

I have started scraping recent merit transactions, so the accuracy of my API will be significantly improved. I still don't guarantee it being 100% accurate, and at worst you can expect it to be only accurate up to the previous week.

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
tranthidung
Legendary
*
Offline Offline

Activity: 2268
Merit: 4005


Farewell o_e_l_e_o


View Profile WWW
January 01, 2020, 02:24:04 AM
 #8

I appreciated your decent time to code the scripts for users but if I was you, I would not going to this one. Users, personally know how many sMerits they have at specific point of time. While if one user is curious how many sMerits left the others have, they might choose to use your scripts and your JSON API for merit data.

I don't say negatively about it but I wonder what is the true reasons to look at the others' unused sMerits (to beg for merits, something like that).

BPIP provided that statistic during a very long period till the day it stopped showing that one. I don't know why Vod decided to stop providing it. As I remembered, BPIP did not provide that statistic before it shown issues with data scraping recent months.

Anyway, thank you for the script and happy new year, @hatshepsut93.

▄▄███████▄▄
▄██████████████▄
▄██████████████████▄
▄████▀▀▀▀███▀▀▀▀█████▄
▄█████████████▄█▀████▄
███████████▄███████████
██████████▄█▀███████████
██████████▀████████████
▀█████▄█▀█████████████▀
▀████▄▄▄▄███▄▄▄▄████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀
.
 MΞTAWIN  THE FIRST WEB3 CASINO   
.
.. PLAY NOW ..
hacker1001101001
Sr. Member
****
Offline Offline

Activity: 1288
Merit: 415


View Profile
January 01, 2020, 07:49:46 AM
 #9

I don't say negatively about it but I wonder what is the true reasons to look at the others' unused sMerits (to beg for merits, something like that).

I agree, sMerits data scraping is Ok, but making that public to the level of viewing it under user's profile seems a bit off the track. Theymos would always appreciate keeping it as much as hidden as he can if you see.

I will not be posting a definitive list of merit sources (so that people don't bug them too much), though you'll soon figure out who they are if you pay attention.

Still a good peice of work, and I appreciate the efforts of the OP. But ya making more info easily accessible is not always profitable to the community overall. Merit Sources would just be fed up with begging requests.
hatshepsut93 (OP)
Legendary
*
Offline Offline

Activity: 2968
Merit: 2145



View Profile
January 01, 2020, 12:33:06 PM
 #10

I appreciated your decent time to code the scripts for users but if I was you, I would not going to this one. Users, personally know how many sMerits they have at specific point of time. While if one user is curious how many sMerits left the others have, they might choose to use your scripts and your JSON API for merit data.

I don't say negatively about it but I wonder what is the true reasons to look at the others' unused sMerits (to beg for merits, something like that).

BPIP provided that statistic during a very long period till the day it stopped showing that one. I don't know why Vod decided to stop providing it. As I remembered, BPIP did not provide that statistic before it shown issues with data scraping recent months.

Anyway, thank you for the script and happy new year, @hatshepsut93.

Huh? Where did you get the idea that I provide any data related to sMerit? I only provide data for merit transactions and merit profiles, and merit profiles have earned and sent merit, and that's all.

As for BPIP, the difference is that this is an API rather than an app, the goal of API is for other people to build on top of it. I have created a userscript that displays user's earned merit below their actual merit in threads, by requesting this data from this API. One can create some merit statistics tool based on my API without having to do any scraping themselves.

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
tranthidung
Legendary
*
Offline Offline

Activity: 2268
Merit: 4005


Farewell o_e_l_e_o


View Profile WWW
January 01, 2020, 02:23:22 PM
 #11

Huh? Where did you get the idea that I provide any data related to sMerit? I only provide data for merit transactions and merit profiles, and merit profiles have earned and sent merit, and that's all.

As for BPIP, the difference is that this is an API rather than an app, the goal of API is for other people to build on top of it. I have created a userscript that displays user's earned merit below their actual merit in threads, by requesting this data from this API. One can create some merit statistics tool based on my API without having to do any scraping themselves.
I am deeply sorry to you. I mis-remembered what I saw in the image you posted with your script, in the second post. That shows eMerits, not sMerits. Once again, I am sorry because of my carelessness.  Cry


Ok, now I understood you do.

As being said, even if you provide sMerits with your script, there is nothing wrong from you. When people (users) want to use any tool, they can do it positively or negatively, you can control that.

▄▄███████▄▄
▄██████████████▄
▄██████████████████▄
▄████▀▀▀▀███▀▀▀▀█████▄
▄█████████████▄█▀████▄
███████████▄███████████
██████████▄█▀███████████
██████████▀████████████
▀█████▄█▀█████████████▀
▀████▄▄▄▄███▄▄▄▄████▀
▀██████████████████▀
▀███████████████▀
▀▀███████▀▀
.
 MΞTAWIN  THE FIRST WEB3 CASINO   
.
.. PLAY NOW ..
akhjob
Sr. Member
****
Offline Offline

Activity: 1050
Merit: 416


Buy Bitcoin


View Profile WWW
January 02, 2020, 05:57:08 PM
 #12

Here's a userscript that utilizes my API do display earned merit of every poster just bellow their total merit score.

Code:
-skip-
[img]https://i.imgur.com/wOHHo8P.png[/img]

I think the data provided by the userscript is wrong for almost all users. Take your profile for example:



It shows Merit = 1040, eMerit = 529, that means the airdropped merit is 1040-529=511, which is not possible.



My Profile shows Merit = 220, eMerit = 210, so Airdropped Merits = 10, but that's not the case, I started with 0 merits
LoyceMobile
Hero Member
*****
Offline Offline

Activity: 1655
Merit: 687


LoyceV on the road. Or couch.


View Profile WWW
January 02, 2020, 07:00:36 PM
 #13

I think the data provided by the userscript is wrong for almost all users.
It's updated once a week, so the difference is the merit earned since last Friday morning.

LoyceV on the road Advertise here for LN Don't deal with this account (exception)
Advertise here for LN Tip my kids Exchange LN (20 coins). 1% fee. No KYC <€50/month
My useful topics: Meritt & Trust & Moreee Art Advertise here for LN Foru[url=https://bitcointalk.org/m
dzungmobile
Sr. Member
****
Offline Offline

Activity: 714
Merit: 369


I stand with Ukraine!


View Profile
January 02, 2020, 11:24:47 PM
 #14

OP has started to scrap recent merit transactions to improve the accuracy, and not only depends on weekly merit data dumps from LoyceV - that is what OP intitially depended upon.

███████████████████████████████▀▀▀▀
███████████████████████████████
█████████▀▀▀▀▀█▀█▀▀▀▀▀█████████
███▄▀▀▀   ▄▄▄▄   ▄▄▄▄   ▀▀▀▄███
███████▀▀▀████▌ ▐████▀▀▀███████
█████▀███▀█▀██▌ ▐██▀█▀███▀█████
███████▀▄▀▄███▌ ▐███▄▀▄▀███████
█████▄██▄██▄██   ██▄██▄██▄█████
███████▄▄▄████   ████▄▄▄███████
██████████▀▀▀▀   ▀▀▀▀██████████
██████████▄▄▄▄▄▄▄▄▄▄▄██████████
███████████████████████████████
███████████████████████████████▄▄▄▄
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
.
TRUST DICE
.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
█▀▀▀











█▄▄▄
#1 RATED CRYPTO
CASINO IN THE WORLD
██ ██ ██ ██ █Trustpilot
▀▀▀█











▄▄▄█
▄█████████████████████████████
██████████████████▀▀█████▀▀████
█████████████████▀█████████▀███
██████████████████████████████
███████████████████████████▄███
█████████████████████████▄▄████
███████████████████████████████
█████████████░░░███████████████
███████████░░░█████████████████
█████████░░████████████████████
█████░░░██████████████████████
███░░█████████████████████████
▀░░░█████████████████████████▀
█▀▀▀











█▄▄▄
▀▀▀█











▄▄▄█
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!