[ANN]Merit API

[PrimeNumber7]

There appears to be demand for quick stats about merit.

As such, I am releasing an API that will allow you to quickly retrieve information about merit a person has sent, received, received recently, and that has been sent between users.

You can access the API by performing the following steps:
Go to https://bitcointalk-merit-api.herokuapp.com/
Go to the URL of one of the endpoints described on the above page.

I will try to keep this thread updated, however, the above page will contain up to date information about available routes.

The following routes are currently available:
/api/v0.1/number_received/[uid]
/api/v0.1/received_recent/
/api/v0.1/number_sent/[uid]
/api/v0.1/between/[uid_from]/[uid_to]
/api/v0.1/received/[uid]
/api/v0.1/sent/[uid]

[uid] is the UID of the account you are querying. [uid_from] is the UID of the account that sent merit, and [uid_to] is the UID of the account that received merit.

number_received - returns the number of merit the account has received
received_recent - returns the number of merit the account has received in the last 120 days
number_sent - returns the number of merit the account has sent
between - returns the total number of merit sent from [uid_from] to [uid_to], and a list of transactions that contain the time, date, day of week and post id of the merited post
received - returns the total number of merit received and a list of transactions that contain the time, date, day of week and post id of the merited post
sent - returns the total number of merit sent and a list of transactions that contain the time, date, day of week and post id of the merited post

The code is available at https://github.com/numberedprime7/merit-api

The merit data is as of the most recent merit data dump (usually happens on Saturdays).

Suggestions for improvements and bug reports are welcome.

As an example as to how you access the endpoints, below is an example of how to access all the merit I have sent:
https://bitcointalk-merit-api.herokuapp.com/api/v0.1/sent/2561166

to do -
write readme on github
create HTML file/render_template code to display endpoints/descriptions better
allow for lists of UIDs to be passed
consider improving dictionary format

[1715063218]

1715063218

[1715063218]

1715063218

[Initscri]

Looks good, good work.

Only thing I would suggest is to change the keys of the returned parameters to be something a tad bit more programmatic friendly.

For example, the following JSON from the following endpoint: https://bitcointalk-merit-api.herokuapp.com/api/v0.1/number_received/2561166

Code:

[{"Total Received Merit":528,"uid":2561166}]

Could, and should IMO be restructured to:

Code:

[{"total_received_merit":528,"uid":2561166}]

I'm not a huge fan of having to parse an array and then having to type keys w/ spaces in. Just a minor rec though.

Otherwise, looks and functions well! Good job!

[Chikito]

Nice, your API worked as well. I tried using it.

I read your script https://github.com/numberedprime7/merit-api/blob/master/merit2sql.py#L13 use loyceV data, the last update is 13 March 2020.

When attempting to see my received merit https://bitcointalk-merit-api.herokuapp.com/api/v0.1/received/938833 the last update is 2 weeks ago (6th March 2020)

Code:

Total Received Merit	604
Month	"March"
Post	"2818787.msg53754362"
Sent from	831523
number of merit	1
time	"2020-03-06 02:24:09"

Maybe isn't reflected into loyce merit data

[PrimeNumber7]

I'm not a huge fan of having to parse an array and then having to type keys w/ spaces in. Just a minor rec though.

I updated the output so that previous spaces is now replaced with "_"

Ultimately, the keys are strings, regardless of spaces.

I updated the DB.

I am pondering a more robust way to automatically update the DB. I updated the script that pulls the merit data (that I did not commit) that drops then (re)creates the table, and loops through each row. (thinking out loud) I think a better approach would be to query the number of rows my DB currently has, and start adding rows at the appropriate index. Once my script to update the DB is better, I can create a function that will update the DB if it was last updated more than 7.25 days ago using a global variable that keeps track of when it was last updated. I am short on time now, and will probably spend some time updating the code more this weekend or later this week.

Thanks for your input DroomieChikito and Initscri

[OmegaStarScream]

I've made something similar to LoyceV's [So.... on which board did you earn (or spend) most of your (s)Merit?] using your API. It's far from being a good code but it works (.NET Core). Hope you don't mind me sharing it here:

Code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using AngleSharp;
using AngleSharp.Html.Dom;
using Newtonsoft.Json;

namespace Merit_Stats
{
    class Program
    {

        public class Transaction
        {
            public string Day_of_Week { get; set; }
            public string Month { get; set; }
            public string Post { get; set; }
            public string post_path { get; set; }
            public int Sent_from { get; set; }
            public int number_of_merit { get; set; }
            public string time { get; set; }

        }

        public class Root
        {
            public int Sent_to { get; set; }
            public int Total_Received_Merit { get; set; }
            public List<Transaction> Transactions { get; set; }

        }

        static async System.Threading.Tasks.Task Main(string[] args)
        {


            var json = "";
            List<int> sumOfTotals;

            using (WebClient wc = new WebClient())
            {
                
                json = wc.DownloadString("http://bitcointalk-merit-api.herokuapp.com/api/v0.1/received/375981");
            }
            var obj = JsonConvert.DeserializeObject<List<Root>>(json);

            int i = 1;
            foreach (var item in obj)
            {
                Console.WriteLine(item.Transactions.Count);
                foreach (var child in item.Transactions)
                {
                    

                    Thread.Sleep(1000);
                    child.post_path = await GetBoardID("https://bitcointalk.org/index.php?topic=" + child.Post);
                    Console.WriteLine(child.Post + "(" + i + ")");
                    i++;
                   
                }

            }
            var list = obj.SelectMany(r => r.Transactions)
                          .GroupBy(t => t.post_path)
                          .Select(g => new { Board = g.Key, TotalLikes = g.Sum(t => t.number_of_merit) })
                          .ToList();


            foreach (var item in list)
            {
                Console.WriteLine("Received {0} in {1}", item.TotalLikes, item.Board);
            }



        }

        static async Task<string> GetBoardID(string post_id)
        {
            var config1 = Configuration.Default.WithDefaultLoader().WithDefaultCookies();
            var UserProfile = post_id;
            var context1 = BrowsingContext.New(config1);
            var document1 = await context1.OpenAsync(UserProfile);      
            var boardsNames = document1.QuerySelectorAll("div#bodyarea div div.nav b a.nav").SkipLast(1);


            var links = boardsNames.Select(m => m.TextContent);         
            var count = links.Count();
            var loop = 0;
            StringBuilder childBoards = new StringBuilder();
            foreach (var item in links)
            {
            
                childBoards.Append(item.ToString() + " > ");
            }
            return childBoards.ToString();
        }
    }
}

Make sure to not change the Thread.Sleep part to a lower value otherwise your IP will get banned.