[TELEGRAM] Yet Another BitcoinTalk Notification BOT (merits, mentions, topics,+)

[Mitchell]

This looks like something I've been looking for since Piggy's bot went down. Thank you TryNinja!

[1715177759]

1715177759

[1715177759]

1715177759

[cygan]

@TryNinja
might be helpful for the other users, if you leave the link in your op behind the post for the respective translations

...
Thanks to:
@Piggy for his first bot, which originated this era of telegram bots in the forum.
@giammangiato for his bot, where I took some ideas and message design.
@csmiami for the Spanish translation.
@sheenshane for the Filipino translation.
@mole0815 for the German/Deutsch translation.
@mikhailr for the Russian translation.
@fillippone for the Italian translation.
@cygan for the Polish translation.
@masulum for the Indonesian translation.

like
@cygan for the Polish translation.

what you think?

BTCump

[LoyceV]

Btw, I managed to get your files in my HD (after ~1 hour of extraction and some freezing) and to make a scrapper that loops through them and makes an entry in the database. But I then noticed that they do not contain a title (only some old ones?) or exact date, so they can't just be inserted in my DB the way it is structured now. I'm thinking about making a page and table just for your posts, so you can at least be able to search by content and author (not sure if that can be done easily with 2m+ posts).

For the old posts I hadn't decided on the exact format yet, hence the title.
I have other topic titles too, I use them here. I have just over 100,000 titles and can get you a text file if it helps. But: those are the topic titles as I scraped them once. The title may have changed later, and each post can have a different title too.
The date in my files is my scrape-date, for most posts it's only a few seconds after the post was created.

I sometimes use a "local" search on the server, looping through a few million posts, but that takes a while. It would be nice if a database can allow quick searches through a web interface.

[mikhailr]

~
Thanks to:
~ list ~

BTCump

You understand that you not only bump post, but also sent a notification in a TelegramBot to everyone who is mentioned in this list  

[Little Mouse]

You have way to build watch list, but it would be pretty good if there was a way to stop getting information from certain topic. I am bored of receiving notification from a topic. I do not want to get any notification from tha topic.

[TryNinja]

You have way to build watch list, but it would be pretty good if there was a way to stop getting information from certain topic. I am bored of receiving notification from a topic. I do not want to get any notification from tha topic.

It's the next feature. I will work on it tomorrow.


Had a small downtime with the scrapper today since I was doing some heavy migrations in the database. But it was worth it:

Now, the https://posts.ninjastic.space has all* of Loyce's 2,5 million archive posts saved (*with the rare exception of a very few ones). Most of them have no title, but you can still search by topic id, content, author and date. Expect some graphic bugs since I wasn't planning on having that many data that early. And please be gentle with the database (text content takes some time and it's heavy work).

[cygan]

which other feature i would like to see on this bot is that just like the merit/s, an announcement is also displayed for a trust (positive, neutral or negative) that a user has just received from another bitcointalk user 

i think this option is also possible or TryNinja?

[TryNinja]

which other feature i would like to see on this bot is that just like the merit/s, an announcement is also displayed for a trust (positive, neutral or negative) that a user has just received from another bitcointalk user  

I honestly don't think this is needed/worth the time. Trust feedback is something that is given to people only a few times in a lifetime. It's more workload for the bot in exchange for very few advantages.

I just got two notifications for your mention, so I freaked out thinking I broke something or the bot wasn't handling a 4,6 GB table. But in the end, it was because I forgot to clean my local db before migrating the data, so it had 2 TryNinjas. Ok, now I can sleep.

[LoyceV]

Now, the https://posts.ninjastic.space has all* of Loyce's 2,5 million archive posts saved (*with the rare exception of a very few ones). Most of them have no title, but you can still search by topic id, content, author and date. Expect some graphic bugs since I wasn't planning on having that many data that early. And please be gentle with the database (text content takes some time and it's heavy work).

I'm not sure if it's still loading or stuck, just searching for username "LoyceV" or Topic ID "1813624" keeps loading indefinitely.

To raise the stakes a bit, I have much more posts, although it'll take a few more months to complete scraping. I expect to be done around August, so if you can handle a lot more data to search by then, you can have it all It's currently 11 GB using bzip2 compression.
I didn't keep the original post dates for this though, but you can always get a raw estimate by scraping one in every 1000 posts.

[TryNinja]

I'm not sure if it's still loading or stuck, just searching for username "LoyceV" or Topic ID "1813624" keeps loading indefinitely.

Not sure why. I just tried searching your username in both “username” and “text search” field and it didn’t take longer than 15 seconds. Can you try again?

edit: Some searches do take some time. I tried doing 3 tabs at the same time: text "theymos", text "LoyceV" and just the last posts from the home page. It took close to 35~50 seconds in the text ones (they are not optimized and search the whole database). I will see if I can make it better.

edit 2: it should be slower now while this edit is here since I'm creating some db indexes.
edit 3: I think it's over. It should be a lot faster now.

It's currently 11 GB using bzip2 compression.
I didn't keep the original post dates for this though, but you can always get a raw estimate by scraping one in every 1000 posts.

I would need to ask my host for some additional storage.

[TryNinja]

Bump!

Some (almost) real time stats about the bot: https://grafana.ninjastic.space

[fillippone]

Bump!

Some (almost) real time stats about the bot: https://grafana.ninjastic.space

Great work! Not only your Bot has been running smoothly since inception, but also you are adding features on a constant basis. That grafana dashboard is very cool. Useful for the average user? Not sure, but cool surely it is!

Ok, time for a little request.
Do you think you can squeeze the total merit number {merits} in your notification message?

I would do something like:

MERITS <b> $ {merits} </b>. You received <b>${amount}</b> merit(s) from <b>${sender}</b> for <a href="${link}">${title}</a>

Or even better (it's a trivial condition, but I thin the result is greatly improved):

if amount=1:
MERITS <b> $ {merits} </b>. You received <b>${amount}</b> merit from <b>${sender}</b> for <a href="${link}">${title}</a>

else:
MERITS <b> $ {merits} </b>. You received <b>${amount}</b> merit(s) from <b>${sender}</b> for <a href="${link}">${title}</a>

EDIT: if it is not realtime, it is ALMOST realtime.
EDIT2: Corrected a typo, but csmiami was quicker than me!
EDIT3: Also Tryninja himself was quicker than me. I am a real person, I am still working on my alien AI to interact with forum.

[Csmiami]

Or even better (it's a trivial condition, but I thin the result is greatly improved):

if amount=1:
MERITS <b> $ {merits} </b>. You received <b>${amount}</b> merit from <b>${sender}</b> for <a href="${link}">${title}</a>

else:
MERITS <b> $ {merits} </b>. You received <b>${amount}</b> merit from <b>${sender}</b> for <a href="${link}">${title}</a>

I beg for the pardon of the coders and techy guys; but I don't see any difference between those 2 lines...

Adding the total merit count to the notification is a great suggestion tough

[TryNinja]

~

I can try that. Will add to the TODO list. The only possible "issue" I can see happening is maybe not showing the exact merit count resulted from each specific transaction (since people can merit you more than once before the bot sees it). E.g: I have 100 merits. I receive 1 merit and 2 merits in a space of 5 seconds. The bot scrapes them and sees you received 1 and then 2. But the message will show only the last merit count of your profile (since that's probably what I will scrape). So it will show:

- MERITS 103. You received 1 merit from X...
- MERITS 103. You received 2 merits from Y...

Instead of

- MERITS 101. You received 1 merit from X...
- MERITS 103. You received 2 merits from Y...

But I don't think this would be an major issue, right?

I beg for the pardon of the coders and techy guys; but I don't see any difference between those 2 lines...

I think he meant:

if amount=1: You received ${amount} merit
else: You received ${amount} merits

Or just take the s altogether.

P.S: I'm still going to add the ignore option (and new features) but I have been wasting my time with other not-that-important stuff that I just thought it was cool. (hello Grafana dashboard...)

Sorry for those wanting it as fast as possible. =P

[libert19]

I didn't get quote mention for this quote (I use also couple other, no notification), something to look into.

https://bitcointalk.org/index.php?topic=5248514.msg54467679#msg54467679

[LoyceV]

Not sure why. I just tried searching your username in both “username” and “text search” field and it didn’t take longer than 15 seconds. Can you try again?

It works better now. I'm glad you're putting "my" data to good use, a search function makes it much more useful.

I didn't get quote mention for this quote (I use also couple other, no notification), something to look into.

You weren't quoted in the original post, which is what (all?) scrapers use.
If it's edited within 10 minutes, the forum won't show the post as edited (and doesn't keep the original).

[Halab]

You can use it here: https://posts.ninjastic.space

I hadn't seen this tool, and it's much more powerful than the search function in the forum. Thank for that.
Only one small problem, there is a problem with french accents (à é è ô, etc...)(and probably with special characters from other languages).
For example :
Tu as plutôt bien cerné le système.
Tu as plutýt bien cerný le systýme.

Translation help needed:
I really appreciate any help as I only did English, Portuguese and French (with the small knowledge I have + some translation tools - sorry native speakers, PM me if any mistakes).

I don't use TG, so I can't tell if there are any errors in the french translation. But if you didn't get help from a french member, you can PM me the translation you did and I will correct it if necessary.

[TryNinja]

I hadn't seen this tool, and it's much more powerful than the search function in the forum. Thank for that.
Only one small problem, there is a problem with french accents (à é è ô, etc...)(and probably with special characters from other languages).
For example :
Tu as plutôt bien cerné le système.
Tu as plutýt bien cerný le systýme.

Unfortunately, some old posts I imported from Loyce's database are showing like this and parsing them again with a potential fix is really hard and time consuming. New ones are ok (here is yours).

I don't use TG, so I can't tell if there are any errors in the french translation. But if you didn't get help from a french member, you can PM me the translation you did and I will correct it if necessary.

Merci! I'll PM you. It's quite small so it shouldn't take much of your time. I had just started studying french a bit before the lockdown but I'm still far from good.

[dkbit98]

@TryNinja after you update it with new features, maybe you should rename the bot to 'Ultimate Bitcointalk Notification BOT' 
I have been using it for several days, and so far it is working great.
Is it possible to port it in Discord also?

[Nellayar]

Because of these telegram bots made by tryninja, greatarkansas and giammangiato. I already memorized my Bitcointalk UID. LoL. Anyway, it's great to use all of your bots because they all have different features and it helps me a lot in notifying posts, merits and trusts(though I don't have trust).

BTW, the translation for my own language amazed me. I just want to modify some of words.

The word, "Topics" is equivalent to "mga Paksa". "Show ignore list" is translated to "Ipakita ang Ignore List". Well, it's up to you if you will consider that change. Thanks



Once again, nice work OP.