Bot to automatically check signatures.

[lulucrypto]

Hello to all,

Note : This post is a translation from my French post.

It's been a while since I'm this topic, as I understand how it works, I started to tell myself that it would be possible to develop a bot to automate all that!

So this project was born.
I did not really give him a specific name, but to put it simply it's a bot that will automatically check the signatures posted by users on the topic mentioned above.

The operation is quite simple, the bot detects the requests ( By analyzing the last messages of the topic ), when it detects a request not treated by another user, it will launch the verification process.
To do this, the bot starts in a first to check the signature using Brainwallet.
If the signature is correct, then a screenshot of the site is taken. Note that if the check is unsuccessful, the bot stops there and does not respond automatically ( it may change later ).
Once the verification is complete, the bot post a first message, with the Quote of the message of the person who made the request, as well as the Brainwallet link + a link to the Brainwallet screenshot.
After that, it goes on archiving the request of the user ( thus integrating the response of the bot ).
He then finishes by editing his post, integrating a link to the quote and a link to the archive.

Some tips to know for the proper functioning:
- Please respect this signature format, otherwise the bot will not detect your request.
- Currently, the original addresses ( Legacy ) and SegWit, are processed by the bot.
- SegWit addresses are processed using this verification method.
- You can ask that the bot does not check your message, simply by adding "no-bot" somewhere in your message.

By the way, I take this opportunity to ask you something, do you know a website to verify the signatures created by P2SH addresses ?

For those who are interested in the technical part, here are some details:
- The bot is developed with Node.JS.
- Packages used :
-- Puppeteer ( Manage the posts, take screens ).
-- Request ( Scraper the latest messages + API calls ).
-- Jsdom ( Help scraping ).
-- Jquery ( Help scraping ).
-- Mysql ( Save the data the proper operation of the bot ).
-- Fs ( To retrieve the screens ).

And so from now on, the bot is launched and functional !

Well that's for the party bot "private", but following the request of some, I decided to develop another bot, this public and open-source!
This second bot is not for verification, but will serve as additional "security". Let me explain.

The problem with my checking bot is that it will centralize a lot of checks ( And lots of quotes ). Which would make this account a real target for hackers.
To put it simply, the purpose of the bot is to archive all messages of the subject.
In addition to archiving messages, he also takes care of detecting message updated and deleted messages.
So I used this detection to develop an alert system by email, allowing anyone who wishes to be alerted by email in case of update / deletion of a message.
The goal is that you can then alert the forum that there has been a change / deletion and therefore should be wary ( Depending on the type of message assigned ).

With this bot, it will become much less interesting to try to hack the account of the bot, since even if the account comes to be compromised, it will be possible to find archives.
The advantage is that since this bot is public and open-source, everyone can run it ( Very easy to launch ).

So for those who want to install the bot, this is where it happens:
- https://github.com/luluwebmaster/bitcointalk-auto-verify-signatures-archive-and-alert

Here, I will list all the sites archiving the messages of the subject in question, thus starting with two sites ( Do not hesitate to share yours if you use the bot ) :
- https://signatures.bitcointalk.luc-mergault.fr/
- https://signatures.bitcointalk.cryptos-currencies.com/

Here is a link to see the addresses linked to a nickname ( Replace my username with yours ) :
- https://signatures.bitcointalk.luc-mergault.fr/user/lulucrypto.html

If you have not yet sent your address, do not hesitate to come and test ^^

https://bitcointalk.org/index.php?topic=996318.new;topicseen#new

If you wish, it is possible to support the project here:
- 1DSXQn7AankhmXUvExfZBbo8zWa3ie3jXc

Note : During the beginning of the operation of the bot, I would be there to check its functioning, and correct the possible errors / bugs of it.
Note : Before the launch of this project, I had permission from theymos.

See you soon !

[hugeblack]

Wow, it seems you did a great job but I would like to inquire about some information:

 - What keywords does a bot search for? You talk about this formula, but is it "----" or "-----BEGIN BITCOIN SIGNED MESSAGE-----"?
 - How is it determined that it was handled by another user?
 - How long does it take to search and to respond?

By the way, I take this opportunity to ask you something, do you know a website to verify the signatures created by addresses other than addresses Legacy ( Segwit ? ) ?

So far, there has been no agreement on a standardized format for signature to act as a verification of Legacy's (P2SH) signature.
For more read this ----> About possibility to Sign messages in Segwit address in future

[lulucrypto]

Hello,

Thank you

1) Currently I use "-----BEGIN BITCOIN SIGNED MESSAGE-----", "-----BEGIN SIGNATURE-----" and "-----END BITCOIN SIGNED MESSAGE-----" to found user request.
2) It's pretty simple, any new address ( Not registered in the database, and detected as a signed message ) is saved as a request. From the moment when another user posts a message with this address ( Quote ), then the request is detected as treated ( I am sure that it is not necessarily the best way, it is possible that it changes thereafter ).
3) For the detection, it takes about 5 to 10 seconds, and for the verification process, it takes +/- 30 to 60 seconds ( It is very variable ).
4) Oh, ok thank you, and for "P2SH" addresses, is it possible ?

Edit : And for information, currently the bot can't verify signatures, he is currently saving all the messages already sent on the subject ( +/- 10 000 ).

Edit 2 :  A screenshot of the console currently

[hatshepsut93]

Nice project, I welcome any automation that frees members of Bitcoin community from doing tedious tasks.

I got a question, do you plan to use Brainwallet for signature verification, or do you plan to eventually switch to some node library? I think it's unnecessary to rely on third-party service to do cryptography for you if you can do it yourself, Brainwallet's site can go offline or change their structure and they can have bugs.

[DiamondCardz]

It's a nice project, but there really needs to be more than one account quoting and verifying signatures. If it ends up being that solely one person is quoting these it's too easy for a malicious actor behind the account to edit or delete quotes others have made. Say for instance that the bot account gets hacked and offers to delete quotes of hacked account's addresses for payment. My worry is that people will get complacent and only this bot will end up quoting signatures.

These problems will obviously be way less significant if more than one person runs this script, or if people still quote as normal. Just saying that this can't be something that entirely 'frees' users from needing to quote, but simply a complement to it.

[hatshepsut93]

It's a nice project, but there really needs to be more than one account quoting and verifying signatures. If it ends up being that solely one person is quoting these it's too easy for a malicious actor behind the account to edit or delete quotes others have made. Say for instance that the bot account gets hacked and offers to delete quotes of hacked account's addresses for payment. My worry is that people will get complacent and only this bot will end up quoting signatures.

I don't think it's a big problem, LoyceV and maybe some other users archive all messages, and the bot itself posts link to imgur and archive.fo, so even in worst case of the bot account getting hacked, the quotes won't be gone forever. Plus, theymos could probably restore posts if the vanishing would be quickly detected.

These problems will obviously be way less significant if more than one person runs this script, or if people still quote as normal. Just saying that this can't be something that entirely 'frees' users from needing to quote, but simply a complement to it.

If the project was open source, we could have multiple competing bots, but it would cause duplicate posting in some cases, so the bot would need some system for deleting its own posts if they duplicate a quote.

[DiamondCardz]

It's true that a mass vanishing would probably be caught fairly quickly. My concern is that a single deletion could lead to a big exit scam on a bought/hacked account and it's very unlikely that it would be detected in time.

Obviously in an ideal world we wouldn't need to have addresses be staked in a thread, it would be much nicer if there was a function on the forum that allowed you to stake an address on your profile which would be verified by the forum software and from then on saved.

[lulucrypto]

Hello,

Thank you for your interest about the project

To answer +/- in order:

1) I could have used a package to directly check the signatures, but it did not fit the idea I wanted. Namely, allow everyone to visually check (Screenshot), and provide the verification link.
2) The purpose of this bot is not necessarily to replace the users who help with the checks. But as you said, rather serve as a complement ( Everyone is free to continue to perform verification ).
3) Now, I understand the problem of hacking, and I will see to try to find a solution to this.
4) The idea of having multiple instances of the bot is interesting, but as it was said, it becomes complicated to manage double posts / who should post / etc ...

Edit : For information, the archive contain the original post and the quotation ( Exemple : https://archive.fo/9DYWr )

The project is clearly not over, so expect changes to come

[pooya87]

By the way, I take this opportunity to ask you something, do you know a website to verify the signatures created by addresses other than addresses Legacy ( Segwit ? ) ?

i would drop using these silly online tools and use a decent library but here is a workaround in case you were interested:
(first a simple explanation of what is "message signature verification") an ECDSA signature is two numbers known as r and s. the math behind it has some characteristics that allows you to recover possible public keys from signature by knowing the message that was signed. so the first step is to base64-decode the signature, take r and s out, hash the message and calculate public key(s).
the second step is to convert that public key to an address and then check it with what user gave you.

knowing that you can easily convert a bech32 address to a legacy address* to fool the "silly" tool to pass the second step. all you need is a bech32 decoder

example https://bitcointalk.org/index.php?topic=996318.msg52763726#msg52763726
bc1qarr0w42t3z7xtrcgwxh6kcckzq47tme5xfnj4n ->
e8c6f7554b88bc658f0871afab6316102be5ef34 ->
1NDp7v4M1fueJDob7mrnJktn1SnT4yramy
now the silly tool passes the verification!

you can't use this for addresses starting with 3 (nested SegWit) because they are hash of a hash of public key (it is more complicated than that, just simplifying it) so you have to have the public key and can't use what i explained above.

* node that this is a  workaround and should not be done for anything else. the decoding is done by knowing bech32 encodes "OP_0 <hash160(pubkey)>" and base58 encodes "version <hash160(pubkey)> checksum"

how about an "opt-out" option so that user could post a string (eg. nobot) in first line and get the bot to ignore the message.

[lulucrypto]

By the way, I take this opportunity to ask you something, do you know a website to verify the signatures created by addresses other than addresses Legacy ( Segwit ? ) ?

i would drop using these silly online tools and use a decent library but here is a workaround in case you were interested:
(first a simple explanation of what is "message signature verification") an ECDSA signature is two numbers known as r and s. the math behind it has some characteristics that allows you to recover possible public keys from signature by knowing the message that was signed. so the first step is to base64-decode the signature, take r and s out, hash the message and calculate public key(s).
the second step is to convert that public key to an address and then check it with what user gave you.

knowing that you can easily convert a bech32 address to a legacy address* to fool the "silly" tool to pass the second step. all you need is a bech32 decoder

example https://bitcointalk.org/index.php?topic=996318.msg52763726#msg52763726
bc1qarr0w42t3z7xtrcgwxh6kcckzq47tme5xfnj4n ->
e8c6f7554b88bc658f0871afab6316102be5ef34 ->
1NDp7v4M1fueJDob7mrnJktn1SnT4yramy
now the silly tool passes the verification!

you can't use this for addresses starting with 3 (nested SegWit) because they are hash of a hash of public key (it is more complicated than that, just simplifying it) so you have to have the public key and can't use what i explained above.

* node that this is a  workaround and should not be done for anything else. the decoding is done by knowing bech32 encodes "OP_0 <hash160(pubkey)>" and base58 encodes "version <hash160(pubkey)> checksum"

how about an "opt-out" option so that user could post a string (eg. nobot) in first line and get the bot to ignore the message.

Hello,

Thank you for your very interesting message.

To be honest, I did not necessarily understand what to do to perform the "conversion" address.

Would you be able to give me an example of your idea in JavaScript ( Or other language ) for example ?

And for your suggestion, I take note, it's a good idea for those who do not want the application to be processed by the bot.

Thank you very much !

[LoyceV]

It's a nice project, but there really needs to be more than one account quoting and verifying signatures. If it ends up being that solely one person is quoting these it's too easy for a malicious actor behind the account to edit or delete quotes others have made. Say for instance that the bot account gets hacked and offers to delete quotes of hacked account's addresses for payment. My worry is that people will get complacent and only this bot will end up quoting signatures.

I don't think it's a big problem, LoyceV and maybe some other users archive all messages, and the bot itself posts link to imgur and archive.fo, so even in worst case of the bot account getting hacked, the quotes won't be gone forever.

The links to archive.fo and imgur will be very hard to find back without the post.
But indeed, I archive messages. See this link for the posts I have on that thread. It's not automatically updated yet, but it will be once I have the time to test it a bit more. I'm not sure how long I'll do this though, staking an address is meant to be used years later, and I will run out of space at some point.

Plus, theymos could probably restore posts if the vanishing would be quickly detected.

As far as I know, deleted posts are stored forever. That's only useful if Cryptios has access to those too, because theymos doesn't do account recoveries anymore.
It might help if the user remembers his post number, I'm not sure how much work it is to find a certain deleted post.

Obviously in an ideal world we wouldn't need to have addresses be staked in a thread, it would be much nicer if there was a function on the forum that allowed you to stake an address on your profile which would be verified by the forum software and from then on saved.

An easy solution would be to move the entire "stake addy"-thread to Auctions, so that users can't edit or delete their posts anymore.

[hatshepsut93]

Would you be able to give me an example of your idea in JavaScript ( Or other language ) for example ?

And for your suggestion, I take note, it's a good idea for those who do not want the application to be processed by the bot.

Thank you very much !

Here's a Javascript library for decoding bech32 - https://github.com/slowli/bech32-buffer

They even have a web demo - https://slowli.github.io/bech32-buffer/

This decoder returns a buffer, so depending on how you handle signature verification, you might need to convert that buffer into a Bitcoin address (like in pooya87's example).

[lulucrypto]

The links to archive.fo and imgur will be very hard to find back without the post.
But indeed, I archive messages. See this link for the posts I have on that thread. It's not automatically updated yet, but it will be once I have the time to test it a bit more. I'm not sure how long I'll do this though, staking an address is meant to be used years later, and I will run out of space at some point.

Yes it is a good thing that you archive the messages too. For my part, all messages posted on the topic dedicated to "Stake" are also saved, and will soon be publicly available on a dedicated page.

An easy solution would be to move the entire "stake addy"-thread to Auctions, so that users can't edit or delete their posts anymore.

Ideally, the bot account should not be able to delete its messages, and can only edit its messages once ( Or for a limited time, for example ). But already if it was possible to make sure that he can not delete his messages, it would be a very good point !

Here's a Javascript library for decoding bech32 - https://github.com/slowli/bech32-buffer

They even have a web demo - https://slowli.github.io/bech32-buffer/

This decoder returns a buffer, so depending on how you handle signature verification, you might need to convert that buffer into a Bitcoin address (like in pooya87's example).

Thank you for that, I understand now !

I'll see what I can do with that

[LoyceV]

Ideally, the bot account should not be able to delete its messages, and can only edit its messages once ( Or for a limited time, for example ). But already if it was possible to make sure that he can not delete his messages, it would be a very good point !

I read a while back about a scammer who years ago lost his edit-privileges on the forum, to stop him from removing evidence. If theymos can take away edit and delete power from your bot (and publicly confirms it), it can offer a very nice service.

[lulucrypto]

I read a while back about a scammer who years ago lost his edit-privileges on the forum, to stop him from removing evidence. If theymos can take away edit and delete power from your bot (and publicly confirms it), it can offer a very nice service.

Yes, I understand the problem, I work to propose a solution soon

[TheBeardedBaby]

The archive link is not available in the quoted message, see below :

Please verify the signature and quote my Bitcoin address, which can be found in my profile as well.

Code:

-----BEGIN BITCOIN SIGNED MESSAGE-----
I'm Mento, this is my account on bitcointalk.org and today is 30/10/2019.
My profile: https://bitcointalk.org/index.php?action=profile;u=486605
-----BEGIN SIGNATURE-----
17ZayXedh7vrN2s4BBXgQqi35Na2GzXbky
IL3S7vfzPGqzZ+3HUxix/y5fYEFYZg4NbgkLKUnKs7WERl2MvyZWOFwpswob7DjPzuU+tNP4Ir/VYkFI8s+jNHY=
-----END BITCOIN SIGNED MESSAGE-----

>--------------------------------------------------------
>Quoted and Verified ( Screenshot ) and Archived.
>--------------------------------------------------------
> How to post a signed message ( Please respect format ).
> If you don't want to be verified by me, please add "no-bot" in your message.
> Addresses currently accepted : Bitcoin ( P2PKH ).
> Note for Modo and Admin : If you receive message from "bot_avsignatures" or "Lulucrypto" to edit or remove this post, it's not me. Please don't accept request.
>--------------------------------------------------------
> This is a auto response send by the "Auto Verify Signature Bot".
>--------------------------------------------------------

[lulucrypto]

Hello,

Yes I saw that, thank you for the report.

I'm still working on the project, new ones should come soon ... 

[DiamondCardz]

Is there any chance of you putting this bot on Github? It would be good to see other people being able to suggest modifications and even better if more than one instance of the bot could be archiving these entries. I think that something like this which is not-for-profit would benefit quite  a lot from being made open source.

[lulucrypto]

Is there any chance of you putting this bot on Github? It would be good to see other people being able to suggest modifications and even better if more than one instance of the bot could be archiving these entries. I think that something like this which is not-for-profit would benefit quite  a lot from being made open source.

The bot in its current state will not be made public ( The code is too rough ).

However, I am preparing a script ( Bot ) that everyone will install, and that will help with the security of the checks.

[TheBeardedBaby]

Just another note here, when the bot is quoting a message which could not be verified, you should post a warning like ( failed to verify), otherwise everyone can stake a fake addresses and possibly scam people.