Verifiable builds need attention. Only 3 of 68 Android wallets are verifiable

[giszmo]

At WalletScrutiny today we finished our first round assessing the 84 apps we had found to look like maybe being relevant Android Bitcoin wallets. The results are grim:

• 3 are verifiably built from the project's published source code
• 21 apps claim to be open source but either we failed to compile them from the information provided on their repositories or the compilation result differed non-trivially from the app found on Google Play. Trivial differences would be file timestamps, differences in few files that can be quickly understood to be harmless, like an API key not being included in the repository, although that is pointless as it sticks out in the diff even more.
• 25 apps are closed source meaning neither the Playstore description, nor their website nor GitHub searched for their appId revealed any source code
• 19 apps are for custodial services, the biggest being Coinbase. Coinbase recently reached 10 million downloads and with no other app reviewed having even 5 million, that is more users on Coinbase than on all open source wallets combined. Being your own bank ... not so much
• 18 apps turned out to be either not wallets, not for Bitcoin or they had only 1000 downloads or less.

This project is only getting started. If you want to look behind the curtain and maybe want to contribute, source for the website is public.

Now the next steps are:

• Automate verification for wallets that were verifiable once
• Efficiently collect wallet updates
• Alert when verification fails
• Build awareness

If you don't understand what this is about or think it is not important, consider this:

If you are the release manager of a wallet, would you tell your brother to trust your app? Should you trust it? After all it was you who pushed that compile button, right?
Well, if your computer has a backdoor, your compiler might bake in some wallet-stealing "feature" into every version of your app without your knowledge.
How big is the incentive to plant such a backdoor? For some wallets it is gigantic. Hundreds of millions of dollars. Criminals would kill for that amount, which brings me to the second issue:
What if somebody puts you under duress? If whatever you build is not being verified by a second person, ideally far away on an unrelated system, you can't trust yourself and nobody can trust you to release the software you should release. If in an open source project, verification is not easily possible, most likely it is not done internally.

[1713871421]

1713871421

[1713871421]

1713871421

[1713871421]

1713871421

[joniboini]

On your website, you said that Trust Wallet has no source ("Without public source available, this app cannot be verified!"). But they do have a GitHub https://github.com/trustwallet. Does this mean you don't take that as a source or you can't find the repo for the app? Or this is because of that appid thing?

They do have a building guide tho https://developer.trustwallet.com/wallet-core/developing-the-library/building, with the source to be https://github.com/trustwallet/wallet-core. Did anyone try it yet?

[giszmo]

On your website, you said that Trust Wallet has no source ("Without public source available, this app cannot be verified!"). But they do have a GitHub https://github.com/trustwallet. Does this mean you don't take that as a source or you can't find the repo for the app? Or this is because of that appid thing?

They do have a building guide tho https://developer.trustwallet.com/wallet-core/developing-the-library/building, with the source to be https://github.com/trustwallet/wallet-core. Did anyone try it yet?

Please read the article on that wallet. It explains all we did to come to our conclusion. Let me know if that finding is outdated.

[giszmo]

Stop recommending Blockstream Green Wallet. They are baddddddddd. A big one

We are not recommending any wallets. Our hope was to drive awareness for the issue of verifiability and there is bad things to say about all 3 wallets listed as "verifiable" but no wallet is perfect and all the other wallets are potentially losing all the money of all their users at once without security researchers having a chance of detecting it before it happens. And most likely even the team is not exercising build verification, so a release manager in distress might be all it takes for all users losing their money.

That said, what is so bad about Green Wallet?

[hugeblack]

Stop recommending Blockstream Green Wallet. They are baddddddddd. A big one

sorry about that. I missed clicking on send.
I edited it.

That said, what is so bad about Green Wallet?

Once they claim " non-custodial," this does not mean that they are telling the truth.
The company uses multi-sig addresses, meaning that in some cases (2 of 2 address) there are two private keys for sending currencies, the first is yours and the second is for the company.
Indeed, the company can not spend money without your permission, but you can't.


What will happen if the internet crashes in the area where the company is located, you will not be able to spend your money. The same thing happens when they charge high fees. Also, you cannot claim Hardforks.

Therefore, I do not recommend using it for beginners, or at least tell them about using 2 of 3 addresses.

[giszmo]

Once they claim " non-custodial," this does not mean that they are telling the truth.
The company uses multi-sig addresses, meaning that in some cases (2 of 2 address) there are two private keys for sending currencies, the first is yours and the second is for the company.
Indeed, the company can not spend money without your permission, but you can't.

I might be wrong there but my understanding is that the script is a slight bit more complicated. Their 2of2 protects you as you can define rules and they enforce them by not signing if somebody tries to empty your account all at once but if they disappear or charge a huge fee, your funds can be spent with just one key - your key - after one year.

[hugeblack]

I might be wrong there but my understanding is that the script is a slight bit more complicated. Their 2of2 protects you as you can define rules and they enforce them by not signing if somebody tries to empty your account all at once but if they disappear or charge a huge fee, your funds can be spent with just one key - your key - after one year.

The use of "non-custodial" is completely wrong. Perhaps we can describe them as "Split Custody Wallets."
The issue is gray, you can spend coins even if the network is not available, but you will need to wait & some effort. I think beginners should be warned about this.

(2of2 Recovery Case)
You can spend using nLockTime feature, which enables you to sign transactions by default after a certain time "90 days by default"  then use a tool to be able to send your coins.

Read more ----> https://github.com/greenaddress/garecovery

[giszmo]

The use of "non-custodial" is completely wrong. Perhaps we can describe them as "Split Custody Wallets."
The issue is gray, you can spend coins even if the network is not available, but you will need to wait & some effort. I think beginners should be warned about this.

(2of2 Recovery Case)
You can spend using nLockTime feature, which enables you to sign transactions by default after a certain time "90 days by default"  then use a tool to be able to send your coins.

Read more ----> https://github.com/greenaddress/garecovery

I see your point and this is not the only wallet where things are not as black or white as we would hope for. I personally consider it a great and unique feature with little down-side but I would also love to allow critical voices to be accessible from the project. What about a block with a Twitter feed showing tweets mentioning both the wallet and @WalletScrutiny? Would also help to spread the word.

[pooya87]

I might be wrong there but my understanding is that the script is a slight bit more complicated. Their 2of2 protects you as you can define rules and they enforce them by not signing if somebody tries to empty your account all at once but if they disappear or charge a huge fee, your funds can be spent with just one key - your key - after one year.

The use of "non-custodial" is completely wrong. Perhaps we can describe them as "Split Custody Wallets."
The issue is gray, you can spend coins even if the network is not available, but you will need to wait & some effort. I think beginners should be warned about this.

(2of2 Recovery Case)
You can spend using nLockTime feature, which enables you to sign transactions by default after a certain time "90 days by default"  then use a tool to be able to send your coins.

Read more ----> https://github.com/greenaddress/garecovery

the right way of implementing a multi signature scheme as some sort of 2FA is how Electrum does it meaning a 2of3 set up where the user owns 2 keys and the third party server owns the one key. user stores one of his keys in his hot wallet and the other he backs up by writing it down on a piece of paper. then if some day the server had any issues he can easily access his funds by accessing that backup key.
the github link suggests that greenwallet supports this but apparently not by default?

[giszmo]

the right way of implementing a multi signature scheme as some sort of 2FA is how Electrum does it meaning a 2of3 set up where the user owns 2 keys and the third party server owns the one key. user stores one of his keys in his hot wallet and the other he backs up by writing it down on a piece of paper. then if some day the server had any issues he can easily access his funds by accessing that backup key.
the github link suggests that greenwallet supports this but apparently not by default?

That is a good point. As they have to get the user to do a backup anyway, pushing to make two separate backups should not be that awkward and it would solve the problem with the timelock being a timelock when you might need the money.

Ping me on this issue if I forget to update the article.

[dkbit98]

Interesting to see that no wallet has ever been audited and only few of them are reproducible, but I doubt if any information from this website is really accurate and I don't see any hardware wallet listed.
You have Bluewallet listed as Custodial, and it is clear that this is non-custodial open source wallet, and there is no provider that holds the coins.
This is probably one of the best Bitcoin mobile wallets today.



github:
https://github.com/bluewallet/bluewallet

[giszmo]

Interesting to see that no wallet has ever been audited and only few of them are reproducible, but I doubt if any information from this website is really accurate and I don't see any hardware wallet listed.
You have Bluewallet listed as Custodial, and it is clear that this is non-custodial open source wallet, and there is no provider that holds the coins.
This is probably one of the best Bitcoin mobile wallets today.



github:
https://github.com/bluewallet/bluewallet

The "audited" section is to avoid confusion of what we do. We do check reproducibility. That is we test if reviewing the code has any relevance for the binary the provider released. We do not audit wallets. Others might have audited wallets and certainly wallet providers make that claim.

If you find any factual errors, please let us know, ideally via our gitlab. The verdicts are very objective and follow the "methodology" linked in the top of the site.

We are exploring what to do about hardware wallets. Those work very differently and need a very different methodology. We will first expand to other software wallets.

The fact that you thought BlueWallet was self-custodial while implying to know the product tells me everything about why we have to keep the verdict as is for the time being. The provider added a pathetic "This wallet is hosted by BlueWallet" in the LN account creation and calls that a disclaimer.

So again, please show me one wrong categorization!

[dkbit98]

So again, please show me one wrong categorization!

You can run your own node with Blue wallet or you can use their hosted Lightning wallet like for most LN wallets.

Are you considering Green wallet by Blockstream with Liquid Network custodial or not?
Because I see it is very high rated on your website, or you think Lightning Network Bitcoin and Liquid Network Bitcoin L-BTC are not equal with real Bitcoin.
Looks like a double standards to me, but maybe I am wrong idk.

[giszmo]

So again, please show me one wrong categorization!

You can run your own node with Blue wallet or you can use their hosted Lightning wallet like for most LN wallets.

Sure, you can but nothing tells the user he should and the website and wallet description claim self-custody while the default LN account is not self-custodial and "- This wallet is
hosted by BlueWallet." does not convey the fact that they can do whatever with the user's funds.

Are you considering Green wallet by Blockstream with Liquid Network custodial or not?
Because I see it is very high rated on your website, or you think Lightning Network Bitcoin and Liquid Network Bitcoin L-BTC are not equal with real Bitcoin.
Looks like a double standards to me, but maybe I am wrong idk.

I personally would not touch Liquid Bitcoins as the current setup is not self-custodial to my own standards but I do not dig deep into all the shitcoins and protocols and personally draw the line around BTC. So if 8(?) federation members collude, they can steal your coins? There are bugs where the federation collapses and Blockstream can single-handedly spend the bitcoins? Yes, not something I would want to get tangled up with but it's not deceptive on the wallet level. It's only deceptive on the protocol level. The wallet does nothing wrong. If I would categorize it as custodial, I would have to do the same with all that support any shitcoin.

Please read the verdict explanation on all the non-verifiable wallets including the custodial ones:

The app cannot be independently verified. If the provider puts your funds at risk on purpose or by accident, you will probably not know about the issue before people start losing money. If the provider is more criminally inclined he might have collected all the backups of all the wallets, ready to be emptied at the press of a button. The app might have a formidable track record but out of distress or change in management turns out to be evil from some point on, with nobody outside ever knowing before it is too late.

WalletScrutiny is about providers of binaries, currently on the Play Store and the App Store, not about the protocol maintainers/developers.

[dkbit98]

...

If you think like that than you should not consider Lightning Network Bitcoin as a real Bitcoin and any wallet that is using LN (custodial or not) should not be on WalletScrutiny website.
Bitcoin Blue wallet is not custodial, and you can create separate page for all Lightning Network wallets and other second layer solutions if you want.
LN Blue wallet wallet can be custodial and non-custodial and there are many shitcoins that can work with LN and not just Bitcoin.
Just my suggestion.

[giszmo]

If you think like that than you should not consider Lightning Network Bitcoin as a real Bitcoin and any wallet that is using LN (custodial or not) should not be on WalletScrutiny website.
Bitcoin Blue wallet is not custodial, and you can create separate page for all Lightning Network wallets and other second layer solutions if you want.
LN Blue wallet wallet can be custodial and non-custodial and there are many shitcoins that can work with LN and not just Bitcoin.
Just my suggestion.

LN Blue wallet is by default custodial and does not warn the user.

I see your point for LN-only wallets like Phoenix but else, the protocol not being as good as Bitcoin in the presence of an actual non-custodial Bitcoin account doesn't make the wallet custodial. Maybe Phoenix is "not a BTC wallet" but certainly not custodial.

[dkbit98]

WalletScrutiny website added many hardware wallets on their website and only four of them had reproducible codes, Trezor model One, Trezor model T, BitBox02 and KeepKey.
I was a bit surprised to see that ColdCard wallet is still under development, but maybe @giszmo and his team didn't have enough time to finish testing for ColdCard and other wallets that known to be Open Source.
They made several categories like Defunct (feature many dead wallets), No Source (Ledger), Bad Interface (Coldlar, Secalot, Bepal), Leaks Keys (Opendime), Development (ColdCard and many other wallets), and No BTC category.
I noticed some hardware wallets are missing from the list, like Keystone that should be open source, and it is now replacing defunct Cobo hardware wallet.
Clicking on each wallet is showing small window with basic information, price, size, review date, links and detailed full analysis report.


https://walletscrutiny.com/?verdict=all&platform=hardware

I have the give props to giszmo and his team for keeping their promise and doing this huge work of adding hardware wallets like they promised.

[giszmo]

Hi dkbit98,

WalletScrutiny is a ton of work and we are a small team, only.

In our Methodology you can read our priorities:

1. Re-evaluate new releases of Reproducible   wallets as they become available. If users opt for a wallet because it is reproducible, they should be waiting for this re-evaluation before updating.

Today I tested the latest releases of AirGap Vault and Green Wallet. Today, Green was a bit more work than usual.

2. Check if any of the Unreproducible!   wallets updated their issues on their repositories.

We really hope to see more reproducible products, so we always have an eye on the dozens of open issues.

3. Make general improvements of the platform

That is the a catch-all for improving scripts, design and often just investigations. It's probably the bulk of the work.

4. Evaluate the most relevant Development   wallets

For Android we have a good proxy for relevance - downloads. For iPhone we don't and neither for hardware wallets.

Unfortunately we are not progressing in the top category as fast as I wish we would but that has to do with severe lack of people to work with code. The k.o. criteria (custodial, bad interface, defunct, ...) are verdicts relatively inexperienced Bitcoiners can come to but when it comes to reproducing a wallet, it's mostly on me. Emanuel also does play with code and does a ton of work but refuses to open merge requests, so writing the difficult reviews is all on one person that also looks into all the other stuff.

So ... if you want to help, there is a ton to do from simple triage to compilation to design to spreading the word.

[dkbit98]

...

How exactly are you testing Hardware Wallets?
I guess you first need to have actual device in your hand (purchased or received for testing from manufacturer) and then try to reproduce the code.

So ... if you want to help, there is a ton to do from simple triage to compilation to design to spreading the word.

I am helping in spreading the word about WalletScrutiny and I am monitoring hardware wallet changes, especially if they claim they are open source.
You can track that in my topics that is updated on regular basis like this one for example: LIST - Open Source Hardware Wallets.

[giszmo]

...

How exactly are you testing Hardware Wallets?
I guess you first need to have actual device in your hand (purchased or received for testing from manufacturer) and then try to reproduce the code.

We look at claims about the functionality of the device to see if it falls into any of the k.o. criteria like not having a screen to verify what you approve. Then we look for the source code and the binary. If the source code compiles into the binary, the wallet is reproducible. Check out our full methodolgy.

So ... if you want to help, there is a ton to do from simple triage to compilation to design to spreading the word.

I am helping in spreading the word about WalletScrutiny and I am monitoring hardware wallet changes, especially if they claim they are open source.
You can track that in my topics that is updated on regular basis like this one for example: LIST - Open Source Hardware Wallets.

I think we have all the products you list. We have to review most of them still.