New project to scrutinize Bitcoin wallets: walletscrutiny.com

[giszmo]

We've been working on walletscrutiny.com for about two months now as a side project and hope to see many wallets that are currently "only" open source to care more about verification and make it into the "verifiable" category but the resonance in the community so far was underwhelming. How can we get users to care about the integrity of the wallets they are using?

With the community's support, this project could turn into a permanent thing, with new wallet versions automatically being checked as they are being published and we certainly would also expand to other platforms and more attributes.

Currently, being verifiable unfortunately doesn't mean that anybody would verify any code and we also have ideas how to fix that, starting with bug bounties, so security researchers actually care.

Any feedback welcome!

[Patatas]

This is a very good service indeed! Really loved the way you guys have analyzed wallets and detailed the errors while running it locally. I was surprised to see Blockchain.com's wallet doesn't match with their source code. Do you have an automated process of doing this or has to be done manually?

[Patatas]

That's not the definition of "Not verifiable!", it means they can't verify/compare blockchain.com application with it's source code.

Generally only open source project with deterministic build support which can be verified.

That's what I meant? Quoting the article,

Not verifiable: The provided Open Source Code could not be verified to match the app released on Google Play

Meaning, when OP's team tried to compile the wallet from the source provided by Blockchain.info, the compiled version didn't match the production version which was released on Google-Play. So they assumed it's Not verifiable?

[DaveF]

There is another discussion about his site here: https://bitcointalk.org/index.php?topic=5209504

Except for my 1 post there and this post I am going to stay out of it since he is a Mycelium developer and my current view of the app has greatly degraded. Because of the issues costing people a lot of time & effort to get their BTC, I don't think I am going to be able to provide a fair view and ranting is not going to help anything.

-Dave

[bitmover]

Any feedback welcome!

Congrats on your initiative!!

Let me ask you a question. I was though Samourai Wallet was open source. Why you couldn't verify that the published code matches the app? Don't they have a gitlab github or something?

[giszmo]

This is a very good service indeed! Really loved the way you guys have analyzed wallets and detailed the errors while running it locally. I was surprised to see Blockchain.com's wallet doesn't match with their source code. Do you have an automated process of doing this or has to be done manually?

So far it has to be done all manually. There are some different ways of building the apps and I will automate stuff once I see people care.

This is amazing idea. Most people who prefer open-source sofware actually don't bother or could verify it by themselves.

The "could" part doesn't matter if others can and do and the built apk is verifiable. That's the point of this project.

Meaning, when OP's team tried to compile the wallet from the source provided by Blockchain.info, the compiled version didn't match the production version which was released on Google-Play. So they assumed it's Not verifiable?

Right. You can read the detailed analysis. We ran into a known issue from their issue tracker.

But verifying/auditing application and it's source code is complex task, so i might be wrong.

We don't verify/audit applications and their source codes. We test if it could theoretically be done. We test verifiability. We do not verify.

Let me ask you a question. I was though Samourai Wallet was open source. Why you couldn't verify that the published code matches the app? Don't they have a gitlab github or something?

Please read our detailed analysis. While we hope that many of the open source wallets come forward and fix their sloppy documentation or release code quicker or otherwise make it verifiable, we also assume that not all will do this. Let's see.

[hugeblack]

The idea of the project is excellent, you need some adjustments to the interface, but as a whole, the idea is very beautiful.

 - I don’t know what the arrangement algorithm is, but I think there should be options for searching so that I can search for some features like Lightning Network, control over fees and others.
 - You can ask questions to choose the best wallet based on the answers, it is like ----> https://bitcoin.org/en/choose-your-wallet
 - One of the verified applications "Blockstream Green Wallet": I think that this wallet gives you a multi-signature address only so that the user can choose either a 2of2 or 2of3 signatures (that the company has control of one of the signatures.) It is especially bad with hardforks so I hope you reassess them.
 - you can add infinito & magnum wallets.
 - you can add Coinstarts price tracker.

[giszmo]

The idea of the project is excellent, you need some adjustments to the interface, but as a whole, the idea is very beautiful.

Thanks! The left side menu being on top on mobile is certainly not ok given almost all users were mobile so far

- I don’t know what the arrangement algorithm is, but I think there should be options for searching so that I can search for some features like Lightning Network, control over fees and others.

Once we diverge into many more apps, we will need filters but at this stage it's not necessary yet.

I wouldn't want the user to filter out the good wallets just because he filtered for pink ones and there are only shitty pink ones. Once more wallets fix their verifiability, we might add more filters but I tend to rather raise the bar and push for actual code reviews so the next criteria to get on the top will be a bug bounty program.

- You can ask questions to choose the best wallet based on the answers, it is like ----> https://bitcoin.org/en/choose-your-wallet

Bitcoin.org is multi-platform. It makes sense to filter by platform, which we do: Android. Else, it's very brief and lacks accountability. Our project explains in much more detail our findings.

- One of the verified applications "Blockstream Green Wallet": I think that this wallet gives you a multi-signature address only so that the user can choose either a 2of2 or 2of3 signatures (that the company has control of one of the signatures.) It is especially bad with hardforks so I hope you reassess them.

We do not look at features yet and will probably only favourably consider hardware wallet and multisig support later.

Their design makes a lot of sense and I don't see an issue with hardforks there, neither. Sure, their company server will not create altcoin transactions but as you are in full control anyways, you can still work around this.

- you can add infinito & magnum wallets.

Have Playstore links for those? Ideally give me a block like this one:

Code:

---
title: "Coinomi Wallet :: Bitcoin Ethereum Altcoins Tokens"

wallet: true
users: 1000000
appId: com.coinomi.wallet
launchDate: 2014-11-01
latestUpdate: 2019-11-12
apkVersionName: 1.17.1
stars: 4.6
commentCount: 20727 # actually this is the rating count
permissions:
website: https://www.coinomi.com/
repository:
icon: "images/wallet_icons/coinomi.png"
bugbounty:
verdict: nosource # May be any of: wip, nowallet, custodial, nosource, nonverifiable, verifiable, bounty, cert1, cert2, cert3

date: 2019-11-14
permalink: /posts/2019/11/coinomi/
redirect_from:
  - /coinomi/
tags:
  - Android
  - Security
---

- you can add Coinstarts price tracker.

If it's not holding coins, it's not of interest for security audits.

[sir_danny]

**Update**

WalletScrutiny is currently running a donation campaign. So far we've 'analyzed':

- 2790 in Cryptocurrency apps in Google Play
- 651 in the Apple Store
- 288 hardware wallets
- 44 bearer tokens

Big thanks to some friends in BitcoinTalk who actually provided some materials regarding some of the archived bearer tokens.

If you're interested to contribute, do visit https://walletscrutiny.com/donate

[Kakmakr]

We need "services" like this to support the millions of people that cannot "verify" the integrity and security of these services themselves. The community will support this service when they benefit from this themselves. The donations for the service will flow in when people use the service and when they post your findings on forums like this.

Your main goal now.... will have to be to market this service, so that people will know where to go to "verify" wallets. Good luck with the project, I will bookmark it for the future. 

[bitmover]

If you're interested to contribute, do visit https://walletscrutiny.com/donate

I am impressed by how much donations you received. 2 BTC!!
This  amazing revenue proves how important for users your service is.

Congratulations on your service.

[Pmalek]

I hope it's ok to make recommendations here regarding the next wallets that you could review.
The Coldcard Mk4 is available now and in stock. I would be interested in reading your opinion on it. Someone posted a great review of the device in the Hardware wallets board and your thoughts about the code could greatly complement it.