At WalletScrutiny today we finished our first round assessing the 84 apps we had found to look like maybe being relevant Android Bitcoin wallets. The results are grim:

• 3 are verifiably built from the project's published source code
• 21 apps claim to be open source but either we failed to compile them from the information provided on their repositories or the compilation result differed non-trivially from the app found on Google Play. Trivial differences would be file timestamps, differences in few files that can be quickly understood to be harmless, like an API key not being included in the repository, although that is pointless as it sticks out in the diff even more.
• 25 apps are closed source meaning neither the Playstore description, nor their website nor GitHub searched for their appId revealed any source code
• 19 apps are for custodial services, the biggest being Coinbase. Coinbase recently reached 10 million downloads and with no other app reviewed having even 5 million, that is more users on Coinbase than on all open source wallets combined. Being your own bank ... not so much
• 18 apps turned out to be either not wallets, not for Bitcoin or they had only 1000 downloads or less.

This project is only getting started. If you want to look behind the curtain and maybe want to contribute, source for the website is public.

Now the next steps are:

• Automate verification for wallets that were verifiable once
• Efficiently collect wallet updates
• Alert when verification fails
• Build awareness

If you don't understand what this is about or think it is not important, consider this:

If you are the release manager of a wallet, would you tell your brother to trust your app? Should you trust it? After all it was you who pushed that compile button, right?
Well, if your computer has a backdoor, your compiler might bake in some wallet-stealing "feature" into every version of your app without your knowledge.
How big is the incentive to plant such a backdoor? For some wallets it is gigantic. Hundreds of millions of dollars. Criminals would kill for that amount, which brings me to the second issue:
What if somebody puts you under duress? If whatever you build is not being verified by a second person, ideally far away on an unrelated system, you can't trust yourself and nobody can trust you to release the software you should release. If in an open source project, verification is not easily possible, most likely it is not done internally.

At WalletScrutiny today we finished our first round of assessing the 84 apps we had found to look like maybe being relevant Android Bitcoin wallets. The results are grim:

• 3 are verifiably built from the project's published source code
• 21 apps claim to be open source but either we failed to compile them from the information provided on their repositories or the compilation result differed non-trivially from the app found on Google Play. Trivial differences would be file timestamps, differences in few files that can be quickly understood to be harmless, like an API key not being included in the repository, although that is pointless as it sticks out in the diff even more.
• 25 apps are closed source meaning neither the Playstore description, nor their website nor GitHub searched for their appId revealed any source code
• 19 apps are for custodial services, the biggest being Coinbase. Coinbase recently reached 10 million downloads and with no other app reviewed having even 5 million, that is more users on Coinbase than on all open source wallets combined. Being your own bank ... not so much
• 18 apps turned out to be either not wallets, not for Bitcoin or they had only 1000 downloads or less.

This project is only getting started. If you want to look behind the curtain and maybe want to contribute, source for the website is public.

Now the next steps are:

• Automate verification for wallets that were verifiable once
• Efficiently collect wallet updates
• Alert when verification fails
• Build awareness

Thanks for your kind words but we do not verify much. We only test if the wallets at the time of our test can be audited. An audit itself would cost tens of thousands of dollars per wallet and would be outdated with the next release. As this is not a practical thing to expect, we will go in a slightly different direction and make sure the incentives are there for finding the one issue through something like bug bounties.

Well, the answer is certainly more nuanced than that.

On your desktop you usually run Windows, Mac or Linux and the security models differ quite dramatically. And the way people use those systems varies. If you keep your system updated and don't install anything, you are more secure than if you regularly install pirated software or otherwise software from untrusted sources.

That said, Android is really good at sandboxing, so one app should not have access to another app's storage, so it's much harder to screw up on Android than on Windows.

Lastly, larus maybe wasn't talking about desktops but about hardware wallets cause with these it's even harder to lose your bitcoins.

For what it's worth, the scary "code" blocks are now limited to 25 lines. Inclined users can still scroll inside those boxes but it's certainly less scary now. We will improve that further after some urgent updates.


We are aware of that. The current iteration of the project is "all manual" but if there is interest (read: money to be earned), we will automate the process and alert users of issues. We are considering the fact that Android can be rolled out in different versions to different countries and that reviewing huge changes is more time consuming than reviewing small changes. It's just that the status quo is miles away from caring about actual reviews. Reviewability is the current mission.


As stated on our page, we considered the Playstore description and the project's website, not the app itself and we will not change that, as apps can't be parsed without having the source code in the first place.


Absolutely! I still do get money. Less money for less work though.


In bitcointalk you mean? I had no idea that that was something of interest to people. For all I know it never was an official company outlet.


There is not much to address there. Syncing issues should be fixed. There is no support, right. Media-flow-opt-out should work from 3.2.0.17 on.


The walllet being verifiable is the subject of our project and having to restore your mnemonic on a different wallet is a minor detail compared with a wallet pulling an exit scam. My mission is to make exit scams harder.

Thanks! The left side menu being on top on mobile is certainly not ok given almost all users were mobile so far


Once we diverge into many more apps, we will need filters but at this stage it's not necessary yet.

I wouldn't want the user to filter out the good wallets just because he filtered for pink ones and there are only shitty pink ones. Once more wallets fix their verifiability, we might add more filters but I tend to rather raise the bar and push for actual code reviews so the next criteria to get on the top will be a bug bounty program.


Bitcoin.org is multi-platform. It makes sense to filter by platform, which we do: Android. Else, it's very brief and lacks accountability. Our project explains in much more detail our findings.


We do not look at features yet and will probably only favourably consider hardware wallet and multisig support later.

Their design makes a lot of sense and I don't see an issue with hardforks there, neither. Sure, their company server will not create altcoin transactions but as you are in full control anyways, you can still work around this.


Have Playstore links for those? Ideally give me a block like this one:



If it's not holding coins, it's not of interest for security audits.

If you are worried about your backup being found by the wrong person, you can always buy 2 hardware wallets and put the same key on both of them and use one as backup.

Importing private keys sounds scary. The Android clipboard can be monitored by all installed apps without even an extra permission and as soon as the user copies the private key, all malicious apps race to broadcast their attempt of stealing the funds

Actually with multisignature you can get very far in terms of security. Create a wallet involving your iphone, your android, your Windows PC, your Linux laptop and your hardware wallet and create a 5-of-5 wallet. What are the odds that this setup steals your coins?


Paper wallets are vastly inferior to hardware wallets as whoever finds them gets control of the coins. The hardware wallets are strictly superior, as you can restore your paperwallet private key on a hardware wallet and gain the protection against thieves without losing anything.


We so far are not offering consultancy. Android wallets cannot offer the same security as hardware wallets even with secure elements but that's a long topic. Android phones would need special hardware to indicate the operation of secure code for that and it would still have a much bigger attack surface than a Trezor for example.

Sorry, what? We are currently two developers following build instructions found on wallet projects. We are both Android Bitcoin Wallet developers, so we certainly are more up to this task than the average user who wants to compile the wallet for himself but we do not code much. Just the website.

We disclosed our findings in great detail, including the date we did our investigation. If back when you did it or now, things are different, that might explain it and we might give it another shot.


That repository has its last code update in June, while the wallet on Google Play's last update was in October. This repository was not linked from Google play or their website. Why should I assume it's relevant?


While we have a commitment with Mycelium, we also have a private life and while we do work for the company, we do not call the shots. Also Mycelium still owes me money since June and Rassah?? Do you live under a rock? He's not with Mycelium since years and said so in various occasions.


That's the problem. Those wallets have not millions of dollars under management, they have millions of users that have god knows how many dollars in their wallets. This is a huge incentive for anybody who could pull it off, to leak the private keys to then empty all wallets at once. This is also a huge incentive to provide a wallet for free that many people use and trust. To pay people to do great support and to go to conferences ...


Yes, this is an issue and why I use a "GoogleOne" phone, which comes with guaranteed updates for years. On the other hand, wallets can alert their users about security issues and I know Mycelium alerts users that try to run it on certain insecure versions of Android. In the end, Android is pretty secure though. More secure than your average Windows certainly.


Totally agree. Never put life-changing amounts in anything but a trusted hardware wallet. If you manage amounts that would change the life of many people, don't trust any single hardware wallet neither. Use multi signature with at least 3 brands of hardware wallet and pay somebody who knows how that works. Don't try that at home with that amount of money.


Thanks to the backup I trust my Android with more money than my leather wallet. Losing one is not like losing the other. For any serious money, other wallets are more suited.


To be fair, sandboxing on Android is pretty good. With your argument you could also say to not use the web cause of Java Script being insecure. Sometimes Browser makers screw up but usually it's so much more secure than to install apps on your Windows where the app can do whatever with the storage of all the other apps ...


Android is what I know as a contributor at Mycelium and I see the issue there, with more than 20 million downloads for the wallets I investigated.
If people care, we might expand to other platforms and maybe also other apps but for now, Android is already quite a lot of apps to look after.


If people care, we will expand. For now, there are only 3 wallets "green", so it's kind of pointless to be more strict and have them all red. My hope is that some 10 of the open source wallets fix their build setup so we can verify it and then we raise the bar. We would love to test new releases in real time and alert users if a verifiable app turned non-verifiable. We would also like to list bug bounties paid by the different projects, so security researchers get on board for the better paying ones.


Android is pretty good at sandboxing apps. One app should not be able to reach into another app's sandbox unless you give it root access. Keep your Android updated and it's pretty secure. More secure than Windows for sure.


There is no such thing as proven to be secure. Any wallet provider in distress might turn evil. So you trust Mycelium? I'm compiling the release versions for that wallet. How do you know you can trust it? Maybe somebody took my family hostage. Do you believe I care more about your bitcoins than their life? Maybe somebody infected my work laptop with a malware that injects a backdoor when I compile the wallet. Those are the reasons why reproducibility matters even for closed source projects. You have to check every build. At Mycelium, if there is a backdoor in our wallet, not only must I have missed it, my whole team and interested outside security researchers must have missed it as every release is compiled and verified by other team members and the public source code is pushed before we push the apk to Google Play. All this happens behind the scenes but it matters greatly and only if the app is publicly verifiable can people hope that it's actually been verified.

We certainly would hope to get the funding necessary to focus on this project and expand it to cover all wallets. For now we have to see how the community receives the project.

Which wallets should we add? Please provide a link to Google Play if you want us to review your wallet of choice. We only looked at what we know and could easily find.
Every browser should have a search feature
Check out our Methodology page.

I'm glad you care and I don't want to say you shouldn't but as you see you are not alone in trusting your money to a wallet that cannot be publicly scrutinized. I hope that many of the anyway open source wallets will come around and fix their setup so they can be easily audited. I bumped this issue and maybe you want to state your concern to them there, too.

So far it has to be done all manually. There are some different ways of building the apps and I will automate stuff once I see people care.


The "could" part doesn't matter if others can and do and the built apk is verifiable. That's the point of this project.

Right. You can read the detailed analysis. We ran into a known issue from their issue tracker.

We don't verify/audit applications and their source codes. We test if it could theoretically be done. We test verifiability. We do not verify.

Please read our detailed analysis. While we hope that many of the open source wallets come forward and fix their sloppy documentation or release code quicker or otherwise make it verifiable, we also assume that not all will do this. Let's see.

If you think we should add them, share the links to the Playstore listing, please.

May I pick on the "reputed publishers" there? Do you know who they are? If the publisher hides in secrecy, that on its own is a huge red flag for me.

I totally agree on hardware wallets being the way to go but my list involves wallets with a combined download count of 20 million and a hardware wallet still costs over $100. You only invest $100 if your expected loss of not doing so is greater than $100. If you estimate your chance of the wallet losing funds at 5%, you won't use a hardware wallet for anything up to $2000. The exit scammer though may empty a million accounts at once, so here there is a lot of money to be made. Lets spot the black sheep and kick them out.


Thanks for the feedback! Scary landing page was certainly intended. Boring techy stuff was more the result of us wanting to justify our conclusions. To not show what we tried would look like we didn't try and we don't want to call out wallets on a whim. Maybe we can move the "analysis protocols" into separate documents for the more technically inclined audience?


So far nobody has discovered the donate button but I was thinking of adding one per wallet and make it kind of a popularity contest that also pays for the project.


Currently it's even worse: Only three are publicly verifiable which is not the same as actually being verified which would be an expensive and ongoing process.

As nobody cares, it probably doesn't mean much yet, as I refuse to believe that the other 34 are scammers but if people wake up to the idea that verifiability matters, we will see which wallets will actually come forward and make their apps verifiable and which don't.

I don't understand sentence 2 and 3. Many wallets are open source but not verifiable. Those I hope to win over. The closed source wallets I consider outright evil.

You mean like a pop-up on the landing page?
Do you care?
O Yes
O No
?


You mean this wallet?


I am considering to mark Android wallets that support hardware wallets for a start but hardware wallets on their own are a very different kind of tool than Android apps.


Honored by your trust but as stated on the detailed analysis of that wallet, contributors to walletscrutiny are also contributors to Mycelium. In fact I introduced verifiability to Mycelium a year or so ago and it's an integral part of our release protocol to have each build be verified by at least a second engineer. No malware on the release manager's machine should be able to sneak in backdoors.

Thank you! I'm polishing it and hope to standardize it more, as it should get automated anyway, so what you see so far is the experimentation phase.


Being a wallet dev myself (lead dev Mycelium for Android) being subjective is not cool. I would love to but there is people hating me with a passion because I rated their wallet one star 3 years ago. If I say something about other wallets, it better be solid


Ah, now I remember how this forum works. OMG. Haven't posted in years. Should I group my posts now or is the harm already done and I will get banned from the forum anyway?

Edit: deleted two posts. Bumping was not my intention.

Strike!


I only started the project and most wallets don't care cause users don't care yet. I expect many of the non-verifiable ones will slip into the verifiable section if people care.

Once people care, wallets that remain non-verifiable turn more into red flags by my estimate. Also I intend to raise the bar once people move towards caring.


Then verifiability is still for you. Only if a wallet is verifiable does the verification anybody does matter. You like 99.999% of all users take what's there on Google Play but if one researcher reviews the code and the code actually has to do with the app you are running (it is verifiable), you gain security from that guy doing his scrutiny.


In the individual articles I only distinguish between "does not apply", "verifiable" and "not verifiable". No source being available is the easier version of not verifiable, as then I don't waste two hours trying to compile stuff but it's certainly alarming as that means the provider does not want public scrutiny as he could release the code under a license that doesn't allow competition instead.


Are those Android wallets? If you want, share a link on our reddit.


If you are talking about the Bitcoin Core I know, then that is not an Android wallet and in terms of verifiability it ranks in its own class well above all there is on Android, as it actually gets smore scrutiny than all other wallets combined.

We've been working on walletscrutiny.com for about two months now as a side project and hope to see many wallets that are currently "only" open source to care more about verification and make it into the "verifiable" category but the resonance in the community so far was underwhelming. How can we get users to care about the integrity of the wallets they are using?

With the community's support, this project could turn into a permanent thing, with new wallet versions automatically being checked as they are being published and we certainly would also expand to other platforms and more attributes.

Currently, being verifiable unfortunately doesn't mean that anybody would verify any code and we also have ideas how to fix that, starting with bug bounties, so security researchers actually care.

Any feedback welcome!

We've been working on walletscrutiny.com for about two months now as a side project and hope to see many wallets that are currently "only" open source to care more about verification and make it into the "verifiable" category.

With the community's support, this project could turn into a permanent thing, with new versions being checked as they are being published and we certainly would also expand to other platforms and more attributes to look at.

Currently, being verifiable unfortunately doesn't mean that anybody would verify any code and we also have ideas how to fix that.

Any feedback welcome

Mycelium will not force users to use bech32. bech32 is still not popular and has clear issues as for example the signing of messages is not yet well defined. If using bech32 puts you into a smaller anonymity set, it degrades your privacy. We opted to improve privacy instead by providing the mixed account feature, where you can seamlessly switch between address types.

I wonder how you can track bitcoin and S&P at the same time though. For me, selling bitcoins to put them into whatever other economic opportunity feels stupid for me, as I'm so bullish on Bitcoin. Earning 3% on bitcoin savings though sounds great.

Insurance is a bet against events happening. "My house will not burn". How much is the house worth? How likely is it that it will in fact burn? This will determine how much it costs to cover the case of it actually burning.

If your house is worth $1million and chances it will burn are about 0.1%/year, the policy should cost $1000/year.

Now this simple example shows why insurances don't like over-covered risks. If your house in fact is not worth a million or you insured it ten times with ten companies, it becomes worth burning it down yourself. Aka over-insured assets tend to have much higher risks. I guess it will be exciting to see how a decentralized insurance deals better with this than today's insurance companies.

Necro-posting cause teambrella seams to take on the challenges addressed in this thread.

They just published this to reddit.