Blockchain Analytics is More of an Art Than Science

[LeGaulois]

Concretely, how the inacuracy is with Chainalysis:

An example with the old mixer Bitcoin Fog

CipherTrace has found a discrepancy in accuracy of about 64% for the behavioral clustering heuristic

Now let's compare:

As opposed to behavioral clustering, FindNext heuristics are able to produce false discovery rates of only 0.62% and 0.02%, respectively.

Hold on,

a non-exhaustive list of errors in Bisbee’s expert report, such as the use of bits instead of bytes leading to incorrect mathematical assumptions as well as multiple apparent incorrect identifications of change addresses. The report further highlights the missing of a number of script types, such as P2PK, P2MS, P2WSH, or P2TR and the incorrect statement that “a SegWit address begins with 3”, which also identifies P2SH addresses

Conclusion:

“The blockchain forensics and tracing tools used in this case were misused to erroneously conclude that Mr. Sterlingov was the operator of Bitcoin Fog when no such evidence exists on-chain.”

https://bitcoinmagazine.com/technical/chainalysis-the-theranos-of-blockchain-forensics

So you could be thrown into court with false accusations against you. And they will say "the investigation says it's you"
Oh great finding
Imagine what a government can do if you are a target for them An example with Julian Assange

[o_e_l_e_o]

Absolutely unbelievable some of the revelations made in this document. (https://storage.courtlistener.com/recap/gov.uscourts.dcd.232431/gov.uscourts.dcd.232431.159.1.pdf)

First look at Bisbee's report. Bear in mind that this a report from the head of investigations at Chainalysis:

The table states that P2SH is “a SegWit address that begins with 3”. That is false.

The table uses the wrong unit of data when referring to compressed and uncompressed keys. The table uses bits instead of bytes.

The report states that “The first known Bitcoin Fog transaction where the change is a P2SH-WPKH Segwit address in block 534129: 9a7e1cdb9f68573eaf64ba4f8908ebf05aee932124 6188c1c746a297e8821ffb”. Reviewing this transaction, reveals there is no witness data and no addresses participating in this transaction that are SegWit enabled.

If RF means RBF, (Replace-by-Fee), then it is important to note that a majority of transactions in the report would never have been able to utilize RBF as it was not present in Bitcoin Core until Bitcoin Core v0.12 which was released November 1st, 2016.

This is a catalogue of incredibly basic errors. These are the kinds of questions which would be asked by newbies on this forum and would be followed by 10 replies within the hour correcting them. This is from the head of investigations, and there is no chance her report was not proof read by a handful of other people in Chainalysis too. How utterly embarrassing for Chainalysis. They have absolutely no idea about the very basics of the thing the claim to be able to track.

And then look at some of the other assumptions that they have made:

The Government uses this IP address to attribute ownership and control of the Mt. Gox accounts #2 and #3 to Mr. Sterlingov. However, this IP address appears to be a VPN, or a proxy server. That is, any number of entities or persons from anywhere in the world could be using this IP address at any one time.

VPNs/Tor, how they hell to they work?

The discovery produced by the Government contains a spreadsheet authored by IRS-CI Devon Beckett, last updated on August 8, 2016. In it, he appears to refer to Chainalysis manipulating the data in this case because it did not fit in to the Government’s preconceived notions.

How very fair and unbiased. Decide their conclusion, then manipulate the data to fit it.

Mr. Sterlingov’s withdrawal pattern from Bitcoin Fog is entirely consistent with user withdrawals; this is corroborated by the Search Warrant Affidavit signed by IRS-CI Special Agent Leo Rovensky “These withdrawals occurred sporadically and in the same manner as a regular user.” The affidavit goes on to make a leap of logic stating that the likely reason Mr. Sterlingov’s withdrawals match other user withdrawals is that he was trying to obfuscate his ownership.

If your pattern of use is completely average and similar to everyone else's pattern of use, the only explanation is that you are doing it deliberately to obfuscate things, and not, you know, that you are just an average user. What a wild assumption.

An incredible collection of incompetence and ignorance. What a tragedy that so much of this space revolves around the complete hogwash that these scam merchants peddle to governments and centralized exchanges.

[kryptqnick]

There are various things that sort of work somehow but aren't fully understood by people. One of them is GPT, as far as I know. Some things are understood by the developers, but a lot is like a black box situation, there's no deep understanding on how exactly something picks up patterns that it does, how it acts based on those patterns, etc. But that's kind of okay because linguistic models are known to be faulty, it's common knowledge they should be used with caution and should not be entrusted with decision-making. Blockchain analytics is a different thing if it can be used as evidence in court or affect major decisions on official policies.
I hope that some sort of research can be conducted to assess how often it makes mistakes and what kind of mistakes. For example, fingerprints are largely accurate but not 100%, and some people have been wrongfully convicted because of that. But there's research assessing the percentage of false positive cases, which is around 0.1%. Of course, if there's a chance of mistake, even a small one, I do think that it should mean that a person can't be convicted based on that evidence alone, but if Blockchain analysis actually have a much larger margin of error or if it's impossible to estimate it, it shouldn't be allowed to be used as evidence at all.

[Spaceman1000$]

Blockchain Analytics is More of an Art Than Science

I am not saying their software is so wrong, but knowing there are false positives, it can be very dangerous.
Even if the false positive rate is only about lets say 0.1%, considering the millions address/transactions being tracked, it can make a lot of dammages.
In example: ""led to unjustified account restrictions and – even worse – land unsuspecting individuals on the radar of law enforcement agencies without probable cause.""

- Hello Chainalysis,
Do you know if your software is accurate?

- I have no scientific evidence for the accuracy.

- Really?

- Really, but clients say so

- Clients, you mean governments lying depending on their agenda?

- Yes

Chainalysis' head of investigations doesn't seem to have a great understanding of whether her company's flagship software even works.

Ekeland said Chainalysis’ Reactor is “a black box algorithm” that “relies on junk science.”

Coindesk, the woke media, didn't like the article talking bad about Chainalysis and decided to remove it
Chainalysis>>>Coindesk>>>DigitalCurrencyGroup
so here it is
https://bitcoinmagazine.com/technical/chainalysis-investigations-lead-is-unaware-of-scientific-evidence
You may be interested in Blockchain Analytics is More of an Art Than Science

Going by the article, there are something's I find even offensive, where is been quoted that Elizabeth Bisbee, the head of chainalysis investigations  doesn't seem to have a great understanding of whether her company's flagship software even works.
That's an indictment already on the company, especially considering the fact it is been said that people financial privacy are been violated.
This are serious red flags. Because user's are already given a negative feedback.

[FanEagle]

I think it is more about who the customer is. If we are talking about governments, they will not give you a problem to solve, they will give you what solution they want, and you to work your way back from that to find the problem they want to find. So that means it is not going to be that easy to do it, and there will be a lot of people who are not certain about it as well. I know that it is not that simple but it could be done and I think it would be a smart decision to not say no to a government.

What do you expect these companies to do, to go out there and end up saying no to government and risk their entire business? They would bankrupt in a moment as soon as government starts attacking them for not doing what they want them to do.

[BlackHatCoiner]

Blockchain analysis companies are the practice of "being guilty until proven innocent". Living under a state which can put me behind the bars because I might be involved with some activity, with absolutely no scientific evidence is totalitarian. Literally everyone in the blockchain is potentially involved in some illegal activity.

But it's just genius. They've convinced the world that it's needed. A weapon to intrude into people's liberty and having them voluntarily handing it over is just the ultimate weapon.

[panganib999]

Such softwares used for tracing addresses only instigate stress on a victim that got scammed or lost their coin. A scam victim, who wants to track the address where he/she sent a transaction, needs to bring evidences; chats, photos and money to the agency that claim to trace blockchain transactions. Most victims end up losing more money added to the lost money. Its existence also gave birth to fraudulent agencies claiming to retrieve lost coins or tracking a scammer; with the slogan, a person that fell for scam can easily fall again. The results always appear differently based on the software used to track an address. If the software can't retrieve the lost coin, why depend on unrealistic results, in the name of tracing a person who may not be behind the crime or scam?

So what are we proposing the victims do should they experience hackings and scammings? Scoff it off and just move on? What if it's massive amounts of money we're talking about, like money that could pay off a mortgage or something, do we still tell them to man up and move on since "they're more likely to spend more money looking for the hacker and the money that was stolen from them rather than just moving on and working to earn that money back? Isn't that a little defeatist and at best even promotive of the heinous acts these hackers do on the daily?

Just cause our current system's crap doesn't mean we can't have people who deserve it be given the access to it, sure there are false positives and all that, but at the end of the day the more we use it the more propensity for improvement. If we keep throwing crap at it without giving it a spin first, how will we get to the point that we're eager to achieve?

[o_e_l_e_o]

What do you expect these companies to do

Not exist in the first place!

They obviously aren't going to say no to governments because governments are their biggest customer. They exist precisely for governments, so they can present made up findings filled with inaccuracies as the "evidence" the government wants to prosecute individuals they've decided they want to prosecute, regardless of any actual guilt or wrongdoing.

What should really happen is that blockchain analysis companies go bust and disappear. If we all stopped using centralized exchanges and used bitcoin properly peer to peer as it was designed, that removes a big chunk of their cash flow. And now we need to fight back in courts and prove as we've seen above that their entire analyses are so flawed that no one should take them seriously for any purpose, certainly not for a criminal trial.

If we keep throwing crap at it without giving it a spin first, how will we get to the point that we're eager to achieve?

The point we want to achieve is for everyone to realize blockchain analysis is provably nonsense.

[Kakmakr]

It would be interesting to know how Chainalysis acquire accurate data that has gone through Mixer services? I can understand that data mining can be very accurate, if they were only dealing with unscrambled inputs and outputs from the Blockchain, but what happens after those tokens go through a Mixer service?

Yes, it is possible to mine through data that were recovered after a Mixer service were shutdown... and if they kept logs, but what happens if you push it through active Mixer services that does not keep any logs? 

[o_e_l_e_o]

It would be interesting to know how Chainalysis acquire accurate data that has gone through Mixer services?

As with everything else they do - they guess. Their guesses may or may not be accurate, depending on the service used to mix the coins. There are good mixers and bad mixers, Chainalysis have demixed Wasabi coinjoins in the past (which is how the DAO Ethereum hacker was identified), people can leak data via external methods such as browser fingerprints or IP addresses, and so on. Chainalysis are never going to willingly reveal their methods, because then they become easier to defend against.

But as we've seen in this thread the government don't seem to care about accuracy at all, and only care about being given "data" which fits their preconceived conclusions.

[LeGaulois]

Howdy people

The story continues. Let's have more fun about Chainalysis

CHAINALYSIS DENOUNCES BITCOIN CORE CONTRIBUTOR AS “UNQUALIFIED”

Chainalysis attempts to render Bitcoin Core contributor Bryan Bishop as “unqualified” to audit Chainalysis’ source code as the surveillance firm’s own experts stumble cluelessly around the blockchain.

With the justice case US vs. Sterlingov (the mixer Bitcoin Fog), since their software is not accurate, the defense requested the Chainalysis’ source code to be audited.
They don't like this idea! Guess why? It's going to put their business at risk.

So their point is to say Bishop is not qualified. Bishop, who
- is a Bitcoin Core contributor,
- has participated in discussions on
. elliptic curve cryptography,
. ECDSA signature schemes,
. Schnorr signature schemes,
. BLS signature schemes,
. signature aggregation schemes,
. post-quantum cryptography,
. quantum mining,
. scrypt password hashing""

it's like saying your dentist isn't qualified to treat your toothache.

Lol.

Yes LOL

Extreme bio hacker


https://www.courtlistener.com/docket/59988850/185/united-states-v-sterlingov/

[o_e_l_e_o]

Lol.

They first refused to allow Laurent Salat to audit the code, simultaneously claiming that he was not a qualified expert, but then also claiming that he would steal their source code for OXT, which he runs. Amazing that he is both unqualified and too qualified at the same time.

Now they claim that a literal Bitcoin Core contributor is unqualified? The same people who as we saw earlier in this thread don't understand the different between legacy and segwit addresses and don't even understand the difference between bits and bytes are in no place to pass judgement on anyone, least of all a Bitcoin contributor. They are absolutely desperate that no one sees their code, and learns just how unscientific and deeply flawed it is.

Blockchain analysis is a scam.

[SamReomo]

You're wrong.

Such companies track more than what you may think and not only to help an individual ripped off.
It's called data mining. They collect millions stuff.
Big companies like Bitstamp , etoro, and other CEXs are using it. So what do you think they're doing. CEXs provide them a lot of information about their customers trading on their platform.

It's not so different than a data broker.

data mining is such a powerfull industry, people do not realize what can be done with it... against them...

I agree with you and truly those companies can track a lot of things that we can't even imagine. They have internal links with many of the centralized bodies and large companies also provide details to them because they may get incentivize later on when cases are solved. Most of the Centralized exchanges willingly share data of the customers with them when any of the hacked coins are transferred into the wallets of those exchanges. That's also a way for them to gather data about the criminals who have committed the crime.

But, I still believe that sometimes the innocent ones can be caught due to the fraud of someone else. The criminals can sell those coins privately to the victims and some less-knowledgeable victims can purchase those coins for cheaper and send them their exchange's hot wallet address to receive to coins. Although those victims are innocent but those companies don't really care much about their innocence and action against them to show off their tracing skills. They may

Those type of companies are known for link building with other companies and thus centralized exchanges and all those other companies willingly provide information about customers and users to the companies like Chainalysis. I may be wrong but I think that governmental agencies also help those companies privately because those type of tracing companies can help to find the criminals and later on those agencies may seize the stolen coins from the victims.

Well, we really don't know their methods in details and they will never share their research details in public, and that's why I also believe that their data gathering can go wrong sometimes and innocent people may face wrong allegations because of the wrong practices of the companies like that. Sometimes they are so concerned about data mining that they forget that innocent people can also face tough times because of the mastermind criminals. Some criminals take ID card and other details of the innocent people and may register face accounts on their name and complete the KYC verification of the sites. In that case they will consider the victim as responsible for the crime not the mastermind criminals who used those innocents victims details to save himself from the crime that he has done.

[nutildah]

What do you expect these companies to do

Not exist in the first place!

...

What should really happen is that blockchain analysis companies go bust and disappear. If we all stopped using centralized exchanges and used bitcoin properly peer to peer as it was designed, that removes a big chunk of their cash flow...

But these aren't very realistic outcomes in real life. If Chainalysis were to fold tomorrow, you can bet there'd be at least a handful of similar entities vying for their old contracts & market share.

With the justice case US vs. Sterlingov (the mixer Bitcoin Fog), since their software is not accurate, the defense requested the Chainalysis’ source code to be audited.
They don't like this idea! Guess why? It's going to put their business at risk.

The code should absolutely be auditable... I agree with the overall sentiment in this thread that its ridiculous to put someone behind bars for what is potentially a very long time without thorough "due process." If Sterlingov was indeed just a user of Bitcoin Fog then it would be an egregious miscarriage of justice to sentence him based on potentially faulty analysis.

[o_e_l_e_o]

Sometimes they are so concerned about data mining that they forget that innocent people can also face tough times

They don't forget - they simply don't care.

The government don't care in the slightest when banks launder literally trillions of dollars. They get a completely meaningless token fine and that's it. No arrests, no criminal charges, no seizures, no shutdowns, and allowed to continue to launder more money in the future. This is all fine because the banks bribe our politicians and freely hand over all your data to the government when they want it. But when a piece of open source code allows the average person to maintain some semblance of privacy against the government's various mass surveillance programs, then all hell breaks loose and they absolutely must prosecute someone. They don't actually care if the people they prosecute are actually guilty, as long as the set an example to the rest of us that you should be good little citizens and never step out of line. This is not about preventing money laundering in the slightest - if it was, they would clamp down on the banks which do it constantly. It's about surveilling and controlling the populace.

But these aren't very realistic outcomes in real life. If Chainalysis were to fold tomorrow, you can bet there'd be at least a handful of similar entities vying for their old contracts & market share.

Absolutely. We already have dozens of blockchain analysis companies out there. My point was that if everyone who used bitcoin stopped using things like centralized exchanges or wallets like Wasabi or Trezor which all directly fund these blockchain analysis companies, then that would cut off a large part of their funding and many would cease to exist (not to mention making general blockchain analysis much more difficult since you would no longer have KYC linked addresses and such).

The code should absolutely be auditable... I agree with the overall sentiment in this thread that its ridiculous to put someone behind bars for what is potentially a very long time without thorough "due process." If Sterlingov was indeed just a user of Bitcoin Fog then it would be an egregious miscarriage of justice to sentence him based on potentially faulty analysis.

It's pretty damning just how desperate Chainalysis are to not let anyone who even remotely understands bitcoin view their code.

[BlackHatCoiner]

Well, we really don't know their methods in details and they will never share their research details in public

This and the part where they won't endorse code auditing reminds somewhat of the open-source / closed-source debate around security (AKA, "Security through obscurity"). That certain developers choose to not reveal the source code for security reasons, which is completely debunked, as most of the times cyber attacks don't happen after thorough research on the code, but on techniques that are source-independent. On the other hand, open source code greatly improves security, as it sets no limit as to who can be a contributor.

My point was that if everyone who used bitcoin stopped using things like centralized exchanges or wallets like Wasabi or Trezor which all directly fund these blockchain analysis companies, then that would cut off a large part of their funding and many would cease to exist (not to mention making general blockchain analysis much more difficult since you would no longer have KYC linked addresses and such).

Chain analysis companies are getting funded with millions of dollars from the US government, so it isn't just Wasabi's and centralized exchanges' fault here. But, sure, if half of the Bitcoin userbase stopped using Binance and switched to self-custody, they'd start panicking.

[o_e_l_e_o]

Chain analysis companies are getting funded with millions of dollars from the US government, so it isn't just Wasabi's and centralized exchanges' fault here. But, sure, if half of the Bitcoin userbase stopped using Binance and switched to self-custody, they'd start panicking.

Sure. There will always be a government owned blockchain analysis company trying to spy on you. But if you cut off a huge amount of the data they get fed from the likes of centralized exchanges providing your KYC data and wallet addresses, payment processors like BitPay sharing details of your spending habits, closed source wallets which depend on their own servers sharing your wallet addresses, and so on, their bullshit "analyses" would be even more provably bullshit.

If some random account showed up to this forum and said "Hey, I've figured out a way to track bitcoin transactions, send me any address and 100,000 sats and I'll tell you who owns it!" while refusing to provide any technical details or let anyone see their code, they would rightly be called a scammer and rapidly be tagged with a newbie warning flag. But somehow if you show up to the US government and say the same thing they give you a multi-million dollar contract.

[spectre71]

As someone mentioned earlier they can bake the desired outcome what the client (GOVT) is looking for really. The govt literally has unlimited resources and can bury an innocent person.

What if someone hasn't a single nefarious intent  but just pumps 1-2 K through a mixer just natural geek curiosity and put it back in their wallet and the whole wallet get's seized or your coins marked? Many people just want to be anon with no criminal intent. If your hiding you must be doing something wrong right?

[SamReomo]

Sometimes they are so concerned about data mining that they forget that innocent people can also face tough times

They don't forget - they simply don't care.

The government don't care in the slightest when banks launder literally trillions of dollars. They get a completely meaningless token fine and that's it. No arrests, no criminal charges, no seizures, no shutdowns, and allowed to continue to launder more money in the future. This is all fine because the banks bribe our politicians and freely hand over all your data to the government when they want it. But when a piece of open source code allows the average person to maintain some semblance of privacy against the government's various mass surveillance programs, then all hell breaks loose and they absolutely must prosecute someone. They don't actually care if the people they prosecute are actually guilty, as long as the set an example to the rest of us that you should be good little citizens and never step out of line. This is not about preventing money laundering in the slightest - if it was, they would clamp down on the banks which do it constantly. It's about surveilling and controlling the populace.

Yes, you're right they simply don't care because they won't be affected if some innocent person's life get destroyed because of someone else's criminal mindset. They have become so cold that only money matters for them and if they get money then everything is okay for them. The governments give free hand to banks and the banks can do whatever they want to do with those trillions of dollars that they launder. The banks have been following those practices for centuries and the politicians have been supporting them. The real culprit in the system are those politicians who sell their souls for the money and they use malpractices to fix all of the written documentation and clear all of the records of the banks that laundered the money. When those records are cleared the banks get courage to do more money laundering activites.

The centralized system is full of corrupt people and when a corrupt person come in power then he/she does everything to fill his/her pockets. I also agree that the ones who go against surveillance activities of the government always face a worst ending. They really don't want that open source code which can fail their surveillance activities because such software can reduce their control over population and anyone who make it can be harmful for them. They will do everything to control the population and they are very successful in their wrong motives. That time isn't far when everyone's life will be controlled by the so called governmental bodies, and the worst thing is that if we even do something to help others to be released form that control then that could be a deadly step for our lives. A laymen isn't anything for them and if a person is not known in social media then they can do whatever they want with that person. I believe that's one of the reasons why Satoshi left the Bitcoin after giving it to public because he knew that if he continue to post on this forum or somewhere else then those governmental authorities will find a way to trace him and could make many charges against him because the created a financial system like Bitcoin.

Well, we really don't know their methods in details and they will never share their research details in public

This and the part where they won't endorse code auditing reminds somewhat of the open-source / closed-source debate around security (AKA, "Security through obscurity"). That certain developers choose to not reveal the source code for security reasons, which is completely debunked, as most of the times cyber attacks don't happen after thorough research on the code, but on techniques that are source-independent. On the other hand, open source code greatly improves security, as it sets no limit as to who can be a contributor.

Very true! The open source code is available to everyone and anyone can improve it if they find flaws in it. On the other hand the organizations that doesn't open their source code will limit it to few people who may not be able to find all flaws of the code and the hackers or the malicious entities may find those flaws in the code by using their techniques and as a result the users and their privacy get compromised. Most of the time the closed source applications are full of vulnerabilities and when a hacker finds those vulnerabilities then he/she can take advantage of it and steal data of millions of users. The open source code isn't much vulnerable by nature because almost everyone can fix the vulnerabilities in the code and they can also improve the security of it by contributing their share on it.

[BlackHatCoiner]

On the other hand the organizations that doesn't open their source code will limit it to few people who may not be able to find all flaws of the code and the hackers or the malicious entities may find those flaws in the code

Or, they themselves could attack their own software if there's a benefit. They won't reveal you the manner which they discriminate coins, so why wouldn't they? It can go completely unnoticed.

Most of the time the closed source applications are full of vulnerabilities and when a hacker finds those vulnerabilities then he/she can take advantage of it and steal data of millions of users.

This is so true. In fact, it's been noticed that many times developers who write closed-source software do follow the "security-through-obscurity" principle. People try obfuscation techniques believing that provides security, in stuff like web apps and steganography. Believing that closed-source is more secure than open-source, besides debunked, is a sign of utter arrogance; believing that no one can contribute to the security of your project as much as you've already done is excessive self-esteem. Windows is tangible evidence that this is bad practice.

Everything that includes cryptography should be developed transparently.