Segwit Adoption. I looked at some data, and we are not there yet.

[tranthidung]

It's only one line per day, so around 250 kB in total. But I'll change my script a bit, see if I can improve performance so it doesn't take days.

Honestly, weeks ago, I downloaded your original data and it is huge, exceed my computer memory so I gave up. With the lighter data set, I can play with it.

I'm kinda interested to see how many 3-addresses are in use before SegWit.

Sure, I can join the party and I am familiar when handling such stuffs.

Anyway, it was requested by bitmover first so I won't break his fun here. If bitmover needs help, please ask. Or if after he releases his work and I see I can make polishment I will join the party.  

A little bit derail:

Could you dump data for UTXO and ages or creation dates that can be used to build up such graph? If possible, one per month or even one per quarter.
https://hodlwave.com/#

[1714617606]

1714617606

[LoyceV]

Could you dump data for UTXO and ages or creation dates that can be used to build up such graph? If possible, one per month or even one per quarter.

Sorry, I'll skip this one. Checking UTXO age means checking outputs and inputs. I haven't downloaded the latter yet. I don't think I can do this without adding everything to a database (which I can't do).

[franky1]

the best metric in my opinion is the number of SegWit inputs per block. for example if a block has 2000 transactions and that is 2500 inputs and out of those 1500 are SegWit inputs then you can say the adoption is 60%.

not really. many blocks are not filled with user transactions.. but mixer and exchange cold wallet fills and empties

also coins per address is not a fair measure because a exchange with 10,000 coins per single p2sh multisig. vs 10,000 people with 1btc on legacy. does not show the reality of usage.

for me segwit is just the gateway address for LN. i still use legacy
yep i know they made legacy 4x more expensive. but thats just corporate politics at play to make people lean into wanting to use other networks. and i refuse to play their game because of it
heck segwit is not even cheaper then the 2015 promise of cheaper transactions.. and yep i stand by my point that its not segwit is cheaper. but devs made legacy more expensive.

heck until i see sipa stop asking for donations in legacy. then i know that he finally trusts his own code.. but so far. he is still asking for donations in legacy so something is up. and its kinda weird he has not converted. and yep its been 2+ years, so not many excuses can explain his delay in using segwit himself.

segwit complex code does not give it any intrinsic feature to explain the cost difference of legacy vs segwit
its just muddy math to miscount some bytes and pretend those bites have no value. and also multiply cost of other bytes
this stuff can be removed without breaking bitcoin and make transaction fee's more honest

infact removing the muck. will allow more transactions into a block. and make all transactions cheaper without ripping up the 4mb area now deemed allowed

so segwit is not a feature benefit. but just a cludge of code and math to make legacy(true) bitcoin seem slow and expensive just to invite people over to other networks

and that is why although i test, play around, bug test the muck...
.. when using bitcoin for actual uses i use legacy. and proud of it

[Carlton Banks]

The reason behind bech32 holding less bitcoin is most of the services don't support bech32, as a result they don't hold their bitcoin in bech32 as well (if they would, they would support too I think).
It's sad. Segwit adoption would give us more space to block.

It absolutely doesn't matter, you can support bigger blocks if you use bech32. Or support small blocks if you use legacy addresses. And you can change your mind anytime you like, the legacy format will always be supported (as long as people still have BTC stored in legacy addresses at the least).

[bitmover]

We only need data since 02/26/2018 (Bitcoin Core segwit release date).

I'm kinda interested to see how many 3-addresses are in use before SegWit.

hum.  I got it. Then we could safely mark those addresses as P2SH-multisig, and all address past 2018 as *possible/probable* P2SH-SegWit.

Anyway, I cannot handle more than 30gb files due to my hardware limitations. Maybe 50gb, but I would need to delete it as fast as possible...

When you have the data, please extract data since february 2018 and I will try to work with it.

Honestly, weeks ago, I downloaded your original data and it is huge, exceed my computer memory so I gave up. With the lighter data set, I can play with it.

I'm kinda interested to see how many 3-addresses are in use before SegWit.

Sure, I can join the party and I am familiar when handling such stuffs.

Anyway, it was requested by bitmover first so I won't break his fun here. If bitmover needs help, please ask. Or if after he releases his work and I see I can make polishment I will join the party. 

Do you work with python? How do you work with data from the forum? We can try to work together.

Pandas can handle such files smoothly, the problem is the file size for me. I cannot store it.

[LoyceMobile]

Anyway, I cannot handle more than 30gb files due to my hardware limitations. Maybe 50gb, but I would need to delete it as fast as possible...

Lol, I said kB, not GB.

[LoyceV]

Here you go:
https://loyce.club/other/daily_transaction_data.txt (not a large file).

Code:

date,1 addy count,3 addy count,Bech32 addy count,1 addy value,3 addy value,Bech32 addy value
2009-01-03,1,0,0,5000000000,,
2009-01-09,14,0,0,70000000000,,
2009-01-10,61,0,0,305000000000,,
2009-01-11,93,0,0,465000000000,,
2009-01-12,106,0,0,487900000000,,
2009-01-13,123,0,0,615000000000,,
2009-01-14,130,0,0,651100000000,,
2009-01-15,140,0,0,680000000000,,
2009-01-16,110,0,0,560000000000,,
2009-01-17,109,0,0,545000000000,,
2009-01-18,108,0,0,550000000000,,
2009-01-19,117,0,0,652500000000,,
2009-01-20,115,0,0,610000000000,,
2009-01-21,103,0,0,520000000000,,
2009-01-22,92,0,0,505000000000,,
2009-01-23,86,0,0,465000000000,,
2009-01-24,202,0,0,1055000000000,,
2009-01-25,192,0,0,960000000000,,
2009-01-26,97,0,0,580000000000,,
2009-01-27,98,0,0,500000000000,,
2009-01-28,112,0,0,560000000000,,
2009-01-29,122,0,0,665000000000,,
2009-01-30,117,0,0,585000000000,,
2009-01-31,139,0,0,790000000000,,
2009-02-01,114,0,0,575000000000,,
2009-02-02,129,0,0,645000000000,,
2009-02-03,169,0,0,768652000000,,
2009-02-04,119,0,0,595000000000,,
2009-02-05,126,0,0,630000000000,,
2009-02-06,122,0,0,610000000000,,
2009-02-07,134,0,0,764800000000,,
2009-02-08,133,0,0,760000000000,,
2009-02-09,135,0,0,890000000000,,
2009-02-10,120,0,0,600000000000,,
2009-02-11,135,0,0,720004000000,,
2009-02-12,119,0,0,595000000000,,
2009-02-13,124,0,0,620000000000,,
2009-02-14,136,0,0,680000000000,,
2009-02-15,131,0,0,730000000000,,
2009-02-16,123,0,0,760000000000,,
2009-02-17,117,0,0,585000000000,,
2009-02-18,118,0,0,590000000000,,
2009-02-19,125,0,0,700000000000,,
2009-02-20,122,0,0,610000000000,,
2009-02-21,114,0,0,570000000000,,
2009-02-22,125,0,0,710000000000,,
2009-02-23,108,0,0,540000000000,,
2009-02-24,106,0,0,530000000000,,
2009-02-25,105,0,0,525000000000,,
2009-02-26,107,0,0,535000000000,,
........
........
........
2020-08-19,455998,418104,90410,136252229216711,38207856950231,15624696315877
2020-08-20,431996,411757,88449,127453217228637,36848455436766,21099695883662
2020-08-21,412221,399109,85242,99638225201061,35501994075309,15331970398334
2020-08-22,393422,388158,79152,100770674397428,24009846308666,11448220279848
2020-08-23,332641,329314,97388,108686689271991,18966996489817,9915334638421
2020-08-24,402641,400183,94616,141110955027421,29514987238757,17162191547198
2020-08-25,412172,402009,86438,107263910642399,32825094002986,22873858572973
2020-08-26,440030,401670,88599,143623748098467,29820813517272,22859099382760
2020-08-27,447042,417158,87891,146995819475449,33395946820914,24963296835877
2020-08-28,448985,423350,93206,159916151556009,33867915491339,23437629524504
2020-08-29,366372,357391,91009,132619714706900,27372611855909,15723818763411
2020-08-30,325873,328772,85654,136552986252641,23511838209173,15911543308935
2020-08-31,386097,384312,90817,203877828495697,38504044452467,28029481732957
2020-09-01,492706,449522,99004,209811748165541,34996845884823,21393838306849
2020-09-02,440824,405601,87240,168865376112661,39423086664485,23795230739108
2020-09-03,422576,390596,82675,193296507251844,48199231230766,33428509966252
2020-09-04,408133,400245,88069,191089640423051,46863139246091,29442770150320
2020-09-05,430865,412027,87480,186516086992197,40759687665788,16787874526182
2020-09-06,332374,314968,85618,196316947553197,19714405301053,13553404911396
2020-09-07,400944,389258,102164,244224191526102,33667144918290,21734247464610
2020-09-08,420383,409097,100932,217298084984760,33082964901277,21141231220076
2020-09-09,418029,381427,93141,270135854861231,30556325713789,18567284072225
2020-09-10,461251,424576,96469,300395770810079,31388572531663,18275356569830
2020-09-11,422281,406332,86883,264023785887788,34335636408123,15458624579473
2020-09-12,371966,341539,101941,224685762724077,22011541491289,13507511387783
2020-09-13,311754,326494,90613,231247951237279,18379452515568,13139930708959
2020-09-14,442100,423395,110181,222791990302199,36774187011682,17800868547361
2020-09-15,439727,414927,94253,240335258985626,34624492186780,16624642317853
2020-09-16,467194,426782,98787,259101781845015,30461023972239,16288246362157
2020-09-17,452775,421765,107356,368884346864198,30106102762854,16055285690140
2020-09-18,423498,388849,97778,330303462627097,31771465450871,17216119427137
2020-09-19,378445,358006,89279,252241323062039,22521877167060,12104279510058
2020-09-20,311033,304381,93879,205697818063972,20447180370893,10391061704735
2020-09-21,385770,365454,94229,299078274803435,29478686288438,18519925666028
2020-09-22,427351,394309,83038,232390648025495,30988326087418,16854306998331
2020-09-23,462538,419047,91267,216135523079559,30705234610575,19836155738220
2020-09-24,423869,387362,87268,177757649122261,48901373076417,19368262370400
2020-09-25,467986,425300,96876,233256358161864,34736176445301,16666682298963
2020-09-26,362458,328718,80320,207125291814868,19878886442851,12318303182361
2020-09-27,319715,302807,88679,181261455107958,18890650791005,12339280217229
2020-09-28,379067,355136,89864,183982536273286,28060841265664,17930614261348
2020-09-29,463680,420909,96630,199153334182133,37652598533664,22538898944529
2020-09-30,463941,423992,92916,186907435829579,29488883184414,22428955318285
2020-10-01,408649,387183,89891,148667867186388,34329773260961,22478894128316
2020-10-02,430700,396608,88743,205582539171409,44434139116039,21686834420686
2020-10-03,405259,369568,86183,183960263488794,18663367656904,15176201005865
2020-10-04,332155,310898,97778,157136459200868,16294938563812,14640577293029
2020-10-05,377931,358868,92951,152604739674116,30914529091541,21342436494352
2020-10-06,424386,401365,94972,172777475136757,26433346060214,25528796101412
2020-10-07,497160,432267,101679,156837025837901,31467070848206,24058196902115
2020-10-08,424525,388665,99234,148279800469648,33195952719264,21323758178204
2020-10-09,446349,409324,101081,219304817528092,35456605310641,22127787414543
2020-10-10,394682,340468,93154,148556601637243,20634870121461,24250808119111
2020-10-11,315537,298082,93751,175778846331667,15871740231264,22141396117966
2020-10-12,398379,384215,95871,164618683866396,30962285663738,22221566024495
2020-10-13,434967,404588,104128,174396414428013,32761557039880,21443908625889
2020-10-14,425288,376365,93526,196135129778647,25406432732324,19028485442051
2020-10-15,470404,417658,100769,213815529141293,30580762465844,25534292426190
2020-10-16,442766,400530,98200,186800827562585,30549276986448,19822545189692
2020-10-17,351471,328273,88531,219815294120207,16664846804095,11510493743563
2020-10-18,301198,296557,72058,185307481115335,15017053699890,13349930096674

The column "value" is in satoshi. Both addy count and value are the total for each day.

It doesn't get automated updates. If anyone wants an update, send me a message.

[bitmover]

Here you go:
https://loyce.club/other/daily_transaction_data.txt (not a large file).

Thanks. I will work on it soon.

let me show what I did today.

I discovered that you have  transaction_count.txt and witness_count.txt in your Bitcoin block data available in CSV format

Then, I just made a quick search in blockchair APi documentation and found that it is very easy to calculate the segwit adoption.
witness_count = Number of transactions in the block containing witness information

You just need to divide witness_count by transaction count to get the percentage.

The problem is that some miners, for some reason, only get SegWit transactions in a block (sometimes just one). This makes the results very weird as we will see a lot of 100% adoption blocks (which is not false, but it is not representative).

I stripped block data before block 481824 (segwit activation)


Very ugly chart.

Then i decided to group transactions by month and see what was the adoption per month (instead of per block). The results were much nicer:



Curious fact: There is a downtrend in segwit adoption at 2020-3, which is the peak of the pandemic crisis. I guess people were moving old legacy coins to cash out and this affected the total "adoption"

[bitmover]

We only need data since 02/26/2018 (Bitcoin Core segwit release date).

I'm kinda interested to see how many 3-addresses are in use before SegWit.

By 2017-08-24 (the day block 481824 was mined ) there were:

Legacy Add Count        426910
P2SH-Multisig Add Count           86988
Bech32 Add Count            17
Legacy Total Balance   1292594.113662 BTC
P2SH-Multisig Total Balance      356041.326315 BTC
Bech32 Total Balance        51.999394 BTC

as there are 17 Bech32 addresses, there might some P2SH-Segwit as well. So I looked one day earlier as well.
2017-08-23
Legacy Add Count        705680
P2SH Add Count          134520
Bech32 Add Count             0.000000
Legacy Total Balance   1977622.672343 BTC
P2SH Total Balance      386197.053188 BTC
Bech32 Total Balance         0.000000 BTC


It is curious to see that the total number of P2SH decreased the day after segwit was activated
47.532 address (~35% reduction) and 30K BTC (~8% reduction).

Could this be something related to BCH fud?

[Wind_FURY]

the best metric in my opinion is the number of SegWit inputs per block. for example if a block has 2000 transactions and that is 2500 inputs and out of those 1500 are SegWit inputs then you can say the adoption is 60%.

not really. many blocks are not filled with user transactions.. but mixer and exchange cold wallet fills and empties

also coins per address is not a fair measure because a exchange with 10,000 coins per single p2sh multisig. vs 10,000 people with 1btc on legacy. does not show the reality of usage.

for me segwit is just the gateway address for LN. i still use legacy
yep i know they made legacy 4x more expensive. but thats just corporate politics at play to make people lean into wanting to use other networks. and i refuse to play their game because of it
heck segwit is not even cheaper then the 2015 promise of cheaper transactions.. and yep i stand by my point that its not segwit is cheaper. but devs made legacy more expensive.

Besides being the least problematic onchain capacity increase, wasn't Segwit mainly a transaction malleability-problem fix, and brought other benefits for the network?

[LoyceV]

Very ugly chart.

Try a scatter graph instead of lines

[BrewMaster]

for me segwit is just the gateway address for LN. i still use legacy
yep i know they made legacy 4x more expensive. but thats just corporate politics at play to make people lean into wanting to use other networks. and i refuse to play their game because of it
heck segwit is not even cheaper then the 2015 promise of cheaper transactions.. and yep i stand by my point that its not segwit is cheaper. but devs made legacy more expensive.

just because the new thing is cheaper doesn't mean the old thing becomes more expensive. the legacy transactions have the same size as they always did and their fee is just as it has always been. the only thing that changed is that we are now computing fee based on virtual size and SegWit transactions have a smaller virtual size while legacy transactions have the same virtual size as before.

segwit complex code does not give it any intrinsic feature to explain the cost difference of legacy vs segwit

it is not complex at all, at least not any more complex than any other part of bitcoin. when you don't understand the bitcoin technology of course it is going to look complex to you.

[tranthidung]

bitmover,

I don't get experience with Python and don't know how to use it. If you need help with time-series plots, I can help you with weekly updates.

Of course, I am supper happy if we can work together in the future.
LoyceV,

There are 4 time-series plots. I might polish them tomorrow with a narrower period and might stratified them over years.

Hope it helps!  

[bitmover]

Try a scatter graph instead of lines

Better result...
The data is not representative when we look at each block, as I said there are far too many 100% segwit blocks.


There are also 100% legacy blocks. Are there miners who still do not want to add segwit transactions in their blocks? That's a possibility. Take a look at the bottom of the chart.

bitmover,

I don't get experience with Python and don't know how to use it. If you need help with time-series plots, I can help you with weekly updates.

Which platform do you use? You are making nice graphics. It is not easy to do that in excel, unless you have a super computer (excel is so slow). powerbi?

This chart is beautiful.
but there is something wrong with the Legacy count in 2020. Legacy add count is still bigger than the bech32 and p2sh.
I made the same chart. Take a look:
Sorry, I understood what you did now: You didn't stack, it is not an area chart but a line chart.
I made a stacked version of yours now. My first chart had problems as well:

I had problems with this chart as well. Those spikes are probably a data problem as I have them in my charts as well. I decided to make a 100% stacked bar, and the results were nicer:

[tranthidung]

Which platform do you use? You are making nice graphics. It is not easy to do that in excel, unless you have a super computer (excel is so slow). powerbi?

You can use Stata, SPSS, R or Python. If you have good experience with Python, there are packages for you to use (on Python and on R as well). People have switched to use R or Python because they are free and of course much stronger and fastly updated with community contributions.

Excel is not good to play with plots and analyses.

I had problems with this chart as well. Those spikes are probably a data problem as I have them in my charts as well.

You can try with different recasting options (area, line, connected, scatter) and choose an option that gives you best plots.  

[Quickseller]

How many addresses out there are cold storage / physical things that have funds sitting on them in legacy addresses?
They will probably be there for a long time.
How many are paper wallets?
There are a few other reasons I can think of that legacy is still around and being used.

-Dave

Some people followed a specific procedure to generate, secure and backup keys, or xpriv keys, and are confident the keys are secure and safe against loss. They may be nervous about generating new private keys for segwit addresses. I started using SW years ago, but this was my biggest hurdle to switch over.

[pooya87]

There are also 100% legacy blocks. Are there miners who still do not want to add segwit transactions in their blocks? That's a possibility.

do you know the number of transactions in blocks with 100% legacy/SegWit transactions?
it is possible that these blocks contain between 1 transaction (coinbase) and a handful of them which makes it easy to have 100% of one type only.

[bitmover]

do you know the number of transactions in blocks with 100% legacy/SegWit transactions?

Yes, I have all of them:

it is possible that these blocks contain between 1 transaction (coinbase) and a handful of them which makes it easy to have 100% of one type only.

You are right, most of them are coinbase tx, but not all.

I filtered all blocks with less than 2 transactions here in these tables:


As we can see, 20% of all 0% segwit have more than 1 transaction (coinbase). Most of them in the begin, when adoption was still struggling...
But only 2% of 100% segwit blocks have more than 1 transaction.

And there was still  a block with 2992 legacy transactions (and zero segwit) in 2020! I searched to find out who that miner was (block 616561) with no success.

https://blockchair.com/bitcoin/block/616561
Miner: Unkown (in blockchair and blockchain.com databases)

[pooya87]

And there was still  a block with 2992 legacy transactions (and zero segwit) in 2020! I searched to find out who that miner was (block 616561) with no success.

https://blockchair.com/bitcoin/block/616561
Miner: Unkown (in blockchair and blockchain.com databases)

interesting find.
that block's coinbase's signature script is this:

Code:

03716809049f203f5efabe6d6d790aeab57d0c2c6db22acdf312204864aa6e0804d236bcfbf3193f216ac0aef8040000000000000008180052c2d7050000142f70726f68617368696e672e636f6d9b1d02002f

when encoded using UTF8 it contains the name "prohashing.com" which is a multi-coin mining pool.

block explorers usually mark these as "unknown" when the mining pool is small or when they don't have the name string in their hardcoded database.

[LoyceV]

We only need data since 02/26/2018 (Bitcoin Core segwit release date).

I only notice this now: SegWit was implemented on August 23, 2017. Why use Feb 26 (a couple months later)?

It looks like there was a dusting spam attack somewhere in 2015, with many output addresses.

The vertical axis shows tens of millions of Bitcoins. That means on one day more than 60 million Bitcoin was received. If that's correct, someone must have been bouncing large funds over many addresses.

And there was still  a block with 2992 legacy transactions (and zero segwit) in 2020!

I made a summary per day already. I can create one per block, but it takes some time so I'll only do it if someone's going to use it.