Bitcoin Forum
October 17, 2017, 03:42:41 PM *
News: Latest stable version of Bitcoin Core: 0.15.0.1  [Torrent]. (New!)
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2]  All
  Print  
Author Topic: (non-ultimate) blockchain compression?  (Read 1527 times)
maaku
Legendary
*
expert
Offline Offline

Activity: 905


View Profile
June 03, 2014, 03:48:11 AM
 #21

As I said, protocol change required.

I'm an independent developer working on bitcoin-core, making my living off community donations.
If you like my work, please consider donating yourself: 13snZ4ZyCzaL7358SmgvHGC9AxskqumNxP
1508254961
Hero Member
*
Offline Offline

Posts: 1508254961

View Profile Personal Message (Offline)

Ignore
1508254961
Reply with quote  #2

1508254961
Report to moderator
1508254961
Hero Member
*
Offline Offline

Posts: 1508254961

View Profile Personal Message (Offline)

Ignore
1508254961
Reply with quote  #2

1508254961
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1508254961
Hero Member
*
Offline Offline

Posts: 1508254961

View Profile Personal Message (Offline)

Ignore
1508254961
Reply with quote  #2

1508254961
Report to moderator
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
June 03, 2014, 04:35:45 AM
 #22

But the protocol layer can fix that. A block that is just header + coinbase + txid list would be pretty short.
Yes, but what if I do not have one or more transactions in my mempool to assemble a block from this template?
This situation occures when a transaction comes to a miner, miner accepts it into a block and solve block immideately after.
Or miner takes very old (month or year) transaction from mempool
So, no one node on network has this transaction in mempool.

Ok, first node receives "template", and have to ask for missing transaction its peer.

The point is that most of the time, most of the full nodes know about 90%+ of the txs.  The current protocol relays them tx they already know about.  It is a very simplistic and not optimized protocol.  In time it will almost certainly be changed to header + coinbase + tx hashes.   If a node doesn't know about a particular tx it will request that from its peers.  That is still far less bandwidth then all nodes relaying full tx list to peers most of which already know about most or all of them who then relay the full list to their peers most of which know about most or all of them.
apxu
Member
**
Offline Offline

Activity: 94


View Profile
June 03, 2014, 06:48:44 AM
 #23

Quote
The point is that most of the time, most of the full nodes know about 90%+ of the txs.

Yes. The problem is the rest 10%-.

Quote
If a node doesn't know about a particular tx it will request that from its peers.

OK, lets imagine. My node is connected to nodes N1, N2, N3... Nx
Once upon a time I receive "block template" from N1
There is a transaction in it which I do not know.
I can not ask this transaction from N1, because N1 already removed it from memory pool
And the other my peers also may not know about this tx or will reject/ignore my getdata packet.
So, I should drop "block template" and ask N1 for block itself.

This increases traffic (in some cases) and slows down block propagation.

This is not very rare scenario, I can prove it with my already working program (which checks for unknown transactions in incoming blocks)
picopicopico
Newbie
*
Offline Offline

Activity: 14


View Profile
June 03, 2014, 08:30:17 AM
 #24

~144 blocks

sorry, you're right of course. But still this is a considerable amount of time to wait.

Btw, is it my correct understanding that it is possible to build and send a valid transaction without having to wait for full wallet sync, provided that your tx inputs don't depend on the unknown part of the blockchain?
picopicopico
Newbie
*
Offline Offline

Activity: 14


View Profile
June 03, 2014, 09:22:56 AM
 #25

But the protocol layer can fix that. A block that is just header + coinbase + txid list would be pretty short.

Yes, this is how I thought this can be done. This looks like a straightforward self-contained feature that I could start working on. Is this a feature bitcoin core developers would be willing to accept into the main branch? Should I open a BIP?
picopicopico
Newbie
*
Offline Offline

Activity: 14


View Profile
June 03, 2014, 09:31:54 AM
 #26

Quote
The point is that most of the time, most of the full nodes know about 90%+ of the txs.

Yes. The problem is the rest 10%-.

Quote
If a node doesn't know about a particular tx it will request that from its peers.

OK, lets imagine. My node is connected to nodes N1, N2, N3... Nx
Once upon a time I receive "block template" from N1
There is a transaction in it which I do not know.
I can not ask this transaction from N1, because N1 already removed it from memory pool
And the other my peers also may not know about this tx or will reject/ignore my getdata packet.
So, I should drop "block template" and ask N1 for block itself.

This increases traffic (in some cases) and slows down block propagation.

This is not very rare scenario, I can prove it with my already working program (which checks for unknown transactions in incoming blocks)


could N1 keep this tx for some time after it's advertised in a block header? Also if the receiving node has already got block header it only needs the transactions, so in the worst case the overhead would be small (significantly less than the size of the header). Also we may consider sending shortened transaction hashes in the block header which would make the header even smaller and worst case less probable.
apxu
Member
**
Offline Offline

Activity: 94


View Profile
June 03, 2014, 10:13:22 AM
 #27

Quote
could N1 keep this tx for some time after it's advertised in a block header?

N1 has this tx in block and *can* provide it of course.
But the current rules allows me to ask only mempool(wild) transactions, not confirmed in blocks.
At least we have to rewrite some other code for supporting this behavior:

Quote
https://en.bitcoin.it/wiki/Protocol_specification#getdata
getdata is used in response to inv, to retrieve the content of a specific object, and is usually sent after receiving an inv packet, after filtering known elements. It can be used to retrieve transactions, but only if they are in the memory pool or relay set - arbitrary access to transactions in the chain is not allowed to avoid having clients start to depend on nodes having full transaction indexes (which modern nodes do not).
picopicopico
Newbie
*
Offline Offline

Activity: 14


View Profile
June 03, 2014, 11:07:41 AM
 #28

Quote
could N1 keep this tx for some time after it's advertised in a block header?

N1 has this tx in block and *can* provide it of course.
But the current rules allows me to ask only mempool(wild) transactions, not confirmed in blocks.
At least we have to rewrite some other code for supporting this behavior:

Quote
https://en.bitcoin.it/wiki/Protocol_specification#getdata
getdata is used in response to inv, to retrieve the content of a specific object, and is usually sent after receiving an inv packet, after filtering known elements. It can be used to retrieve transactions, but only if they are in the memory pool or relay set - arbitrary access to transactions in the chain is not allowed to avoid having clients start to depend on nodes having full transaction indexes (which modern nodes do not).


yes, the spec. needs to be changed. repeating my question from another reply - should I start a BIP if I want to implement this?
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
June 03, 2014, 02:09:20 PM
 #29

I can not ask this transaction from N1, because N1 already removed it from memory pool

As pointed out 5 times now ... this would require a change in protocol.

yes, the spec. needs to be changed. repeating my question from another reply - should I start a BIP if I want to implement this?

No.  From this thread it is very clear you lack the basic knowledge and logic skills necessary to implement a BIP.  I mean we just played a 14 post game of "who's on first" because you couldn't grasp the concept that in an changed protocol the protocol would be changed and thus referencing the current protocol would be pointless.

Of course that would be my opinion.  You don't need to ask permission to write up a BIP or make a pull request.
picopicopico
Newbie
*
Offline Offline

Activity: 14


View Profile
June 04, 2014, 06:59:40 AM
 #30

I can not ask this transaction from N1, because N1 already removed it from memory pool

As pointed out 5 times now ... this would require a change in protocol.

yes, the spec. needs to be changed. repeating my question from another reply - should I start a BIP if I want to implement this?

No.  From this thread it is very clear you lack the basic knowledge and logic skills necessary to implement a BIP.  I mean we just played a 14 post game of "who's on first" because you couldn't grasp the concept that in an changed protocol the protocol would be changed and thus referencing the current protocol would be pointless.

Of course that would be my opinion.

Assessment of a person's professional skills (positive or negative) is an offtopic for this thread. I guess you misread the replies, but this cannot be an excuse for throwing your negative emotions onto a technical discussion.

It is clear that implementation of proposed modifications will likely result in protocol modifications. What is not clear (to me) is:

1. are such changes welcome by the core team in general (provided that they meet coding guidelines, quality standards, etc) ?
2. Is there a process of BIP review/approval by the core team ?
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!