Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: aplistir on September 17, 2017, 01:54:15 PM



Title: We need a light version of Bitcoin Core
Post by: aplistir on September 17, 2017, 01:54:15 PM
Running bitcoin core, requires:  

  • Downloading over 120 Gb of data
  • Verifying every transaction ever made
  • Waiting for a very long time (for it to finish)  

Doing all that just for wanting to run your own wallet in bitcoin core seems like overkill.
And it is only going to get worse as time goes on. After ~5 more years the blockchain will probably be >300Gb

I think it would be a GREAT idea to create a lighter version of Bitcoin core. "Bitcoin Core Mini"
It could be done, because for creating and verifying new transactions you do not need to have a full copy of the whole blockchain. All you really need is the info about all bitcoin addresses, that contain any bitcoins, and the inputs to those addresses.
The old transactions do not really matter for the purpose of creating and verifying new transactions.

For running "Bitcoin Core Mini" you would need
the balance and inputs of every address containing bitcoins, Eg. from today when blockchain was at block  #485732

For this info it would be enough to download a signed, trusted file of about 10Gb, and then verifying every transaction that has happened since the file was created eg. at block #485732
(New ~10Gb file could be made accessible every year or 6 months)
 
Estimated size of the file comes from:
10000000           Addresses with balance
          * 1 kb       for info of the address (inputs, outputs balance.)
                          Some addresses would need more than 1kb, some less
10000000 kb                          
            =
          10 Gb

The size of the file could be made a lot smaller, if we forget the "dust addresses", that contain only a few satochis, and are unlikely to ever be used. In those cases we could just have a list of those addresses without any additional info. (it would take minimum amount of space)  And if any of those addresses would ever be used, the "Bitcoin Core Mini" would then ask the needed info from the network, from a full node.

Anyone interested?
With "Bitcoin Core Mini" you would get almost all the advantages of a full Bitcoin Core. You could create and transmit your own transactions but you would not need to download and store over 120GB of data.
In the long run the size difference would only be getting bigger. In the future,  when the blockchain size will be  >500Gb, the Mini version would probably still remain under 20Gb  

Who would want to run "Bitcoin Core Mini" (if it is made)?
Would anyone want to be part of developing it?

I might be interested in developing it, if there is enough interest. Maintaining it in the long run is another thing.


Title: Re: We need a light version of Bitcoin Core
Post by: ranochigo on September 17, 2017, 02:54:10 PM
I think it would be a GREAT idea to create a lighter version of Bitcoin core. "Bitcoin Core Mini"
It could be done, because for creating and verifying new transactions you do not need to have a full copy of the whole blockchain. All you really need is the info about all bitcoin addresses, that contain any bitcoins, and the inputs to those addresses.
The old transactions do not really matter for the purpose of creating and verifying new transactions.
You need to download the entire Blockchain, if you can't download it, you wont be able to index and verify the transactions and form your own UXTO set.

The purpose of the whole process is to ensure that every transaction is valid and the client is able to build their own UXTO database. It doesn't make sense for a client that can't do these basic stuff to exist. If you just want to only create and send transaction, you just need the UXTO.
signed, trusted file of about 10Gb, and then verifying every transaction that has happened since the file was created eg. at block #485732
(New ~10Gb file could be made accessible every year or 6 months)
Great... There goes our trustless and decentralisation nature. If you're only going to send the UXTO set, your client will not be able to see the transactions that was made (at least till a certain date). I don't really see a point in making a client that doesn't show the old transactions. How am I going to track my spendings? Blockexplorers?
Quote from: aplistir link=topic=2185592.msg21930254#msg21930254 date=1505656455
[b
Estimated size of the file comes from:[/b]
10000000           Addresses with balance
          * 1 kb       for info of the address (inputs, outputs balance.)
                          Some addresses would need more than 1kb, some less
10000000 kb                          
            =
          10 Gb

The size of the file could be made a lot smaller, if we forget the "dust addresses", that contain only a few satochis, and are unlikely to ever be used. In those cases we could just have a list of those addresses without any additional info. (it would take minimum amount of space)  And if any of those addresses would ever be used, the "Bitcoin Core Mini" would then ask the needed info from the network, from a full node.
So in the future, if someone DOES actually spend the dust outputs, how does the client handle it? It doesn't recognize it from the UXTO set that it has. Its quite bad if you were to query another node when you receive a transaction with an output that you don't have stored locally. Additionally, how do you ensure that the full node that you make the request from doesn't lie to you? How are you going to verify if what they tell you is true?
Anyone interested?
With "Bitcoin Core Mini" you would get almost all the advantages of a full Bitcoin Core. You could create and transmit your own transactions but you would not need to download and store over 120GB of data.
The real advantage of Bitcoin Core is mostly the fact that its trustless. If you're going to destroy it, might as well as just use an SPV client. Sure, they may be less secure than Bitcoin Core in a sense that they are susceptible to sybil attack, the client that you are going to develop will essentially be worse.


Title: Re: We need a light version of Bitcoin Core
Post by: neurotypical on September 17, 2017, 03:59:04 PM
I asked for this a while ago, but unfortunately it seems it is not possible to use pruned mode before you download the entire blockchain once. The holy grail would be to be able to run prune mode and only download a reasonable amount of the blockchain like 10GB (because who the hell is going to have the resources to go back in time 10GB of transactions) then you could be able to use Bitcoin Core pruned in 10 GB and without downloading the entire thing but this seems just not possible, im not a coding wizard so I don't know the details but that is what I've heard. Maybe some genius some day manages to give us a prune mode without downloading the entire blockchain once and we can have a legit Bitcoin Core light but I doubt it.


Title: Re: We need a light version of Bitcoin Core
Post by: btctousd81 on September 17, 2017, 04:53:14 PM
this is the only reason, i was involed in bitcoin so late.

at first boot, it took so much time on my slow connection., i gave up, on running bitcoin-core and even minining bitcoins, back in 2012.

i can see for wallet purpose , it needs to scan the entire blockchain, to find uxto., but for non wallet functionality, i dont see why this cant be done.

i mean when running in prune mode, only last few blocks are stored in disk, same can be done . at the initial start, but for non wallet things.



Title: Re: We need a light version of Bitcoin Core
Post by: Ucy on September 17, 2017, 11:00:46 PM
Wondering if what you are looking for is Blockchain that can run on mobile phone?
Wanted opening a thread to ask whether Mobile Version of Blockchain exist but I felt that this section is too sophisticated for such dumb question Lol


Title: Re: We need a light version of Bitcoin Core
Post by: Argon2 on September 17, 2017, 11:36:05 PM
Running bitcoin core, requires:  

  • Downloading over 120 Gb of data
  • Verifying every transaction ever made
  • Waiting for a very long time (for it to finish)  
Correct and it will always remains this way because it must by design. Use SPV if you want a thin client.


Title: Re: We need a light version of Bitcoin Core
Post by: neurotypical on September 18, 2017, 11:49:06 AM
Running bitcoin core, requires:  

  • Downloading over 120 Gb of data
  • Verifying every transaction ever made
  • Waiting for a very long time (for it to finish)  
Correct and it will always remains this way because it must by design. Use SPV if you want a thin client.

Yeah but isn't prune mode better than SPV? Trusting a small part of the blockchain (like said, let's say 10GB for convenience's sake because 10GB is not much these days, and it's a lot of data already for an attacker that wanted to modify the blockchain, it's simply impossible to go 10GB back in time) is better than trusting someone's server by a long shot.

The main problem is... the initial sync, for some reason, requires the entire blockchain. For some reason, you can't just download the last 10GB and keep it at 10GB and discard the rest, you are forced to get the entire thing. That is why I said, the holy grail for nodes to be everywhere is to solve this.


Title: Re: We need a light version of Bitcoin Core
Post by: ranochigo on September 18, 2017, 01:02:38 PM
Yeah but isn't prune mode better than SPV? Trusting a small part of the blockchain (like said, let's say 10GB for convenience's sake because 10GB is not much these days, and it's a lot of data already for an attacker that wanted to modify the blockchain, it's simply impossible to go 10GB back in time) is better than trusting someone's server by a long shot.

The main problem is... the initial sync, for some reason, requires the entire blockchain. For some reason, you can't just download the last 10GB and keep it at 10GB and discard the rest, you are forced to get the entire thing. That is why I said, the holy grail for nodes to be everywhere is to solve this.
Bitcoin Core, however you configure it will definitely be superior to SPV clients. Your pruned node will not trust your peers which is pretty much what Bitcoin Core does too. The only reason why you need to keep at least ~550MB is because your client won't have to resynchronize if there is a major block re-org, nothing about security. It is definitely to attack the Blockchain and replace the older blocks if you have sufficient hashrate.

The initial synchronization is extremely important; its the main thing that lets Bitcoin Core be trustless. It will probably never be possible to run a full node that is completely trustless without at least downloading the blockchain (at least for now). Pruning is only expected to fix the problem with space in hard disks. In all honesty, SPV clients are relatively secure for an average joe.


Title: Re: We need a light version of Bitcoin Core
Post by: trickyriky on September 18, 2017, 01:40:33 PM
It is a pity but now this is not possible.
But you can use many others coins, that works immediately.


Title: Re: We need a light version of Bitcoin Core
Post by: alTPR on September 18, 2017, 01:51:24 PM
I think it would be a GREAT idea to create a lighter version of Bitcoin core. "Bitcoin Core Mini"
It could be done, because for creating and verifying new transactions you do not need to have a full copy of the whole blockchain. All you really need is the info about all bitcoin addresses, that contain any bitcoins, and the inputs to those addresses.
The old transactions do not really matter for the purpose of creating and verifying new transactions.
You need to download the entire Blockchain, if you can't download it, you wont be able to index and verify the transactions and form your own UXTO set.

The purpose of the whole process is to ensure that every transaction is valid and the client is able to build their own UXTO database. It doesn't make sense for a client that can't do these basic stuff to exist. If you just want to only create and send transaction, you just need the UXTO.
signed, trusted file of about 10Gb, and then verifying every transaction that has happened since the file was created eg. at block #485732
(New ~10Gb file could be made accessible every year or 6 months)
Great... There goes our trustless and decentralisation nature. If you're only going to send the UXTO set, your client will not be able to see the transactions that was made (at least till a certain date). I don't really see a point in making a client that doesn't show the old transactions. How am I going to track my spendings? Blockexplorers?
Quote from: aplistir link=topic=2185592.msg21930254#msg21930254 date=1505656455
[b
Estimated size of the file comes from:[/b]
10000000           Addresses with balance
          * 1 kb       for info of the address (inputs, outputs balance.)
                          Some addresses would need more than 1kb, some less
10000000 kb                          
            =
          10 Gb

The size of the file could be made a lot smaller, if we forget the "dust addresses", that contain only a few satochis, and are unlikely to ever be used. In those cases we could just have a list of those addresses without any additional info. (it would take minimum amount of space)  And if any of those addresses would ever be used, the "Bitcoin Core Mini" would then ask the needed info from the network, from a full node.
So in the future, if someone DOES actually spend the dust outputs, how does the client handle it? It doesn't recognize it from the UXTO set that it has. Its quite bad if you were to query another node when you receive a transaction with an output that you don't have stored locally. Additionally, how do you ensure that the full node that you make the request from doesn't lie to you? How are you going to verify if what they tell you is true?
Anyone interested?
With "Bitcoin Core Mini" you would get almost all the advantages of a full Bitcoin Core. You could create and transmit your own transactions but you would not need to download and store over 120GB of data.
The real advantage of Bitcoin Core is mostly the fact that its trustless. If you're going to destroy it, might as well as just use an SPV client. Sure, they may be less secure than Bitcoin Core in a sense that they are susceptible to sybil attack, the client that you are going to develop will essentially be worse.



Is there any specific reason why you are writing UXTO instead of UTXO?


Title: Re: We need a light version of Bitcoin Core
Post by: ranochigo on September 18, 2017, 01:57:20 PM
Is there any specific reason why you are writing UXTO instead of UTXO?
It really doesn't matter and I don't think it does as long as people understand what it is. It has become a habit for me to refer the unspent transaction output set as UXTO instead of UTXO, I guess you can see why after you read it in your mind for sometime. Either way, you get what I mean.


Title: Re: We need a light version of Bitcoin Core
Post by: kobradobra on September 18, 2017, 07:41:11 PM
And what so bad about Electrium or Multubit or any other l-client?. It's not like Lighter BC version would any different from already existing L-clients..


Title: Re: We need a light version of Bitcoin Core
Post by: nikisev on September 18, 2017, 09:01:08 PM
Yes. I would say its renders bitcoin core wallet useless for PC user.
Wallet have to be server constantly connected to internet.


Title: Re: We need a light version of Bitcoin Core
Post by: rouhaud on September 18, 2017, 09:22:29 PM
And what so bad about Electrium or Multubit or any other l-client?. It's not like Lighter BC version would any different from already existing L-clients..

They are centralised on server, that's what make it different, we need a bitcoin protoccole evolution to have an desktop wallet lighter with all the secuity that allows to download only few blocks and not the entire blockchain