CryptoNote coins (BCN, Aeon, XMR, Monero...) and blockchain size - Trouble?

[robolove]

I, along with many others, am fascinated with CryptoNote coins, such as Aeon, but see an obvious problem with no apparent solution: the astronomical size of the blockchains.

Bitcoin has existed since 2009, and its blockchain is around 20 GB, which is manageable.

Aeon has been out for 38 days, and its blockchain is nearly 1 GB. Ouch.

This is clearly not sustainable. Am I missing something fundamental, or are CryptoNote coins doomed because of this design flaw?

[Luwonder]

The size of the Cryptnote wallet also grows very fast. It is easy to reach 20MB. Is there a way to reduce the wallet size?

[DaveyJones]

The size of the Cryptnote wallet also grows very fast. It is easy to reach 20MB. Is there a way to reduce the wallet size?

How is 20MB nowadays a problem? 

But to answer your question the wallet file is just your scanned blockchain file, what matters is the .keys file for your coins... so you could possibly delete that if its to big for your and rescan everytime you want to use it

[Snail2]

The size of the Cryptnote wallet also grows very fast. It is easy to reach 20MB. Is there a way to reduce the wallet size?

How is 20MB nowadays a problem? 

But to answer your question the wallet file is just your scanned blockchain file, what matters is the .keys file for your coins... so you could possibly delete that if its to big for your and rescan everytime you want to use it

I guess he mistyped something. Probably that 20MB actually means 20GB.

[Andretti83]

The size of the Cryptnote wallet also grows very fast. It is easy to reach 20MB. Is there a way to reduce the wallet size?

How is 20MB nowadays a problem? 

But to answer your question the wallet file is just your scanned blockchain file, what matters is the .keys file for your coins... so you could possibly delete that if its to big for your and rescan everytime you want to use it

I guess he mistyped something. Probably that 20MB actually means 20GB.

No, he means something like cache for wallet file. It has the same name as wallet and can reach 20 MB.

The real wallet file (*.keys) is less than 1 KB in size (mine is 544 bytes).

[cryptodromeda]

FFS... July 2014??

I think we need to devise a Bladerunner-style replicant test to determine whether someone is a concern troll.

[Qastessa]

Will the change of the block time from 1 min to 2 min or longer reduce the bloating rate of block chain size?

[Snail2]

The size of the Cryptnote wallet also grows very fast. It is easy to reach 20MB. Is there a way to reduce the wallet size?

How is 20MB nowadays a problem? 

But to answer your question the wallet file is just your scanned blockchain file, what matters is the .keys file for your coins... so you could possibly delete that if its to big for your and rescan everytime you want to use it

I guess he mistyped something. Probably that 20MB actually means 20GB.

No, he means something like cache for wallet file. It has the same name as wallet and can reach 20 MB.

The real wallet file (*.keys) is less than 1 KB in size (mine is 544 bytes).

I see. Probably you are right. A couple of dozens of megabytes shouldn't be a problem (even on a mobile device) in these days, indeed.

[aeoncurrency]

I, along with many others, am fascinated with CryptoNote coins, such as Aeon, but see an obvious problem with no apparent solution: the astronomical size of the blockchains.

Bitcoin has existed since 2009, and its blockchain is around 20 GB, which is manageable.

Aeon has been out for 38 days, and its blockchain is nearly 1 GB. Ouch.

This is clearly not sustainable. Am I missing something fundamental, or are CryptoNote coins doomed because of this design flaw?

This is potentially a long term concern but there are solutions available.

Aeon introduced a pruning test release in August https://bitcointalk.org/index.php?topic=641696.msg12272943#msg12272943

Boolberry has had pruning since last year http://boolberry.org/files/Boolberry_Reduces_Blockchain_Bloat.pdf

There are some technical differences in the way Aeon and Boolberry handle pruning. Monero has plenty time to decide when and if it wants to introduce pruning and if so how it should be done.

At this point there are more pressing development priorities than the size of the blockchain.

[aeoncurrency]

Will the change of the block time from 1 min to 2 min or longer reduce the bloating rate of block chain size?

Yes and (longer than one minute) block times have already been implemented in several CryptoNote coins including Aeon and Boolberry.

Monero will have two minute block times with its next release (0.9 which is now in beta).

[Rias]

Your figures are wrong

1) Does AEON really exist for 38 days? The blockchain shouldn't be even close to 1GB then. AEON was launched 1.5 years ago, AFAIK. Did they relaunch the blockchain?

2) Bitcoin blockchain is twice more than 20GBs as of now:
https://blockchain.info/charts/blocks-size

3) Also, CryptoNote wallet file is indeed 20-something megabytes, not more (for ordinary wallets).

Bloating problem is likely irrelevant

I honestly never saw a problem with the blockchain size for CryptoNotes. Each tx is 2-3 times larger than in Bitcoin (as discussed by CryptoNote devs). It is a linear function, so there's absolutely nothing wrong with CryptoNote blockchain.

It is slightly bigger, but still not that much. Bytecon, which has the longest and possibly the largest blockchain from all CryptoNotes is currently 6GB when fully indexed.

Eventually, there will be full and light nodes just like in Bitcoin. Infrastructure will be sorted out by the evolution.

So you are either ignorant or fuding.

[rustynailer]

Bitcoin compatibility is probably the biggest issue with adoption on all these coins.

[tolikkk]

agree that the size of the blockchain and its complexity (having already 50GB and due to a question about increasing blocks) very uncomfortable on household computer and the need for more smaller blockchain with such functionality, and tx processing, I think the impressiveness is not always the best, though

[Rias]

Bitcoin compatibility is probably the biggest issue with adoption on all these coins.

This is likely to be the case. CryptoNote was designed to be different from Bitcoin; even its whitepaper is written as opposed to Bitcoin architecture. Therefore, the API is incompatible. This would not be a drawback in case of higher demand/hype, but in reality this may hinder CryptoNote much more than blockchain bloating.

[languagehasmeaning]

Bitcoin compatibility is probably the biggest issue with adoption on all these coins.

This is likely to be the case. CryptoNote was designed to be different from Bitcoin; even its whitepaper is written as opposed to Bitcoin architecture. Therefore, the API is incompatible. This would not be a drawback in case of higher demand/hype, but in reality this may hinder CryptoNote much more than blockchain bloating.

Monero is doing API work and other CryptoNote coins will as well:

https://bitcointalk.org/index.php?topic=949393.0

Monero API .NET
API for data transmission and retrieval from Monero applications through the Microsoft .NET Framework

GitHub repository | NuGet package

This project allows developers to flawalessly access the functionailty of Monero applications through the Core assemblies' RPC services. As the software is open-source, everyone can contribute, achieving the best performance and reliability for end-users.

Why is this API necessary?
It makes the implementation of Monero extremely easy for merchants, wallet builders, and all the other developers who are interested in working with (or accepting) Monero. Also, the core developers can concentrate more on the internals (instead of spending time on the official GUI client), which means that the currency should advance faster.

Is it cross-platform?
The first 3 releases of the API were only made for desktop PCs running Microsoft .NET Framework 4.5 or higher, thus, resulting in a smaller userbase than wanted. Since v3, the project has been split into 2 subprojects, one which is responsible for communicating with the daemon and the account manager's RPC channel, and one which can manage the required processes on PC.

What kind of platforms can utilize the power of Monero API .NET, then?
Basically every device which has support for running the Microsoft .NET Portable Framework v4, and more: Microsoft is planning to provide native (C++) compilation of C# assemblies very soon, and this process can be even faster since .NET Core has been made open-source.
The project can run on PCs (Windows; Mac and Linux with Mono .NET), mobile phones (Windows Phone; Android and iOS with Xamarin), and even on video game consoles like Xbox.

How does it work?
As I have already mentioned above, the project consists of 2 parts: MoneroAPI, and MoneroAPI.Extensions. The first one is built as a PCL (portable class library) and is multiplatform, while the extensions are desktop-only and contain functions for hosting a full node. Basically, MoneroAPI is preferable for lightweight, client-only applications, and MoneroAPI.Extensions adds support for hosting the Monero Core processes with ease.
There is a demo project included in the GitHub repository for a more technical explanation.

Please consider donating to one of the addresses in my signature if you like this project or want to support its development.

However this thread is about blockchain size not APIs.

Both Boolberry and Aeon already offer pruning. Boolberry has done so for over a year already

3. Blockchain bloat solution (already implemented):
http://boolberry.org/files/Boolberry_Reduces_Blockchain_Bloat.pdf

[languagehasmeaning]

Both Boolberry and Aeon already offer pruning. Boolberry has done so for over a year already

3. Blockchain bloat solution (already implemented):
http://boolberry.org/files/Boolberry_Reduces_Blockchain_Bloat.pdf

Pruning FAQ

Q: What is pruning?

A: Pruning refers to removing unnecessary information from the blockchain once it is no longer needed.

Q: What are the advantages?

A: Pruning reduces the amount of storage needed for the blockchain. If the blockchain is loaded into RAM (which is the case in the current implementation), then it also reduces RAM usage. Finally, it also reduces the rate that these resources are consumed over time as the blockchain grows. In the current implementation, each time one new block is accepted, one old block is pruned, on average freeing up a large portion of the resources needed for the new block.

Q: How does pruning affect functionality?

A: The only inherent functional limitation of AEON's pruning is the inability to restore (also known as rescan) a wallet which was used for spending transactions. All other functionality including sending and receiving coins, mining, updating a wallet after any period of inactivity, cold storage, mining, etc. remain fully supported. There is a current limitation that after extended period of disconnection (>28 days) a node may have trouble resynchronizing with the network, and would need to be reset. This is not inherent and should be addressed later.

Q: Does pruning reduce the effectiveness of ring signatures for transaction privacy?

A: No, ring signatures and privacy are unaffected.

Q: How is pruning enabled and disabled?

A: Using the --pruning option on the daemon command line. With this option the daemon will prune the blockchain and will also switch from using the blockchain.bin file for storage to blockchain-pruned.bin. To switch an existing node into pruned mode, copy blockchain.bin to blockchain-pruned.bin before starting the daemon with --pruning. To switch pruning off, remove the --pruning option. Do not, however, copy blockchain-pruned.bin to blockchain.bin. This will not work. You will need to have an unpruned blockchain.bin file or resync unpruned from the network.

Q: How does AEON's pruning compare with Boolberry's pruning?

A: AEON prunes slightly more information from the blockchain, so the required storage is slightly smaller (given equivalent usage), though the difference is likely not particularly significant. BBR prunes the blockchain on the entire network while AEON prunes on each node individually (and only if pruning is enabled). This means that new nodes can come online faster with BBR, but those new nodes are unable to independently verify the entire blockchain. It is possible to download an unpruned BBR blockchain from a web site to independently verify it, but in that case the amount of data downloaded would be the same as AEON. It also means that every BBR node is able to serve the chain to new users but in AEON this function falls to nodes that are unpruned, also known as "archive nodes" (or alternately via a trusted bootstrap file).

Q: How does AEON's pruning compare with Bitcoin's pruning?

A: In Bitcoin 0.11, the same model of pruning is implemented as in AEON. That is, nodes prune blocks locally, after syncing from an unpruned chain and verifying the chain independently. Like AEON, Bitcoin pruned nodes can't rescan or reindex wallets. Bitcoin 0.11 does not support wallets on pruned nodes at all, so it currently has more limitations that AEON. Old Bitcoin blocks must be retrieved from unpruned archive nodes, as with AEON.

Q: What other coins implement pruning?

A: Other than Boolberry, Bitcoin, and possibly some coins which have inherited their implementation by being Bitcoin forks, no other coins implement pruning at this time.

Q: What's this about "archive nodes"? How can I run one?

A: When nodes connect to the network, they retrieve blocks from other nodes. If only blocks within the most recent 10 000 (approx.  28 days) are needed for syncing, even pruned nodes can provide them. However, in the case of nodes which are brand new (syncing from the genesis block) or which have been offline for >28 days, pruned nodes will be unable to supply the older blocks. Instead this task falls to unpruned nodes, also known as archive nodes. For the time being the project-run seed nodes will always run in unpruned mode, and others with sufficient RAM and storage space who wish to support the network are also encouraged to do so. To run an archive node, simply start the daemon in the normal manner, without the --pruning option.

Q: What are the numbers? How much does pruning reduce the amount of memory and storage needed?

A: The exact numbers will vary according to OS, compiler, etc. and also depend on the usage of the blockchain in the future. One early report from BoscoMurray stated, "RAM usage down from 4.8GB to 2.4GB and blockchain file size down from 3.2GB to 1.7GB"

Q: That doesn't seem like a big reduction. Why is the benefit not greater?

A: To explain why the reduction is not larger and understand what this means for the future, let's first review some basics of how a blockchain works.

Every time a coin is spent, a digital signature accompanies the transaction in order to show that the owner of that coin authorized spending it. Once this signature is checked, it is no longer needed. This is the largest portion of what is being pruned. In the cryptonote signature scheme (used by AEON), ring signatures used for anonymity, which means while a signature shows the coin owner authorized the transaction, unlike conventional signatures, it does not identify the specific coins being spent, and therefore does not allow tracing and analyzing the blockchain. As a side effect of this functionality, these signatures are much larger than ordinary digital signatures (a fact sometimes described as "cryptonote bloat"). Thus in AEON, pruning offers great benefit and eliminates the "cryptonote bloat".

So why is the savings not larger? Because in the early history of AEON, there was a very large number of very large transactions and many of those transactions did not using ring signatures. This happened for several reasons, including a major flaw in the early versions of cryptonote mining pool software. Thus, while the chain is relatively large, a relatively small amount of storage is saved by pruning the early transactions.

The newer transactions are a different story. The pool software flaw was fixed long ago, and most transactions now do use ring signatures. So the savings from pruning going forward should be much higher (perhaps 75-80%). Of course, since ring signatures are now being routinely used, it means that that actual anonymity of the coin in practice is greatly improved, and with pruning there is no long term storage cost (i.e. no "cryptonote bloat") for most nodes (other than archive nodes). We get the best of both worlds!

Q: What about a database or disk-based block storage? Doesn't that also reduce memory usage?

A: Storing the blockchain out-of-memory in a database or in cache files reduces memory usage but does not reduce storage usage nor reduce the rate of growth of storage usage over time. In AEON, the plan is to later support out-of-memory storage of the blockchain along with pruning, so memory usage will be further reduced, and storage usage will remain low and grow very slowly over time.

[smooth]

Your figures are wrong

1) Does AEON really exist for 38 days? The blockchain shouldn't be even close to 1GB then. AEON was launched 1.5 years ago, AFAIK. Did they relaunch the blockchain?

There was no relaunch.

The figures probably were right at the time (you may not have noticed this thread is very old), but that shows the irrelevance of the issue.

The original open source pool code was extraordinarily inefficient in making tiny payouts to every single miner (could be hundreds or even thousands) for every single block. That caused enormous bloating of the chain. Once that got fixed a few months after AEON launch the chain bloat largely disappeared. So while there might have been 1 GB in the first month, after 1.5 years it is only 3.4 GB (smaller with pruning).

As you say, the size is very manageable. I recently downloaded Fallout 4 at >20 GB for one video game.

[john-connor]

Boolberry only has the ability to prune ring signatures and aged alternative chains. You can verify this via source code here: https://github.com/cryptozoidberg/boolberry/blob/master/src/currency_core/blockchain_storage.cpp

So pruning alone is not going to help what I consider a faulty blockchain design. I've also looked into the database code that is supposed to "one day" take the blockchain mostly out of RAM but it has serious problems of it's own.

Here is a quote in their own words "This provides over a 55% reduction in block chain size." From "basic math" calculations anything less than ~92.5% reduction is both a short and long term scalability issue.

My conclusion is that no cryptonote coin will scale (much further) without serious changes to a majority portion of it's codebase. We will see who is right and who is wrong within the next year so there need be no debate this.

[Luwonder]

Bitcoin compatibility is probably the biggest issue with adoption on all these coins.

Why should cryptonote coins be compatible with bitcoin? How can it be compatible? There will exchanges for the trading of these coins.

[rustynailer]

Bitcoin compatibility is probably the biggest issue with adoption on all these coins.

Why should cryptonote coins be compatible with bitcoin? How can it be compatible? There will exchanges for the trading of these coins.

I dont think mom and pop websites and business will go the extra mile after first learning about and implementing bitcoin and then have to go to the extra trouble of adapting to other protocols.  And yes, there will be exchanges that will set aside the resources to take this on.