New version out in ffreeze, please test away. There's a wealth of robustness changes and it should also be a bit faster. However no one at Armory has experienced the issues Carlton Banks and btcchris ran into, so most of these changes were speculative. It's possible these bugs will still be there partially or in full.

As usual, looking forward to your bug reports and logs, and again thank you for the time and effort you put in helping us to test things out =)

Unless I missed something huge, the proposal is not only to penalize large blocks, but to redistribute the penalties collected from these blocks back to other miners. In that sense there is a rollover of mining rewards (since penalized miners stand to earn their own penalties back), just not "fees rollover" per say.


I don't think they would collude, rather I expect a large miner can deplete the mempool of high fee transactions (and only these) while staying under the soft cap (thus paying no penalties), and leave other miners to pick up the slack. This behavior is incentivized by the fact that the other pools, if they do pick up the slack (and try to deplete the mempool with less regard for fees) stand to pay penalties for the extra service rendered, and the large miner in return stands to gain from that.

This results in one of 2 scenarii:

1) Other miners decide to pick the slack from the large miner, they naturally loose in profitability as the large miner is vampirizing their rewards by collecting penatlies. As a result, a lot of users from the other pools will point their hardware to the large miner, which will get an ever increasing share of the network hash rate, which enables his scheme even further and so on.

2) As a reaction to the large miner's behavior, every other miner adopts his policies, which is to at least avoid the penalties, by always emitting blocks below the softcap, and this change never fixed what it was meant to.


Other pools need to verify a block before they start to mine on top of it, so very large blocks, that propagate slower and take longer to validate could supposedly give a head start to the miner who emitted it, and be a disadvantage to smaller pools which stand at a risk of having their block orphaned the longer it takes for other miners to receive and validate their work.

I expect any centralized pool, however small they are, can afford the bandwidth and processing power to deal with much larger blocks than we have at the moment. This could hurt p2pool miners though.


Those examples do not stand. They hinge on the premise that there is endless demand for transactions paying 1mBTC fee. I understand the need to simplify these demonstrations but that defeats the underlying premise of this entire discussion. You example assumes there is no competition over fees, which is the premise of a "no block limit + hard fees" system. Your system sets both soft and hard caps to the block size, so there is no reason to believe people will sit at a 1mBTC fee when there is an endless demand for transactions

Model your demonstration with a fee structure using the Pareto principle i.e. 20% of the transactions pay for the 80% of the total fees available in the mempool (which is a lot closer to the current network's fee distribution than your examples), and the system falls apart. Anyone building blocks large enough to get penalized is just giving his rewards away to miners that prioritize big fee transactions and make a point of staying under the soft cap.

The issue with your proposal is not the penalty per say, it's the reward: there is a point where it is more profitable to let others get penalized. The existence of this point creates a threat that keeps all miners functioning below the soft cap. The threat is that they lose profitability in comparison to other pools, and those pools start siphoning away their hash rate as miners migrate.

If you were to take away the reward from the system a few things would be smoother:

1) No opportunities to game the system anymore. It all comes down to where the acceptable margin of fee vs penalty stands for the given mempool.
2) Very simple to implement.

The drawback is that since there is no reward, obviously the penalties are just destroyed. I'm not sure that's a drawback per say, for the following reasons:

1) It's trivial to blackhole bitcoins and it's been agreed that this is not damaging to the system. So this method isn't introducing some new DOS attacks on the system.
2) By destroying the penalty, the value of every other BTC just went up. As opposed to your system where you want to reward other miners from the penalties, this time everyone is getting rewarded, albeit in a much smaller magnitude. This means both miners, but everyone else holding coins is rewarded when a miner builds a block above the soft cap. Incidentally, that also means people running nodes (as long as they hold BTC, which is expectable).

This is perhaps the only proposition so far that has some sort of reward mechanism for node maintainers (granted it's tiny) which take equal part in the cost of node propagation and validation as miners.

It's not criticism directed towards anyone per say. In the course of my work with Armory, I get suggestions to implement this and that, but an idea that can be summarized in a single sentence can often demand 10k LoC. I'm much more inclined to look at a pull request than just some formulated concepts. As I said, having a PoC to support the idea has several advantages, one of which is to make the task of reviewers simpler, another which is to go through the most obvious optimizations right away. An idea without a PoC is not diminished, but an idea with a PoC is certainly improved. I felt like I should share that. It wasn't even an attempt to defend Gavin.

Obviously if you can find someone to work a PoC for your proposal, that would be fantastic.

You're an idea man, I'm a nuts and bolts guy, I can't help but look at this from my perspective. Your natural stance towards people with my skill set is "you don't sophisticate enough". My natural stance towards people with your skill set is "you complicate too much". This isn't about to change anytime soon, yet that doesn't make it a personal attack. Present a patient with some general syndrome to N different medical specialists, all in different fields, and they will come up with N different diagnosis. They're not all necessary wrong.

If you think there is some underlying ad hominem in my criticism of your proposal, that is not my intent. There are plenty of other sections in this forum which are ripe for this kind of rhetoric. I'm going to defend my point of view with every opportunities I get, I don't expect less from others. The intensity of the criticism may come across as unwarranted but that's only cause I'm genuinely interested in this discussion. That should vouch on its own for the importance I bear to theoretical research.

I'd be in his position I'd would too ask to see some code or at least some data analysis supporting the design. You can't just propose stuff and expect the people reviewing it to do all the leg work. An implementation at least proves your design is conceptually sound. It's easy to forget certain aspects when you theorycraft, and having to implement at least the PoC certainly motivates you to keep it as simple as possible.

I assume you are referring to the debate on "hard block size limit + organic fees" versus "no block size limit + hard fees", the third option (no block limit and organic fees) being a non solution. Obviously an "organic block size limit + organic fees" is the ideal solution, but I think the issue is non trivial, and I have no propositions to achieve it. I don't even know if its philosophically possible.

In this light, a "pseudo elastic block size limit + organic fees" is the better and most accessible solution at the moment, and I will argue that my proposal cannot be reduced to "no block size limit + hard fees", and that it actually falls under the same category as yours. Indeed, like your proposal, mine relies on an exponential function to establish the fee expended to block size ratio. Essentially the T-2T range remains, where any blocks below T needs no fees to be valid, and the total fee grows exponentially from T to 2T.

In this regard, my proposal uses the same soft-hard cap range mechanics as yours. As I said, ideally I'd prefer a fully scalable solution (without any artificial hard cap), but for now this kind of elastic soft-hard cap mechanic is better than what we got and simple enough to review and implement. The fact that my solution has caps implies there will be competition for fees as long as the seeding constants of the capping function are tuned correctly. On this front it behaves neither worse nor better than your idea.

Since I believe fees should be pegged on difficulty, fees wouldn't be hard coded either. Rather the baseline would progress inversely to network hashrate, while leaving room for competition over scarce block room.


I will again argue to the contrary. As a matter of fact, I believe your solution offers no room for such adaptive market behavior, while mine does. To take both your examples in order:


With T being the soft cap and 2T the hard cap, your solution proposes to penalize all miners creating blocks larger than T. This pegs blockchain space to fees, the same as my proposal: the more txs waiting in the mempool, the higher the fee you need to get included in the next block. Inversely, the less items in the mempool, the more likely you are to have your low/zero fee tx mined right away, which creates an incentive to emit transactions during low traffic periods.

While your approach supports emitting txs during low traffic by imposing extra costs on high traffic, mine simply doesn't do with the extra cost. That doesn't mean emitting transactions during low traffic is NOT cheaper. As a matter of fact, it is, but the difference between low and high traffic isn't as significant.

The true difference between my solution and yours is that while mine allows miners to exceed the T soft cap as long as there are enough fees to go by, yours penalizes all blocks outgrowing T, which effectively locks all blocks at size T.

Indeed a selfish miner would have no incentive to build blocks beyond T, and they will also benefit in not including 0/low fee transactions. Indeed, by leaving all 0/low fee transactions in the mempool and only creating small blocks, where the block size is defined min(total size of high fee txs, T), a large selfish miner can deplete the mempool from all high fee transactions, leaving the rest of the network to pick up the slack.

Other miners are left with the choice to fill blocks up to T, but not further. Due to the selfish miners action (who are pumping high fee, small size blocks), there are not enough fees to be redeemed from the mempool, and the penalties for including transaction past T would come out of the good willed miners' coinbase rewards. You may have "benevolent" miners who would rather empty the mempool than follow game theory, but they only stand to earn less money than everybody else. On the other hand, selfish miners still qualify to get a cut of the fee pool, to which they make a point not to contribute to, effectively siphoning revenue from good will and "benevolent" miners.

The true effect on the network is that no one will bother creating blocks larger than T, and we will still have a defacto hardcoded block size cap.

My proposal offers to allow miners to take in extra fees as long as they remain below the curve defined by the capping function. Selfish miners do not have an opportunity to vampirize good willed miners anymore so while the entire behavior (high fee, low size blocks) is not deterred, it is at least not encouraged.

Please keep in mind that this analysis of your system relies on my current understanding of it. I'm still not 100% clear how the "fee pool" functions. I'm assuming it is either only funded in penalties, and all fees are paid to miners directly, or that all fees are pooled and distributed equally per blocks. The later assumption seems pointless since it can be easily bypassed, so I'm using the former one as the basis to my critics of your proposal.


Again I will argue that my proposal supports hardware improvement while yours doesn't. In your case, T will act as a defacto hard cap on block size limit, so there is no incentive for miners to be able to handle more traffic and create blocks beyond that value. As long as miners won't output blocks larger than T, there is no reason for the rest of the network to upgrade either.

With my solution, as long as there are enough fees to go by, up until 2T blocksize (or whatever the hard cap ends up being), miners are motivated to include transactions paying fees beyond the soft cap, which justifies hardware improvement to handle the extra load, with the consequences this has on the rest of the network.


Both in the current state and the solution you propose, malevolent miners can disturb the network by mining either empty blocks, or blocks full of "useless transactions", sending their own coins back to themselves. In my solution malevolent miners also have the added opportunity to mine large blocks by paying fees to themselves. Let's analyze what counter there is to these 3 disturbance attacks.

1) In case of empty blocks, all good willed and selfish miners should simply ignore them. It increases their revenue and impeaches the attack.

2) In case of "useless transactions", either the transactions were never made public, in which case the blocks are easy to identify (full of txs that never hit the mempool) and can be ignored for the same reason as above, or the transactions have been published publicly, and the attacker is making a point of mining only these. At this point you can't really distinguish this miner from good willed ones with either solution.

3) With my solution, malevolent miners can pay themselves fees and enlarge blocks. However that only holds true if they are keeping the transactions private. In this case other miners can identify such blocks as malevolent and ignore them entirely (2T wide blocks full of large fee txs that never hit the mempool). If the attacking miner makes the transactions public, he can't expect to rake all fees, so the solution to this attack is no different from 1 & 2

4) With your solution, what is to stop a malevolent miner from maxing blocks with 0 fee transactions? Sure he would give up the coinbase reward in form of penalties, but now these are available for the taking to other miners. Good willed miners may ignore such blocks, but selfish miners probably won't.

With my solution, there is an incentive for both good willed and selfish miners to ignore blocks outright constructed to bloat the network. With yours, there is a built in cost to such disturbance, so selfish miners can choose to ignore the disturbance for the benefit of the reward. In my system, the only viable economic option is to ignore the blocks.


2 years ago I would have opposed pegging fees and block size to difficulty, because ASIC technology was catching up to current manufacturing processes and as such was growing much faster than every other hardware supporting the network. That would require too much manual adjustments of the pegging function to be acceptable. As time passes by, that criticism loses ground, and now is not a bad time to consider it.

I would be interested to see if you have room to factor difficulty in your current function.


I try to not be so quick with drawing such conclusions. I'm not savvy with the Core codebase, but my experience with blockchain analysis has taught me that the less complicated a design is, the more room for optimization it has. You can't argue that adding a verification mechanic will simplify code or reduce verification cost, although the magnitude of the impact is obviously relevant. I'm not in a position to evaluate that, but I would rather remain cautious.

The point still remains, you don't need a fee pool to establish a relationship between fee, block size, and possibly difficulty.

Don't get me wrong, I believe the idea has merits. What I don't believe is that these merits apply directly to the issue at hand. It can fix other issues, but other issues aren't threatening to split the network. I also don't think this idea is mature enough.

As Gavin says, without an implementation and some tests it is hard to see how the system will perform. If we are going to “theorycraft”, I will attempt to keep it as lean as possible.


It seems your implementation pays the fee pool in full to the next block. That defeats the pool purpose in part. The implementation becomes more complicated when you have to gradually distribute pool rewards to “good” miners, while you keep raking penalties from larger blocks.

Otherwise, a large block paying high penalties could be followed right away by another large block, which will offset its penalties with the fee pool reward. The idea here is to penalize miners going over and reward those staying under the soft cap. If you let the miners going over the cap get a cut of the rewards, they can offset their penalties and never care for the whole system.

As a result you need a rolling fee pool, not just a 1 block lifetime pool, and that complicates the implementation, because you need to keep track of the pool size across a range of blocks.

I personally think a P2P network cannot rely on magic numbers, in this case 1MB nor 20MB blocks. The bar is either set too low and it creates an artificial choke point, or it is set too high which opens room for both DOS attacks and more centralization. As such, a solution that pins block size to fees is, in my point of view, the most sensible alternative to explore. Fees are the defacto index to determine transaction size and priority, so they are also the best candidate to determine valid block size.

However I have a couple divergence with Meni's proposal:

First, while I find the fee pool idea intriguing, and I certainly see benefits to it (like countering "selfish" mining), I don't believe it is a necessary device to pin block size limits to fees. Simply put, I think it's a large implementation effort, or if anything a much larger one than is needed to achieve the task at hand. It also requires modifying, or at least amending consensus rules, something the majority of the Core team has been trying to keep to a minimum. I believe there is wisdom in that position.

Second, I expect a fee pool alone will increase block verification cost. If it is tied to the block size validity as well, it would increase that cost even further. The people opposing block size growth base it on the rationale that an increase in resource to propagate and verify blocks effectively raises the barrier to entry of the network, resulting in more centralization. This point has merits and thus I think any solution to the issue needs to keep the impact on validation cost as low as possible.

Meni's growth function still depends on predetermined constants (i.e. magic numbers), but it is largely preferable to static limits. Meni wants to use it to define revenue penalties to miners for blocks larger than T.

I would scrap the fee pool and use the function the opposite way: the total sum of fees paid in the block defines the maximum block size. The seeding constant for this function could itself be inversely tied to the block difficulty target, which is an acceptable measure of coin value: i.e. the stronger the network, the higher the BTC value, and reciprocally the lower the nominal fee to block size balance point.

With an exponential function in the fashion of Meni's own, we can keep a healthy cap on the cost of larger blocks, which impedes spam by design, while allowing miners to include transactions with larger fees without outright kicking lower fee transactions out of their blocks.

As Meni's proposal, this isn't perfect, but I believe it comes at the advantage of lower implementation cost and disturbance to the current model, while keeping the mechanics behind block size elasticity straight forward. Whichever the community favors, I would personally support a solution that ties block size limit to fees over any of the current proposals.

Armory is an advanced wallet, not meant for 6yo or users otherwise uneducated in the basic functions of the Core implementation, therefor I cannot provide an ELI5 level response.

Now, consider the following: Armory only runs on top of a binary implementing the bitcoin consensus and networking layer. It ultimately trusts that binary which means it doesn't check blocks against its own set of consensus rules.

Considering the difference between Core and XT (i.e. minimal), Armory won't know any better whether it is ran against Core or XT. Now if you understand how Core stores its block data on disk and know that Armory reads the block data from disk, you will know not to run XT and Core on the same home folder if you want to guarantee Armory will see only the fork of your choosing.

Depends on the binary you make available to Armory.

I guess that's the proper name for the BIP specified HD wallets. Confusing me >_<"

As long as your Core implementation only records the relevant fork on disk, Armory will only see that.

Armory trusts the local Core instance for block validity so it doesn't check that transactions are valid (according to this or that consensus rules). If somehow your Bitcoin Node maintains both forks on disk (why would it if the divergence in consensus caused a fork?), then Armory will only care for the longest chain.

The new wallet format supporting BIP32 will be backwards compatible with every previous Armory wallet format.

0.93.x does not work on x86 OS. Wait for 0.94

Are you using a 32bit OS? How much free disk space do you have?

force your block file location with --satoshi-datadir

first check box in File -> Settings

Turn off auto bitcoind management and Armory will ignore bitcoin.conf entirely

Both are obvious symptoms. If Armory is aggressively asking for memory it will take priority over processes in the background or that aren't requesting as much I/O. That's basic OS optimization.

As for memory requirements, Armory expects to have about 2~4GB of RAM at its disposal for it's own scanning needs. The rest is mmap'ed files requested by LMDB. That's controlled by a hardcoded value and will probably be dynamically assigned in the upcoming version.

That's a good idea, I'll do that.

Buffering, processing and writing data are all different tasks that take place in parallel. A batch of blocks is buffered, processed and written, then the batch is cleaned up. Only after a batch has been written will the DB resume past that point. The buffering threads have a bottleneck (only 3 buffers in the waiting queue at most). The processing threads have no bottleneck (and usually out pace the writing threads in supernode) which is why the scanning is getting really slow (RAM gets filled with too much data to allow the writers to run properly). As for why this happens, it's cause I forgot to turn the processing threads bottleneck back on when I was doing some tests o.o"!

As for progress, I'll improve the log message with the upcoming fixes.

The wording is off. It signals the highest buffered block, not the highest processed and written block (cause I was lazy and didn't change this kinda stuff)