Don't all the big miners already use this private relay network:
 https://bitcointalk.org/index.php?topic=766190.0
?

that's even stronger evidence against this attack.

if the available unconf tx's are mostly invalid b/c of dust limits or being non-std, then those couldn't be included in such a bloated block.

Let me ask again.

Does anyone know the average number of unconfirmed TX's that exist in mempool at any given time, the average size in MB (i know <1) and what % of them get cleared into a block every 10 minutes or so?

I agree this has potential. It'd be a clean, permanent solution with nothing arbitrary about it. Sometimes we just don't see the simple things?

Too-large-blocks are already disincentivized by their slow propagation and resulting higher orphan risk. I read that many pools already use a "soft limit" well below 1 MB because of this. It would probably make most sense for a miner to calculate the cost of the orphan risk (needs some assumptions, I guess) and contrast that with the potential additional tx fees being earned and make tx inclusion decisions based on that calculation.

thanks for that.  but even if a miner tries to incl a non std tx into his solved block, it's quite probable that the majority of nodes/miners  still reject the block as a result, no?

How are transactions selected for blocks?
Transaction Processing
Current transaction selection policy:

Both standard and non-standard transactions, including arbitrary P2SH, are mined if they pay a reasonable fee.
Probable spam incurs a penalty on transaction priority.
Dust, confirmed spam, "bare" multisig, and custom "dangerous" scripts will not be mined under any conditions.

Are blockchains here to stay, in one guise or another? Just because bitcoin didn’t succeed as a currency
doesn't mean blockchain will succeed as a technology, but the experiment is important to run," says Patrick
Collison of Stripe, a payments processor. The possible uses are legion, but the killer app is still missing.

can one of you tech guys comment on Peter Todds claim that there was an entire book embedded in the BC a month ago?  how is it done?

https://www.reddit.com/r/Bitcoin/comments/34uu02/why_increasing_the_max_block_size_is_urgent_gavin/cqystoo?context=3

    // Extremely large transactions with lots of inputs can cost the network
    // almost as much to process as they cost the sender in fees, because
    // computing signature hashes is O(ninputs*txsize). Limiting transactions
    // to MAX_STANDARD_TX_SIZE mitigates CPU exhaustion attacks.
    unsigned int sz = tx.GetSerializeSize(SER_NETWORK, CTransaction::CURRENT_VERSION);
    if (sz >= MAX_STANDARD_TX_SIZE) {
        reason = "tx-size";
        return false;
    }

/** The maximum size for transactions we're willing to relay/mine */
static const unsigned int MAX_STANDARD_TX_SIZE = 100000;

Thanks for the correction!
And the last point answers something that was puzzling me.

Since then, Rusty's LN paper has appeared, and there is even less reason for delay in buying time, because there is a very real chance that a high-volume layer can be built over Bitcoin while keeping main-chain usage to a minimum. Why won't the devs buy a few years?

can one of you tech guys comment on Peter Todds claim that there was an entire book embedded in the BC a month ago?  how is it done?

https://www.reddit.com/r/Bitcoin/comments/34uu02/why_increasing_the_max_block_size_is_urgent_gavin/cqystoo?context=3

Is this a consequence of p2sh?

Gloves starting to come off already. Suppose it was inevitable.

Personally, I'm rather strongly against any commitment to a block size
increase in the near future. Long-term incentive compatibility requires
that there be some fee pressure, and that blocks be relatively
consistently full or very nearly full. What we see today are
transactions enjoying next-block confirmations with nearly zero pressure
to include any fee at all (though many do because it makes wallet code
simpler).

Justus laid out a structure.  There are a great number of missing pieces between what exists today in the consensus code and that structure.  There are also undefined and missing elements in the ecosystem.  I like the structure too, but we aren't anywhere close to it yet and it won't be soon.

The current proposal is Gavin's most reasonable one yet.  He dropped the assumptions about the future, took out most of the guesswork, and is implementing the method satoshi offered when the antispam 1mb limit was created in 0.3 or so.

More would be great, and future proof would be even better, no limit best....  but... remember, we are still in beta.  The pieces to do any of that are not written, tested, integrated, accepted and secured.

Bitcoin is still a baby.  So as eager as we all are to get there, we will get there by baby steps or end up not getting at all.  There are a great many looking for us to fail (check the short interest lately?) we help that failure by over-extending what can be done.

20mb gives us some room to build more of the missing pieces.  Its enough for now.

interesting logarithmic Monte Carlo simulation of tx confirmation delays related to progressive filling of current 1MB block size.  bottom line is we need an increase in the block size otherwise confirmation delays will exponentially increase as the fill reaches the max.  note in the conclusion who would benefit from NOT increasing the block size; "SC's".  no wonder gmax and LukeJr are spamming Reddit in defiance of Gavin's proposal:

http://hashingit.com/analysis/34-bitcoin-traffic-bulletin

With UTXO merkle tree the whole thing does not need to be in phone RAM.  Or even on the phone.  phone clients could validate and fwd txns iff they are connected to wifi.  The only parts of the UTXO merkle tree that needs to be processed is the logn route from each UTXO involved in a txn to the tree root.  So very doable on today's mid range smart phone esp with a good sized uSD expansion.

Use a special tree data structure to organize all unspent-TxOuts on the network, and use the root of this tree to communicate its "signature" between nodes.  The leaves of this tree actually correspond to addresses/scripts, and the data at the leaf is actually a root of the unspent-TxOut list for that address/script.  To maintain security of the tree signatures, it will be included in the header of an alternate blockchain, which will be secured by merged mining.  

This provides the same compression as the simpler unspent-TxOut merkle tree, but also gives nodes a way to download just the unspent-TxOut list for each address in their wallet, and verify that list directly against the blockheaders.  Therefore, even lightweight nodes can get full address information, from any untrusted peer, and with only a tiny amount of downloaded data (a few kB).  

A proposal to help SPV nodes verify blocks is to commit the root of an (unbalanced) Merkle tree containing all unspent (and spendable) transaction outputs.

It is possible to create proofs of validity for each modification of the set.  The proof would prove that inserting an entry into a tree with a root of X will give a new tree with a root of Y.  There would also be proofs for removing entries.

This link contains an RFC for a new type of Bitcoin address called a "payment code"

Payment codes are SPV-friendly alternatives to DarkWallet-style stealth addresses which provide useful features such as positively identifying senders to recipients and automatically providing for transaction refunds.

Payment codes can be publicly advertised and associated with a real-life identity without causing a loss of financial privacy.

Compared to stealth addresses, payment codes require less blockchain data storage.

Payment codes require 65 bytes of OP_RETURN data per sender-recipient pair, while stealth addresses require 40 bytes per transaction.

Justus's seminal work on incentivizing full nodes will take some time to be assimilated into the mainstream of the community, but I think it will have to be. Either through some high-powered explanatory tool or by the market forcing it on everyone when rubber meets road, which will be unsettling for many.

Speaking of high-powered explanatory tools, it would be great if there were a sort of "hard money wiki/FAQ," for lack of a better term, that explains the economic points often raised in this thread for a general (Bitcoin-conversant) audience. Not just why Bitcoin is replacing gold, though that's a core part of it, but all the points about incentives and economics that are are typically misunderstood in the wider community. Things like Money as Memory, the role of investors ("who controls Bitcoin?"), why Bitcoin is more valuable as a store of value than as a digital currency, how and why to introduce market economics to fee payment, node incentivization, etc. etc. Instead of a traditional wiki format, I'm a big fan of prezi.com's platform for visual explanations. In that format it could serve as a visual background for hundreds of easy-to-produce videos where someone is simply explaining a bite-sized point while using one small aspect of the larger Prezi wiki, effectively narrating it.

To the best of my knowledge, they are aware of it. None of them have commented, however.

I don't know of any immediate, specific plans to implement this.

Without such a visual communication medium and readily-producible videos, I think the explanatory load to actually educate the larger community about economics and such is just too much. The Nakamoto Institute is doing pretty well, but relying solely on essays is too slow in Bitcoin time for the permeation of understanding that it would be ideal to have.

It is true that we can't simply rely on charity for full nodes. There has to be an incentive structure.

Kinda like bittorrent ? This would just optimize the p2p layer of the protocol but need to
ensure its safe from attacks maybe thats why we didnt
do it to be on safe side.