Huge increase in satoshidice spam over the past day

[caveden]

Everyone deprioritizes every address that is heavily used, thats the point.  

Please, really please don't make that a rule embedded in bitcoind.

Chain download is limited by disk speed, but mostly CPU speed at checking signatures.  

Really? CPU time checking signatures is slower than the database indexation? I always thought it was the other way around. At least my laptop seems to access the disk like crazy when it's getting up to date... I'll check CPU usage.

By the way, I heard you were working on a patch to make the chain update more asynchronous, and thus faster, and that you were obtaining good results. Is that so?

[1714153032]

1714153032

[1714153032]

1714153032

[1714153032]

1714153032

[Matt Corallo]

You're being too harsh on SatoshiDice.
AFAIK, they're paying fees for every transaction they send. In the end, they're contributing to the network security by financing miners. The fact that non-miners are having problems to follow the chain progress is less relevant IMHO, as that will necessarily be the situation in the future if bitcoin "succeeds".

The tiny fees they pay are very, very, very far from paying for the damage they are doing to people's disks/cpus/bandwidth.  Miners (currently) have plenty of finance with 50BTC/block, but even if we switched to 0BTC/block tomorrow, the amount that satoshidice is paying compared to the load that they put on mining bitcoinds is very close to not worth it to the point its almost easier to just drop them and not have to worry about spending too much time generating blocks.

I'd argue they're doing more good than damage with all these paying transactions. They're actually being generous, because if they were to use send-many and reduce their number of transactions, they'd be paying less fees to miners.

Being generous to miners does not help end-users.  My point here is the incentive structure is really a deal between transactions senders and miners, without thinking about end-users.  Hence why end-user nodes could deprioritize forwarding high-volume address transactions to tweak the incentive structure.  

(and about having a balance, I think they've been so successful precisely because they don't require you to have an account).

Agreed, but sadly, in the end, its hugely to the detriment of Bitcoin as a whole.  Hence why Im suggesting we adjust the incentives to discourage bad players like SatoshiDice.  That said, using multisend would not require any user-facing changes while lowering the load on bitcoin clients significantly.  

And if miners/pools are still accepting free transactions, it just means they don't give a damn to the overhead. (at least not yet).

It means they are interested in supporting Bitcoin's growth.  In fact, because of SatoshiDice's volume, free transactions are being force out of blocks, when it is important for users to be able to use Bitcoin.  The idea is to deprioritize high-volume address transactions so that regular users can get their transactions into blocks in reasonable time, while sites that dont need fast confirmations (like satoshidice) can have their transactions fairly delayed.

[Realpra]

The size of the blockchain will soon be a huge hindrance to widespread adoption

Exactly! I am glad someone else gets it.

Right now the BTC network is very small, but the VISION that we have all invested in is HUGE -> "BTC replaces dollar/VISA/Mastercard".

To get there you can't have the network breaking down over something like satoshidice NOR miners starting to ask ridiculous fees.


If you just want BTC to be another small time linden dollar with no future wtf are we all doing here?


Yes a bunch of super computers COULD run the BTC network at VISA size with normal users being light clients, BUT that would leave us wide open to attacks, being shut down or miner-dictatorship = VISA.

Pruning needs to get done or yes bitcoin might not die per say; it will just become pointless.

[nimda]

"Normal" users should not be using bitcon-qt.

Well, it's the official client for one.
For two, the more people that stop using a "full" client, the fewer full client nodes we have, and the less secure the network is.

[Matt Corallo]

Please, really please don't make that a rule embedded in bitcoind.

Why not? the point is its to the benifit of the people running bitcoind, so its in their best interest to do such rate-limiting.  In the end, its really less of a deprioritization and more of a "stop-DDoSing me, Im gonna rate limit your transactions"-feature.  The goal is to tweak incentives so that the end-users and those running forwarding nodes can have a say in the incentive structure, as is ultimately fair, as they are putting in way more effort than miners in terms of verifying transactions.

Really? CPU time checking signatures is slower than the database indexation? I always thought it was the other way around. At least my laptop seems to access the disk like crazy when it's getting up to date... I'll check CPU usage.

Its a bit of both, and really depends on the disk.  On an SSD/tmpfs its more CPU, on a spinning disk, its more disk.  Note that multisend helps with the database lookups a /TON/.

By the way, I heard you were working on a patch to make the chain update more asynchronous, and thus faster, and that you were obtaining good results. Is that so?

https://github.com/bitcoin/bitcoin/pull/1233 Depends on where in the chain you are, but it does help some.

[etotheipi]

This should be a non-issue in the long term:  If the network can't handle 50k tx per day, Bitcoin was never meant to succeed.  The only reason it's such a big issue right now is because it's a single entity producing the volume -- and thus there's somewhere to point the finger when it was really our lack of preparation to handle it.  It could've just as easily been a piece of some other market that latched onto BTC and produced 50k tx per day in a less-centralized manner.  

It's a downside of having a completely open, decentralized, apolitical, global currency scheme -- people must have the ability to use it as they please, and the rest of the system must find equilibrium.  Talk of deprioritizing certain behavior, especially targeted at a specific service is completely counter-productive and inappropriate:  people who want to repeat the discouraged behavior will find a way around it, and other people who are otherwise legitimate may suffer by being inappropriately affected by the change.

In my mind, the only reason this is an issue is because the community was not prepared to handle the rapid increase in size -- and it will work itself out once users/dev figures out the way to accommodate it.  There just may be some turbulence in the short-term.  

[interlagos]

[For reference:  the idea I proposed is a special blockchain pruning method that allows nodes to verifiably query any address balance with only a couple kB download -- and the pruned data would be maintained & enforced on a second/alternate blockchain using merged mining]

A quick question.

Does downloading of few kilobytes imply a connection to full nodes for that or the same lightweight nodes might also happen to have this info?

If network is mostly populated by nodes who are asking for those small downloads and most of the nodes don't have the data then where is it going to come from?

[SgtSpike]

[For reference:  the idea I proposed is a special blockchain pruning method that allows nodes to verifiably query any address balance with only a couple kB download -- and the pruned data would be maintained & enforced on a second/alternate blockchain using merged mining]

A quick question.

Does downloading of few kilobytes imply a connection to full nodes for that or the same lightweight nodes might also happen to have this info?

If network is mostly populated by nodes who are asking for those small downloads and most of the nodes don't have the data then where is it going to come from?

From trusted nodes.

[Matt Corallo]

This should be a non-issue in the long term:  If the network can't handle 50k tx per day, Bitcoin was never meant to succeed.  The only reason it's such a big issue right now is because it's a single entity producing the volume -- and thus there's somewhere to point the finger when it was really our lack of preparation to handle it.  It could've just as easily been a piece of some other market that latched onto BTC and produced 50k tx per day in a less-centralized manner.  

The point isnt that we cant handle the volume.  We clearly can.  I dont see Bitcoin dying right now.  The issue is that its much better to have the network /slowly/ transition into SPV nodes so that we can closely monitor and make sure there are enough listening full nodes.  Additionally, events like this are important as it lets us reexamine the existing incentive structure and address issues like what is, to most end-user essentially a DoS attack.  The issue isnt that we have a place to point a finger at, its that one user is abusing the network.  Im suggesting a way to discourage people from abusing the network in such obvious ways, hopefully encouraging people to use more sane send-methods.

It's a downside of having a completely open, decentralized, apolitical, global currency scheme -- people must have the ability to use it as they please, and the rest of the system must find equilibrium.  Talk of deprioritizing certain behavior, especially targeted at a specific service is completely counter-productive and inappropriate:  people who want to repeat the discouraged behavior will find a way around it, and other people who are otherwise legitimate may suffer by being inappropriately affected by the change.

Actually, its not targeted at a specific service.  Its targeted at all services that have high volume and can reasonably wait for confirmations.  I never said it would be hard for people to work around the deprioritization, quite the opposite.  But it would force people to think about what they are doing.

In my mind, the only reason this is an issue is because the community was not prepared to handle the rapid increase in size -- and it will work itself out once users/dev figures out the way to accommodate it.  There just may be some turbulence in the short-term.  

Thats what Im suggesting - a way to attempt to accommodate the rapid increase - make people who insist on DoSing the network in an obvious way wait longer for confirmations.

[Realpra]

"Normal" users should not be using bitcon-qt.

Well, it's the official client for one.
For two, the more people that stop using a "full" client, the fewer full client nodes we have, and the less secure the network is.

+1

I don't get this talk of alternate chains or why it should be so damn difficult.

If there is a block with an invalid tx someone reports it over the peer network. The bandwidth usage, even if you have send the documenting block, would be negligible as it would happen so rarely.

To get your own balance you query your peers as with torrents and if there's an update they send that block to you.

To work you would need more than 8 peers/a smarter protocol, but that's hardly rocket science.


That is like 2-3 updates, what reason NOT to do it?

[caveden]

Please, really please don't make that a rule embedded in bitcoind.

Why not? the point is its to the benifit of the people running bitcoind, so its in their best interest to do such rate-limiting.  In the end, its really less of a deprioritization and more of a "stop-DDoSing me, Im gonna rate limit your transactions"-feature.  The goal is to tweak incentives so that the end-users and those running forwarding nodes can have a say in the incentive structure, as is ultimately fair, as they are putting in way more effort than miners in terms of verifying transactions.

If it's just a "flood defense" then I'm fine with it, as long as it is configurable. Like, in my config file, I add that a node sending >10tps during >3s is flooding, so I stop downloading some or all of his tx.
Such a thing could finally allow the devs to drop the hardcoded transaction fee policy, which should never have existed IMHO.

Now, where I strongly disagree with you, is that SatoshiDice is flooding the network. It is not. Their transactions are legit. Something should be considered DoS if that's the sole goal of it. Like, someone creating as many transactions to self as he manages and broadcasting them all, only to flood the network. That's bad behavior.
SatoshiDice is not flooding and is legitimately paying for his transactions. Transactions paying something above a certain low limit in fees should never be considered DoS - and this limit should also be configurable.

[interlagos]

[For reference:  the idea I proposed is a special blockchain pruning method that allows nodes to verifiably query any address balance with only a couple kB download -- and the pruned data would be maintained & enforced on a second/alternate blockchain using merged mining]

A quick question.

Does downloading of few kilobytes imply a connection to full nodes for that or the same lightweight nodes might also happen to have this info?

If network is mostly populated by nodes who are asking for those small downloads and most of the nodes don't have the data then where is it going to come from?

From trusted nodes.

That would apply to Electrum/Stratum architecture, but there is no such thing as a trusted node in etotheipi's proposal.

[Matt Corallo]

If there is a block with an invalid tx someone reports it over the peer network. The bandwidth usage, even if you have send the documenting block, would be negligible as it would happen so rarely.

That is an interesting change to the trust model...But it is significantly easier to simply have a full node/SPV node distinction and have all full nodes do the checking.

To get your own balance you query your peers as with torrents and if there's an update they send that block to you.

So...SPV clients after the Bloom filter change...

To work you would need more than 8 peers/a smarter protocol, but that's hardly rocket science.

>8 outgoing connections is not really an option.  Last I checked (which was a while ago) the ratio of open incoming connection slots on listening nodes to outgoing connection slots on all nodes wasnt great.  Even if it is, increasing the outgoing connection slots on every node has a huge effect on that ratio, and it would be very easy to blow through the listening slots available on the network.  Even hurting that ratio makes some attacks much easier.

[Matt Corallo]

If it's just a "flood defense" then I'm fine with it, as long as it is configurable. Like, in my config file, I add that a node sending >10tps during >3s is flooding, so I stop downloading some or all of his tx.
Such a thing could finally allow the devs to drop the hardcoded transaction fee policy, which should never have existed IMHO.

Flood defense against individual nodes doesn't really make sense and is just punishing that node for no reason.  The point is to protect against flood from particular addresses.  Also note that it doesnt really effect the hard-coded fee policy which addresses radically different aspects of a tx's "badness".  That said, I do agree that hardcoded fee policy is bad, but its really there to prevent other trivial DDoSing, as most standard txes should never hit that.

Now, where I strongly disagree with you, is that SatoshiDice is flooding the network. It is not. Their transactions are legit. Something should be considered DoS if that's the sole goal of it. Like, someone creating as many transactions to self as he manages and broadcasting them all, only to flood the network. That's bad behavior.
SatoshiDice is not flooding and is legitimately paying for his transactions. Transactions paying something above a certain low limit in fees should never be considered DoS - and this limit should also be configurable.

SatoshiDice isnt flooding the network because they have high volume, they are flooding the network because they refuse to use features like multisend which would keep their network load down, while still allowing them to have the same volume.

[caveden]

"Normal" users should not be using bitcon-qt.

Well, it's the official client for one.

There's nothing "official" in Bitcoin, and the fact that you think there is shows an understanding issue.
Saying that bitcoin-qt is the "official Bitcoin client" is like saying Microsoft Outlook is the "official POP3 client" or that Firefox is the "official HTTP client".

Granted, it is the "reference implementation", the only one so far which fully implements the protocol, and as consequence, the one which actually defines it. That's different from being "official" though.

For two, the more people that stop using a "full" client, the fewer full client nodes we have, and the less secure the network is.

Network security is much more related to the amount of computing power behind mining than the amount of full nodes in the network.
It's unlikely that someone manages to DDoS or hack all full nodes at the same time, even if only solo-miners and pool operators were running full nodes.

And, please, understand: if bitcoin succeeds, it is just a matter of time until this happens (few full nodes). If that really makes Bitcoin less secure as you say, then you may say Bitcoin is not secure by design.

[caveden]

Flood defense against individual nodes doesn't really make sense and is just punishing that node for no reason.

Err... for the reason of punishing flooders? If you're relaying the DoS you're a flooder yourself too.

  The point is to protect against flood from particular addresses. 

But then the flooder just generates more addresses. What's the point?

Also note that it doesnt really effect the hard-coded fee policy which addresses radically different aspects of a tx's "badness".  That said, I do agree that hardcoded fee policy is bad, but its really there to prevent other trivial DDoSing, as most standard txes should never hit that.

But it is avoiding these "trivial DDoS" I'm talking about. But doing so via download limits and all, not via fee policy.

SatoshiDice isnt flooding the network because they have high volume, they are flooding the network because they refuse to use features like multisend which would keep their network load down, while still allowing them to have the same volume.

Regardless, they're making a legit use of the technology, not a DoS attack. Specially if they're paying for it.

[Realpra]

If there is a block with an invalid tx someone reports it over the peer network. The bandwidth usage, even if you have send the documenting block, would be negligible as it would happen so rarely.

That is an interesting change to the trust model...But it is significantly easier to simply have a full node/SPV node distinction and have all full nodes do the checking.

The idea is to avoid the need for full nodes: I want a 1 tier network.

After re-reading the scalability wiki it seems you could split the block up in to a list of hashes.

This ALSO means you should be able to document an invalid tx with just a few hashes sent along with your report.

To get your own balance you query your peers as with torrents and if there's an update they send that block to you.

So...SPV clients after the Bloom filter change...

Yes except they DO store part of the chain AND they do verify against those parts.

Alone none of the nodes are full, but together they act as a network of full nodes.

To work you would need more than 8 peers/a smarter protocol, but that's hardly rocket science.

>8 outgoing connections is not really an option.

Alright my bad.

In that case do "peer switching" if all your 8 peers don't have what you need make them give you one of their peers that DOES.

That way ALL nodes could be storing/sending/verifying >>1% and still function as a full node network with the same safety.

EDIT: Miners would still need to build and send full blocks, however building said blocks could become outsourced to mining pool clients. (from the scalability wiki)

[Raoul Duke]

Why are you all so eager to get the network comprised of only "trusted nodes"(trusted by whom?) and "lightweight clients"?

Way to kill decentralization, IMO.

[Realpra]

Why are you all so eager to get the network comprised of only "trusted nodes"(trusted by whom?) and "lightweight clients"?

Way to kill decentralization, IMO.

I shall not be paying for your silence

[Matt Corallo]

Err... for the reason of punishing flooders? If you're relaying the DoS you're a flooder yourself too.

You are the one who insists it be configurable as to how much you forward (and I, mostly, agree with you).  You cant have it both ways or you end up punishing everyone and dropping connections.

But then the flooder just generates more addresses. What's the point?

As stated above, punishing high-traffic addresses by no means will remove the potential for the problem to occur.  The goal is really because people who do use one address usually dont care about confirmation times, so they can gladly still use them and some might.  Really its to make people think twice before designing their site in an entirely braindead way.

But it is avoiding these "trivial DDoS" I'm talking about. But doing so via download limits and all, not via fee policy.

Doing so via download limits removes the possibility of users using Bitcoin for some things that may be very desirable.  Forcing fees before forwarding transactions that we otherwise wouldnt forward at all allows those transactions to exist.

Regardless, they're making a legit use of the technology, not a DoS attack. Specially if they're paying for it.

a) they arent making a legitimate use.  Arguing that it works under the current rules is not a reason to not change the rules to make it not possible.
b) they aren't paying for it.  Paying miners does not help or help the high load it inflicts on end-users.