[ANN] BitEN - bitcoin erlang node

[kokjo]

Code:

5> Start time..2013.03.27 20:44:01 UTC
Cur time....2013.03.27 21:18:05 UTC
Connects:...61110
Connected:..433
Accepted:...1
Answers:....433
Cur peers...320
Relayed TX:.37001
Mempool:
 exception: exit:{timeout,{gen_server,call,[mempool,get_stats]}}
Errors:
 timeout:...24231
 refused:...3412
 unreach:...1873
 other:.....30466

[aantonop]

Do not think he will be impressed - it's my first "real" OTP project. I hope it will be able to run more than 24h without problems - right now after 2-3 hours it terminates.

He is impressed when people know about and try to use erlangOTP. Heck, I'm impressed too.

Working code. That's all that matters. The rest is only a matter of time. Thank goodness Linus didn't say: It's only a floppy driver a screen driver and a kernel that keeps crashin, I'd better not show it to anyone until it's "ready". I downloaded that "piece of shit" in 1993 on 210 floppy disks. I hear Linux turned out ok.

[r.willis]

I think I fixed it.
Please test. Pull latest version, recompile, then:

Code:

git clone git://github.com/vinoski/erlsha2.git
cd erlsha2
rebar compile
cd /path/to/biten
erl -pa ebin -pa /path/to/erlsha2/ebin
application:start(sasl).
application:start(biten).

Later I'll add copy of erlsha2 library to my repository.

[r.willis]

Start time..2013.03.27 22:50:33 UTC
Cur time....2013.03.27 23:29:55 UTC
Connects:...3281
Connected:..1199
Accepted:...24
Answers:....1197
Cur peers...1001
Relayed TX:.204155
Mempool:
 inv table..49
 req table..20
 tx table...571
 peer table.49
 check inv..0.0003 s
 clean inv..0.0002 s
 clean req..0.0001 s
 clean tx...0.0005 s
Errors:
 timeout:...1612
 refused:...289
 unreach:...167
 other:.....5

No signs of lags, etc. This is on my low-end VPS with 256 MB RAM without swap.

[tvbcof]

Interested.  Sorry for the spam, but I would rather have thread updates as replied vs. watchlist.

[Matt Corallo]

So...this is some kind of way to try to DoS bitcoin?
It connects to tons of nodes, eating listening sockets on them all, without listening to add more to the pool.
Does it relay things? It would appear that it relays unchecked objects, so anyone can use a node of this to amplify its bandwidth when attacking the network.

[aantonop]

So...this is some kind of way to try to DoS bitcoin?
It connects to tons of nodes, eating listening sockets on them all, without listening to add more to the pool.
Does it relay things? It would appear that it relays unchecked objects, so anyone can use a node of this to amplify its bandwidth when attacking the network.

It should be run on testnet, ok. But DoS? Come on!

This is working bitcoin code in a language that scales massively and you're looking the gift horse in the mouth.

If someone wants to DoS bitcoin, there are far easier ways to do it, all of which won't succeed or do much for long. None that involve this much coding effort.

[Matt Corallo]

Writing clients that are capable of relaying tons of verified messages and helping the network is great.  But this doesnt actually do that.  Making a few outbound connections and listening for tons of inbound connections is hugely useful for the network, the opposite is just a DoS.
Dont be confused, add some basic anti-DoS verification and change one or two constants to limit the maximum outbound connection count and this would be a really awesome project, as-is, however....

[aantonop]

Writing clients that are capable of relaying tons of verified messages and helping the network is great.  But this doesnt actually do that.  Making a few outbound connections and listening for tons of inbound connections is hugely useful for the network, the opposite is just a DoS.

You're the CI/jenkins tester for bitcoind, right? Thanks for that, it's a very important contribution.

I am surprised you would discourage code contributions in other languages when the current network is so dangerously mono-culture and susceptible to major problems because of bugs. It seems to me that bitcoind does a fine job DoS-ing itself every now and then, and that fact that there are no alternative clients may have something to do with it.

Insular attitudes make insular code which makes fragile and brittle code.

[Matt Corallo]

I am surprised you would discourage code contributions in other languages when the current network is so dangerously mono-culture and susceptible to major problems because of bugs.

No, I totally agree with such sentiment, but there is a difference between doing that and creating something that blindly relays any messages to every peer it can find

It seems to me that bitcoind does a fine job DoS-ing itself every now and then

No disagreement here.

and that fact that there are no alternative clients may have something to do with it.

Not sure about this, but I certainly agree it would help prevent some simple attacks. 

Again, I have to say, I like the idea, it just needs to be modified a bit to avoid being a distributed DOS client.

[Zeilap]

To be fair, r.willis acknowledged the issue earlier, before he released the code. Perhaps it was the reason he was reluctant to release the source to soon.

But if it does not verify transactions/blocks it relays, I effectively becomes threat to network, as if there is many such nodes, spammers can use them to amplify their abilities to flood nodes with malformed/fraudulent transactions.  

Or maybe the plan was indeed to have no verification in order to reduce transaction and block propagation time.

[gmaxwell]

Insular attitudes make insular code which makes fragile and brittle code.

This is really orthogonal, as is the code uses 125x network resources compared to normal node software. That should be changed. Otherwise no big deal.

[r.willis]

Yes, I'm aware of possible misuses of my software, and I told about them earlier in the thread. It does not check TX it relies. This is bad, and out-of spec, or is it? (Oh, wait, spec is the Satoshi client code, need to check what it says about it).
Verifying TX is my next goal.
There is connection limit (in config.erl, it's 1000 by default). It does have incoming connection support (not limited, counted towards 1000). For it to work, you must configure your external IP in config.erl.
How many nodes does testnet have? I'm interested in maintaining thousands of connections to reduce network diameter and propagation speed. I'm sure network part will work with 10 (or 100) nodes.
It does not do more than 1 connection per host:port pair, and tries  such pairs only once in lifetime. So, impact on each node is minimal (as long there is not many of nodes running my software).

[aantonop]

This is really orthogonal, as is the code uses 125x network resources compared to normal node software. That should be changed. Otherwise no big deal.

I think it is very much the problem. Look at the attitude displayed from the very get go, on a thread where someone is contributing code.

Insular is the perfect description, of that response, the attitude of some of the devs and the resulting code.

Where in the spec does it say what the maximum nodes should be? Oh, right - there is no spec. Just the "normal node software".

[r.willis]

For now, I'v complied with gmaxwell's suggestion and changed default number of peers to 10.
After verification code is in working shape, we will discuss this once more.

[tvbcof]

This is really orthogonal, as is the code uses 125x network resources compared to normal node software. That should be changed. Otherwise no big deal.

I think it is very much the problem. Look at the attitude displayed from the very get go, on a thread where someone is contributing code.

Insular is the perfect description, of that response, the attitude of some of the devs and the resulting code.

Where in the spec does it say what the maximum nodes should be? Oh, right - there is no spec. Just the "normal node software".

Off hand I would suspect that 'reducing network diameter and propagation speed' might have a variety of advantages and dev might want to start thinking about the ramifications if this happened to become the 'new normal'.

[Matt Corallo]

Yes, I'm aware of possible misuses of my software, and I told about them earlier in the thread. It does not check TX it relies. This is bad, and out-of spec, or is it? (Oh, wait, spec is the Satoshi client code, need to check what it says about it).

Actually, the client (and other  documents) clearly define the nServices bit NODE_NETWORK to indicate verification and full node status.  Though you may end up relaying crap and getting banned as a DoS'er by full nodes, if you just did SPV verification and relayed without NODE_NETWORK it would technically be within spec (though non-optimal).  

Verifying TX is my next goal.

Full verification or SPV verification? In any case, that is very good news.

There is connection limit (in config.erl, it's 1000 by default). It does have incoming connection support (not limited, counted towards 1000). For it to work, you must configure your external IP in config.erl.

OK, I did not see that it could accept incoming connections, in any case, that is very nice.  I would, however, like to point out that the two types of connections represent two completely different purposes:
Incoming connections allow you to serve data to other peers who need connections.  It helps the network by using your bandwidth to provide relaying.
That said, we cant purely rely on incoming connections because there is no way to know if they are all connections from one attacker.  So we make a few outbound connections to make sure we have diversity.  
I'd like to note that more outbound connections by everyone does not result in greater propagation speed.  It eats /tons/ more bandwidth and you will end up shooting yourself in the foot.

How many nodes does testnet have? I'm interested in maintaining thousands of connections to reduce network diameter and propagation speed. I'm sure network part will work with 10 (or 100) nodes.

Again, attempting to create more connections to reduce network diameter ends up shooting yourself in the foot, it actually wont end up decreasing propagation speed.  Furthermore, propagation speed isnt an issue in today's Bitcoin network.  Sure, miners want the best propagation speed they can find so they dont mine stale blocks, but the large pools mostly have private peering with other large miners to ensure that it is never a problem for them.  P2Pool also relays the minimum data required to relay blocks found to all p2pool nodes in addition to announcing them on the Bitcoin network at the same time, ensuring the blocks get introduced to the Bitcoin network at many points at once.  Finally, if you simply allow incoming connections, you will almost always get around a hundred nodes after a day or two, which results in really damn fast relaying for anything you generate.

It does not do more than 1 connection per host:port pair, and tries  such pairs only once in lifetime. So, impact on each node is minimal (as long there is not many of nodes running my software).

"Impact of my DDoS tool is very small, as long as only a few people are running my DDoS tool." You see the problem here...

Again, no one has issues with some of the ideas of this project, its some of its specific implementation details that are broken.

[r.willis]

Matt Corallo, thank you for your kind words.
What if I want not 100, but 1000 incoming connections - can I open 10 (100, 1000) listening ports and announce all of them via addr messages - will it increase likelihood of other peers selecting me as their peer or would it fail due to binning? Or is the only viable alternative getting more IPs from different bins?
As for verification, I'm planning to do full verification eventually.
UPD:

Actually, the client (and other  documents) clearly define

Which documents?
Actually, the only document I found (wiki) defines it in this manner:

1    NODE_NETWORK    This node can be asked for full blocks instead of just headers.

"Impact of my DDoS tool is very small, as long as only a few people are running my DDoS tool." You see the problem here...

If I wanted to make DDoS tool, I'd make one. I did not ask people to run it, people asked for code. If bitcoin network can be brought down by couple weak PCs, then something needs to be done to "standard client". There I see the problem.

[Zeilap]

Clients will not propagate invalid blocks (or invalid transactions)

[r.willis]

Please define "invalid transaction".

Furthermore, propagation speed isnt an issue in today's Bitcoin network.

Do you have some hard data on this? I have some, suggesting otherwise. https://bitcointalk.org/index.php?topic=160326.0