Bitcoin Forum
January 20, 2017, 05:50:43 AM *
News: Latest stable version of Bitcoin Core: 0.13.2  [Torrent]. (New!)
  Home Help Search Donate Login Register  
  Show Posts
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 ... 232 »
1  Bitcoin / Development & Technical Discussion / Re: Post your SegWit questions here - open discussion - big week for Bitcoin! on: January 18, 2017, 12:07:21 AM
Bitcoin hard forks happened on On August 15, 2010,
That is totally untrue.  Sad   What should people think of the rest of your comments?
2  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 16, 2017, 03:27:16 PM
Instead, when we meet at the next Bitcoin event we'll both be attending, I'll approach you and we'll handle our arguments like real men. Promise.
"Violence is the last refuge of the incompetent."
3  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 15, 2017, 06:05:56 PM
"I started making keys, starting with ones with fewest cuts and systematically working through all possibilities. To learn if these keys matched any that had been used in the past, I tried each one in every door in the neighborhood.  After a bit I found a few valuables. What was I supposed to do, leave them there?"
4  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 15, 2017, 04:20:21 PM
Ok. Hasn't explained why cmov implementation looks like it does, because right now - after this explanation - it seems to me cmov is short for "constant time move"
It is a conditional move, and does exactly what the comment says it does: "If flag is true, set *r equal to *a; otherwise leave it. Constant-time.". The construction used for it is one that results in constant time behavior on our target systems, isn't undermined by the compiler, and which is relatively fast.   A normal move is constant time but not conditional, so "constant time move" wouldn't make much sense.

I am trying to find a hash160 collision.
If so, only by a highly inefficient means. hash160 is a hash function, there is no EC involved at all. If you were merely attempting to  find a hash160 collision you'd simply need to run hash160. You also wouldn't have any reason to check your results against assets in the bitcoin chain...  You might also use an efficient collision finding algorithm instead of brute force.

If you have evidence that I have stolen something in the past or evidence I plan to steal something in the future, please either present that evidence

You stole funds in this transaction:

And if you continue with your allegation(s), I really have no problem to step out of (pseudo)-anonymity and let my lawyers drill your sorry ass until they hit crude oil. And this is not even a rude statement.
Bring it on.
5  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 15, 2017, 03:25:27 PM
Well - it isn't for me, right now I am interested in generation performance.

Yes, we've already established that you're primarily interested in stealing from people and that you're unable to understand that many others are not exactly supportive of your goals.

You seem to think that people who don't slavishly help you steal from others are bad or incompetent people, or at least that you can get them to help you by insulting them as though they were. For someone who claims to be so old, you sure don't seem very wise. Smiley

my advanced intuition tells me (1) is the magnitude of the result,

Yes, the comments trace the magnitude of the results; and tie the code back to the algebraic verification of the functions (in sage/). There is no point in a comment that merely repeats the code.

if (degenerate) {
n = m;
else {
secp256k1_fe_sqr(&n, &n);

Hm. Works. Is faster. Tests says "no problems found". And now the code makes sense.
Of course it is only a trivial thing. Probably not worth the effort. Right?

So conditional move you say?

/** If flag is true, set *r equal to *a; otherwise leave it. Constant-time. */
static void secp256k1_fe_cmov(secp256k1_fe *r, const secp256k1_fe *a, int flag);
yes? Why do I see then this epic kludge in the code?

Congrats, by ignoring the _constant time_ in the function documentation you just leaked the users secret to a timing side-channel-- and _still_ managed to end up with code considerably slower than simply changing to non-constant time competently.

6  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 14, 2017, 06:59:07 PM
No - I came to you saying that I would like to work on improving the block download times.

I really suggest you read the log. That might be what you were intending on doing; but what you did was some and insist on a specific protocol change to allow downloading blocks in tiny chunks.

We explained why this was unlikely to improve performance, was likely to create vulnerabilities, and suggested some things that we thought more likely to be successful.  We also suggested you try making the changes in your own protocol so you could measure the result, and not just believe us or rely on speculation.

Why don't you want to discuss the protocol  changes needed for bootstrapping clients with utxo snapshots?
Because you obviously don't.

Huh? Because it's far far offtopic and you're just switching subjects-- but I've talked about this many times here before. If you're referring to just trusting miners to give you a correct UTXO snapshot it is a _massive_ change in the security model-- and if you're happy with that change you can just use SPV.  If you're talking about static cached UTXO data, then there isn't any consensus change needed... and software implementing that just ... implements it.

And you told me off,  saying basically that it wasn't necessary. Plus also some other bullshit about BIP37 that didn't turn out to be true.

10:03 < tonikt> Like being able to download a block in fragments
10:03 < sipa> tonikt: BIP37 allows that, in a way
10:05 < gmaxwell> tonikt: Not a very worthwhile thing in my opinion, and as sipa points out its already possible.

What you were specifically suggesting wasn't necessary or helpful. And what we told you about BIP37 was true (afaict), you could have used it to download chunks of blocks. (and though it wasn't mentioned there Pieter had already tried it out, though mostly to eliminate already transfered txns... but the overheads made it not a useful improvement).

But then few years later,  suddenly improving the block download times became so much desirable and you've been so proud of you delivering it to the public.
Don't confuse some specific proposal being unhelpful for the goal which you hoped for it which it could not accomplish with that goal being an unhelpful.

You were very aggressively pushing a particular change based on a flawed belief that it would help, all I was doing was pointing out why I didn't expect that particular change to be helpful (that transferring data in 32KB chunks would be slower due to latency/overhead and more vulnerable), as well as suggesting what you could to prove out the effectiveness of the change or do it on your own without worrying about what we thought.  It seems you only understand working on things exactly the way you think they should be done, and believe that anyone who doesn't is saying they don't care about improvement. I think you should consider some other possibilities.
7  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 14, 2017, 06:12:24 PM
Repeating the some unsubstantiated claim doesn't make it substantiated.   You just made a claim about this very thread-- and yet when we look-- nothing behind it.

Just me, years ago, trying to talk on #bitcoin-dev about ways to speed up block downloading times. That was a no no - block propagation didn't need improving because you didn't think it was important... It almost got me banned from the channel..  And yet we are; 2016 - a new brilliant feature that the bitcoin core team is so proud of: compact fucking blocks!

The logs are all public, please point us to this conversation:   If you can recall a name/phrase from the discussion, I will happily search for it.

Edit: Reading all logs from you in that channel only took a couple minutes, here is the conversation (trimming irrelevant side discussions):

10:02 < tonikt> Hi guys. I was wondering whether there have been any discussion on changes in the network protocol?
10:03 < tonikt> Like being able to download a block in fragments
10:03 < grau> tonkit: there is already bloom filter download for blocks
10:03 < tonikt> Or: see a transaction size, before downloading it
10:03 < sipa> tonikt: BIP37 allows that, in a way
10:03 < tonikt> grau: bloom filter is not good for full node
10:04 < grau> block fragments are no good for full node either
10:04 < tonikt> grau: why not?
10:04 < sipa> you can have two complementary bloom filters set on two connections
10:04 < sipa> and download half blocks from each
10:04 < tonikt> so the answer is 'no'?
10:05 < gmaxwell> tonikt: Not a very worthwhile thing in my opinion, and as sipa points out its already possible.
10:05 < grau> tonkit: a fragment can not prove that funds are not spent in the other part
10:05 < sipa> the answer is 'partially', BIP37 supports (some way) of downloading blocks in fragments
10:05 < tonikt> I believe the network could really use being able to download one block from diffetent peers in i.e. 32KB chunks
10:05 < sipa> that'd only slow things down imho
10:05 < tonikt> gmaxwell: but I was checking these bloom filters and it wasn't possible to download a block like that
10:05 < sipa> and there is no way to prove that a transaction has a certain size without actually sending that transactions
10:06 < sipa> tonikt: it is
10:06 < gmaxwell> tonikt: it is, but as sipa and I are telling you— it's not obviously useful.

10:06 < grau> tonkit: it is already allowed by the protocol to use several peers for download. BOP actually does that by block not by block fragment
10:06 < tonikt> sipa: yes - it isnt possible to confirm it, but if you see a mismatch you can at least ban the peer
10:06 < sipa> tonikt: set nHashFunctions=1, and use random complementary bits in the filter
10:06 < tonikt> grau: I al talking about a fully synchronized node
10:06 < gmaxwell> tonikt: what are you talking about?
10:07 < tonikt> you suddenly get an INV with a new block - you want to download it ASAP
10:07 < sipa> what is the actual problem you're trying to solve?
10:07 < tonikt> ... so why not to split the work into parts and downlaod it in paralell?
10:07 < sipa> we've just told you how to do that using BIP37

10:07 < tonikt> wait, I cannot read that BIP37 cause wiki is broken
10:08 < gmaxwell> Because doing so will _slow_ transmission except to the extent that it gets you an unfair share of the channel capacity.
10:08 < tonikt> maybe you are talking about a different thing Smiley
10:08 < sipa> 19:06:43 < sipa> tonikt: set nHashFunctions=1, and use random complementary bits in the filter
10:08 < sipa> tonikt: in practice it'd be very hard to coordinate that, but if block sizes would grow a lot, that may be a viable strategy
10:09 < tonikt> just to be clear: BIP37 was about downloading the header and the tx hashes, followed by the actual transactions?
10:09 < sipa> BIP37 = bloom filtering
10:09 < tonikt> yes
10:09 < tonikt> so how does it help me to split a block download amoung my 50 peers?
10:09 < sipa> so you give node A a random filter that selects 50% of the transactions
10:09 < sipa> and you give node B a complementary filter that selects the other 50% of the transactions
10:10 < tonikt> yes, but I have more than 2 peers.
10:10 < sipa> ok, then give node A a filter that selects 33% of the transactions
10:10 < tonikt> your solution seems more like a work around
10:10 < sipa> it's a very neat solution, as you don't need to keep track of what to download from whom
10:10 < tonikt> ok, sorry. let me ask a question then
10:10 < sipa> they figure it out themself
10:11 < grau> tonkit: and you seem to work around a problem not really there until blocks are the size we know.
10:11 < gmaxwell> tonikt: you're just going to end up in N connections all in slow start. Plus users setting your house on fire because you use _all_ their bandwidth in a burst once the connections come out of slowstart.
10:11 < sipa> maybe this will become an actual problem if the block size limit is increased
10:11 < sipa> if it is, we'll deal with it
10:11 < tonikt> why cant there be like a command "getsomething" that would return me the length of that something, plus a list of hashes of its data split into i.e. 4KB chunks
10:11 < tonikt> ... and that would be same for txs and blocks
10:12 < gmaxwell> sipa: sure if there are larger blocks then eventually at some point it makes sense. A few hundred K is really sketchy for any benefit there.
10:12 < tonikt> gmaxwell: no - now I use their bandwidth in a burst, asking each peer for entire block
10:12 < tonikt> I'm talking about a situation when we have a new block mined
10:12 < tonikt> which is every 10 minutes
10:12 < gmaxwell> You don't ask each peer for the entire block.
10:13 < tonikt> well, you should if you care to have it ASAP Smiley
10:13 < tonikt> I do Smiley
10:13 < gmaxwell> You ask a single peer for the entire block, or as sipa points out multiple peers for mutually exclusive subsets.
10:13 < tonikt> yes, I understand that is the status
10:13 < gmaxwell> tonikt: then you're abusing the network.
10:13 < gmaxwell> and also hurting your own performance.
10:13 < tonikt> but I was talking about a possible improvement
10:13 < tonikt> that is how bittorrent works
10:14 < tonikt> yes, call me an abuser Smiley
10:14 < sipa> do you use bittorrent for 1 MB data?
10:14 < tonikt> sometimes
10:14 < gmaxwell> sipa: normally torrents use 4-16 mb chunks.
10:14 < sipa> gmaxwell: i know
10:14 < tonikt> no
10:14 < gmaxwell> tonikt: what you're describing there sucks, because you can't tell who's screwed you and given you invalid data. With what sipa told you to do you can tell.
10:14 < sipa> it can use smaller chunks
10:14 < tonikt> I'm sure they use smaller ones - like 32KB
10:15 < sipa> gmaxwell said "normally"
10:15 < gmaxwell> tonikt: no, yes— it can, but thats not what actually gets used normally (except for tiny files, which torrent takes much longer to transfer than http)
10:15 < tonikt> OK, I get it - someone can send me a corrupt list of block's chunks hashes
10:15 < tonikt> but then...
10:15 < sipa> this is an engineering question, and it depends on very specific data (latency, bandwidth, variation, distribution)
10:16 < sipa> once it becomes a problem
10:16 < sipa> we can find an appropriate solution
10:16 < sipa> and there are several ways to deal with it

10:16 < tonikt> why don't you add a protocol command "give me this tx from this block?"
10:16 < sipa> because that would be absolutely horrible for the peers
10:16 < sipa> they need to look up the block for you, but only give you a small piece of it

10:16 < tonikt> sipa: with all due respect, but for you to find a solution, one needs to wait 2 years in average Smiley
10:17 < tonikt> It is actually quite easy to calculate
10:17 < tonikt> how much time you need to download lets say 1MB block
10:17 < tonikt> having 1Mbps connection - about 10 seconds
10:17 < sipa> right now the problem simply doesn't exist with 1 MB blocks, unless on very slow links (mobile?) where you don't want full blocks anyway
10:18 < tonikt> 10 senconds from peer to peer (and that's not counting checking it)
10:18 < tonikt> dont you think it is already enough to try decreasing it be a few folds?
10:20 < gmaxwell> tonikt: go convince bittorrent to never use chunks larger than 100k and come back. Tongue
10:20 < tonikt> I believe all the API is there already, except that it should be able to download a tx that is already mined, though by specifying a block where it is
10:20 < sipa> if you're on a 1 Mbps connection, there's no way you'll get it faster than in 10s anyway
10:20 < tonikt> gmaxwell: but bittorrent does not care about latency
10:20 < tonikt> while bitcoins hould
10:21 < sipa> if it's your peer that is limited to 1Mbps, but your connection is faster, you won't be downloading from him (as he'll be slower to announce it to you anyway)
10:21 < tonikt> ok, so you dont want to change the net protocol, before it becoming a problem
10:21 < gmaxwell> tonikt: generally parallel fetching is not great for latency.
10:21 < tonikt> fine
10:21 < gmaxwell> As you end up waiting on the most latent response before you can validate any of it.
10:22 < sipa> tonikt: i'm against changing the protocol before having clear information about the benefits
10:22 < sipa> and no, i don't think right now there is much that can be improved
10:22 < tonikt> gmaxwell: think if I could ask a node for a block and all its transaction hashes - and other nodes for transactions
10:22 < gmaxwell> tonikt: no, we're also saying that for the current maximum blocksize what you're suggesting is likely to _hurt_. Without a bunch of analysis it would be hard to know.
10:22 < sipa> in the future that can certainly change

10:22 < tonikt> I just dont see a problem with adding a block hash to inv while asking for tx data
10:23 < tonikt> it doesn't seem like a development challenge
10:23 < sipa> it's not
10:23 < tonikt> and it could solve the problem
10:23 < sipa> it's a maintainance overhead
10:23 < sipa> and a compatibility burden
10:23 < gmaxwell> We could also throw some virgins into volcanos, that might "solve the problem"
10:23 < gmaxwell> (not that you've established that there is even a problem to be solved)
10:24 < tonikt> ok, I get it - you don't see a problem
10:24 < sipa> *yet*
10:24 < tonikt> you are obviously not so much a perfectionists, as I am Tongue
10:24 < sipa> well we're not dealing with a nicely theoretical problem where the optimal soluton is obvious
10:24 < sipa> everything has downsides
10:24 < gmaxwell> tonikt: I don't see evidence of perfectionism in you in this discussion. If there were you'd be doing some careful analysis to establish some tests to determine the level of improvement possible.
10:25 < gmaxwell> Instead you're just shooting from the hip with a blind guess at something that would maybe help or hurt a problem which may exist in the future.
10:25 < tonikt> gmaxwell: all I can tell you is that, if I want to download a block ASAP, I ask each of my peers for the entire one - is this perfect for you?
10:26 < gmaxwell> tonikt: that won't actually fetch you the block faster than asking a single peer in many cases.

10:26 < tonikt> if I could download it in parts/transactions - that would be perfect, unless I'd screw up my implementation
10:26 < gmaxwell> No, in fact it wouldn't be.
10:26 < tonikt> Smiley no, it would
10:27 < sipa> you'd still have to wait for the slowest one to respond
10:27 < gmaxwell> tonikt: or look at it another way— why not ask them each for one bit of it?
10:27 < gmaxwell> as sipa says, you have to wait for the slowest response.
10:27 < sipa> if the only constraint is bandwidth, and processing speed and latency don't exist, your solution is optimal
10:27 < gmaxwell> sipa: and overhead doesn't exist.
10:27 < tonikt> guys, do I really need to explain you how to implement it?
10:27 < sipa> and attackers
10:28 < tonikt> you just have a list of txs to download and your peers - whenever any of them is busy, you ask him for the next ts
10:28 < gmaxwell> Right, so in a world where the only constrain is the remote peers bandwidth, and process speed, latency, attackers, and overhead don't exist— then indeed, thats optimal.
10:28 < tonikt> it must be faster
10:28 < sipa> how do you know he's busy?
10:28 < sipa> by waiting?
10:28 < tonikt> i know he is busy because he has not responded to my previous data request yet
10:29 < gmaxwell> It's unknowable without waiting because of latency.
10:29 < tonikt> so it does not make any sense to ask him for more data
10:29 < gmaxwell> tonikt: if he's 80ms away he _cannot_ answer faster than 80ms.
10:29 < tonikt> sure - but you have 1000 txs
10:29 < sipa> and you need all of them
10:29 < tonikt> eventually he will answer and you will have 900+ left anyway
10:29 < gmaxwell> so you're going to fetch them one at a time without pipelining? lol good luck with that.
10:29 < tonikt> and then you will ask him for the 913th one
10:29 < bmcgee> … I get the impression now's not a good time for asking potentially stupid questions …
10:30 < tonikt> gmaxwell: it's at least 200+ bytes
10:30 < gmaxwell> bmcgee: well, your question doesn't have a neat answer.
10:30 < sipa> tonikt: that is all very sensible, given the right bandwidth/latency tradeoffs
10:30 < tonikt> but youre right, it would be better to ask for several tx at the same time
10:30 < sipa> tonikt: it's something you certainly want to do at the ~megabyte level
10:30 < tonikt> except that some of thme may be 100kb big
10:30 < gmaxwell> tonikt: lol. just the TCP overhead from the request is going to instantly give you 50% overhead on 200 bytes.
10:30 < sipa> (and we don't by the way, so let's fix that first)
10:31 < tonikt> gmaxwell: now it gives you much more than 50%
10:31 < tonikt> most of the txs you have already anyway
10:31 < gmaxwell> tonikt: no, it doesn't the overhead on transfering a block is about 2%.
10:31 < sipa> tonikt: again BIP37 to the rescue (it doesn't send transactions it knows you already have, without extra latency)
10:32 < gmaxwell> okay, ignoring that. But as sipa says bip37 takes care of that.
10:33 < tonikt> gmaxwell: but using bip37 is working around a solution. if I need this tx from this block - why do I need to bother with bloom filters and statistics?
10:33 < sipa> tonikt: because it allows you to do things without extra latency
10:33 < sipa> tonikt: you don't have to be told about the list of transactions first, and you don't have to reply with which transactions you want
10:34 < gmaxwell> tonikt: statistics?! you set a series of complemetary bits. and don't have to taken another 80ms of round trip time to send extra tiny requests with redundant hashes.
10:34 < tonikt> sipa: but the biggest latency comes not from the ping - it somes from the data that needs to be finished, before they can be used
10:34 < gmaxwell> @#@*$(@#
10:34 < sipa> tonikt: on a 10 Mbit/s link, a not outrageous 200ms ping time (meaning 400ms extra for a roundtrip) means 400 kilobyte that could have been downloaded while they just waited for you to answer
10:34 < gmaxwell> tonikt: go look at the bandwidth numbers you gave before! on a 1mbit connection the data to transfer a fee hundred bytes is way less than typical latency.
10:35 < gmaxwell> er the time to transfer.
10:35 < sipa> that's more than an average block
10:36 < tonikt> ok guys, whatever. I see you have your world and don't really want to notice  mine.  I guess I will have to wait for it to become a problem Smiley

10:36 < tonikt> but if I might add something, not as a question, but as a proposal
10:37 < gmaxwell> Yes, my world has latency in it. Not sure where you get one that doesn't, but I'd like one of those. Tongue
10:37 < tonikt> 1) allow to ask for a size of a transaction/block before downloading it (so you can ban anyone who is trying to send you more)
10:37 < sipa> tonikt: what if the peer lies?
10:38 < gmaxwell> sipa: there are no attackers in tonikt's world.
10:38 < tonikt> 2) imagine that you are connected to 30 peers and a new 1MB block have just been mined: what is the fastest way to download it from your peers?
10:38 < sipa> depends on your bandwidth and latency
10:38 < gmaxwell> tonikt: if you think you can do better, just write a second transfer protocol. If it's better it should be easy to demonstrate. You'll probably learn something in the process.

10:38 < tonikt> sipa: if the peer lies, than you will find it out and ban it
10:38 < sipa> with very low bandwidth and low latency at the same time, downloading in parallel will certainly be faster
10:39 < tonikt> gmaxwell: believe me, I can write a protocol, but it would be quite silly to test it in my home network
10:39 < gmaxwell> tonikt: then use a network simulator.
10:39 < gmaxwell> It's pretty straight forward to simulate actual network behavior.
10:39 < sipa> tonikt: well to deploy it, we'd first need a way to download *anything* in parallel first

10:40 < tonikt> gmaxwell: but I dont need to simulate it to know that downloadin a block in parts from several peers at the same time will be faster
10:40 < gmaxwell> tonikt: But you're incorrect. The way you're describing that involves lots of round trips will actually be _slower_ in a case where there is considerable latency.
10:41 < gmaxwell> The exact balance depends on a number of factors.
10:41 < gmaxwell> Basically you never want to make a request that is smaller than the bandwidth delay product.

10:41 < tonikt> gmaxwell: no, becasue shorter transactions could go in bulk (like 20 * 200+ bytes)
10:41 < tonikt> .. thats why you need to have a way to find out tx size before downloadin it
10:42 < sipa> and just finding out that size means an extra round-trip, which (in some cases) may be slower than just downloading the whole block
10:42 < gmaxwell> Now you're transmitting a bunch of data to make those decisions, and then you can do nothing until it shows up. During that time you could have sent a whole block.
10:43 < tonikt> I think we should be talking numbers here, otherwise it's just baseless accusations
10:43 < gmaxwell> There are numbers above.
10:44 < tonikt> Can we agree that an average node would have 1mbps upload speed?
10:44 < tonikt> download is bigger - I know
10:44 < gmaxwell> Then when you get bad data it's impossible to tell who is giving you bad data until you have the whole block... which means that you have to fetch it from one peer if some peers is giving you bad data.. pretty cheap dos attack.

10:45 < tonikt> gmaxwell: of course you can say who sent you bad data, because you ask for transactions, which hashes you know
10:45 < gmaxwell> tonikt: e.g. I give you the wrong hashes for the block.
10:45 < tonikt> gmaxwell: with a proper difficulty? Smiley
10:45 < gmaxwell> huh?!
10:46 < tonikt> I can live with that
10:46 < gmaxwell> ...
10:46 < tonikt> I will compare the hashes against the merkle from the block
10:46 < sipa> which you don't have yet?
10:46 < gmaxwell> So now you have to fetch the whole merkle tree first. keep adding overhead.. (you'll end up with bip37 in a few more minutes)

10:46 < tonikt> The header is 80 bytes long
10:47 < sipa> the average transaction is 250 bytes or so
10:47 < sipa> a txid is 32
10:47 < tonikt> the minimal transaction is 250 or so Smiley
10:47 < sipa> that means you have to download 1/8 of the block's size before you can make any decision
10:48 < tonikt> I can live with that as well
10:48 < gmaxwell> tonikt: and again, you don't need to change the p2p protocol to expirement— you can just use an alternative p2p protocol, and simulate actual internet conditions.

10:48 < tonikt> its still 8:1 compression Smiley
10:48 < tonikt> I know what I can experiment with, guys
10:49 < tonikt> its just that I dont need to experiment to know that it would be a good thing to do
10:49 < gmaxwell> There are some bandwidth delay mixtures where some strategies are better than other ones, and different strategies are better in other conditions.
10:49 < tonikt> like this think recently, that they fixed
10:49 < tonikt> a peer sends you a longer tx than it should be
10:50 < sipa> that was actually an intentional design decision
10:50 < tonikt> you should ban it - but you cant, because it is likely a legit client, with a bug Smiley
10:50 < gmaxwell> tonikt: imagine— for a moment— that you peers are on mars with a 40 minute latency, and your bandwidth to each peer is 1gbit/sec, and your pay 1 BTC per megabyte transfered in aggregate. What is the optimal strategy?
10:50 < tonikt> gmaxwell: in this case I get your point
10:51 < tonikt> ... but I thought that we were on Earth
10:51 < phantomcircuit> gmaxwell, put your btc node on earth and send it instructions
10:51 < gmaxwell> tonikt: I used an extreme example because you keep rejecting the idea that different situations require different tradeoffs and you keep suggesting ones which have additional round trips, when it's possible to do this _without_ adding them.. so clearly you're not thinking about _something_.
10:51 < sipa> tonikt: banning would be a very bad idea - not because there are buggy clients that add random junk to transactions, but because you'd be hurting the ones forwarding an attacker's junk, not the attacker themself
10:52 < tonikt> so really, please, add an option to ask for tx/block size without a need to download it and allow to do getdata for tx giving a block hash as a reference - that's all I ask Smiley
10:52 < gmaxwell> banning on transitive behavior is a superfantastic way to convert mining dos attacks into network partitioning.
10:52 < gmaxwell> tonikt: We will not add that. Sorry.
10:53 < tonikt> gmaxwell: I know Smiley
10:53 < tonikt> but dont tell me later, that I did not suggest it Tongue
10:53 < sipa> tonikt: the first is unauthenticated data (someone can just lie, and you can claim you protect against it, but your optimal behaviour still depends on them being honest - i really prefer solutions that do not need such an assumption)
10:53 < gmaxwell> tonikt: Please create an alternative transport— in the process you'll learn something about the evils of roundtrips for performance, come up with a better proposal which is potentially useful.
10:54 < gmaxwell> Even a protocol that depends on honesty would be not the end of the world as an alternative transport: just use it between friends.

10:54 < tonikt> sipa: but if someone lies, you will find it out and you will ban it - that is the whole point
10:54 < gmaxwell> but it's not something that makes a lot of sense as the standard p2p protocol.
10:54 < sipa> tonikt: and you'll still have lost time doing so
10:54 < sipa> tonikt: something the attacker may not care about, but you do
10:55 < tonikt> sipa: yes, but you will pay this time to ban the bastard. now you have the same problem, but you cannot ban the bastared
10:55 < gmaxwell> tonikt: IPs are cheap, we regularly get trolls on IRC with access to thousands of IPs. Someone doing that could force all your nodes into wasting a ton of bandwidth and fall back to single peer fetching.
10:55 < gmaxwell> (and make you take many times longer to fetch the block)
10:55 < gmaxwell> I fully endorse my mining competition adopting such a protocol. Tongue
10:55 < tonikt> gmaxwell: 99kb is probably cheaper than in IP
10:55 < tonikt> an*
10:56 < tonikt> so I can download 99kb from the IP,  just to ban it for being wrong
10:56 < tonikt> especially if I know the size up front and I do not donload anything bigger than 10kb
10:57 < gmaxwell> or, you could, you know, use a protocol which doesn't depend on unauthenticated data and which doesn't require extra round
 trips.. and still fetches in parallel (if the bandwidth/latency ratios make it profitable to do so)
10:57 < tonikt> .. and if I see the 10001th byte - I can it already at that moment

10:58 < gmaxwell> And, in fact, BIP37 already gives us that, along with automatic (zero round trip) elimiating of known-already-sent data.
10:58 < tonikt> bip37 was a nice invention. the only problem with it is that nobody wants to use it
10:59 < tonikt> why wont you once invent something that people would want to use it, for a change? Wink
10:59 < gmaxwell> what are you talking about??
10:59 < gmaxwell> lol
10:59 < gmaxwell> every peer connected to me at the moment supports bip37.
10:59 < tonikt> supports, but does not get adventage of it
10:59 < gmaxwell> Now I think you're just trolling.
11:00 < tonikt> I guess you can always find someone who'd kick me out Smiley
11:00 < sipa> between satoahi clients, he's right
11:00 <@gmaxwell> I suppose I could. Tongue
11:00 <@gmaxwell> but seriously, what the heck.
11:00 < sipa> but my cell phone just loves bip37

11:01 < gmaxwell> tonikt: all the bitcoinj clients happily use it— but we don't think parallel fetching is currently useful in the satoshi client.
11:01 < phantomcircuit> iirc the fetch queue ends up pulling more blocks even before the queue has been processed right
11:01 < gmaxwell> After a bunch of archectural changes it might be useful.

11:02 < sipa> it's definitely useful at the block level
11:02 < phantomcircuit> so the pipeline stays full
11:02 < sipa> and we don't even do it there
11:02 < sipa> and we absolutely should
11:02 < gmaxwell> sipa: ::nods::
11:03 < sipa> phantomcircuit: yes, you can have up to 500 queued getdata requests
11:03 < sipa> whuch may take minutes to download
11:03 < gmaxwell> Doesn't involve the unauthenticated data / latency tradeoffs. And couple hundred k blocks are large enough that there isn't an overhead tax from doing that— beyond fetching the headers seperately.
11:11 < tonikt> so anyway guys, to wrap up, I did not mean to be mean, just to indicate my needs. and I appreciate your advise, but I am not going to make a network simulation just to convince you Smiley

So, you proposed instead requesting blocks 32KB at a time using more round trips-- basically demanding protocol changes without doing any testing or analysis to determine the benefit. We invited you to try out the protocol to observe the results, not merely telling you why we expected in common cases it would harm performance.

What you proposed is basically the _opposite_ of compact blocks (which eliminates round trips, and avoids transmitting most of the block at all). Sad Disappointing that you are posting here claiming it was your proposal.
8  Bitcoin / Development & Technical Discussion / Re: How to get sending address of Multisig/Escrow transaction? on: January 14, 2017, 05:22:01 PM
9  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 14, 2017, 04:49:28 PM
Just go back to the top of this topic.
Guys came with some ideas to optimize the lib - maybe not the most brilliant ones,  but also definitely not a stupid ones...
And what was your reaction?
You basically told them off.
You do it all the time.

Wtf?  Someone asked about two publications, asking if each would be helpful.   I responded that one would likely not be, the other would be somewhat and I pointed out it would be fairly easy to try. I certainly didn't tell them off!

I'll upload here 2 versions of /src/field_5x52_asm_impl.h that I've kind of hacked, one using memory, the other xmm registers.

The commentary is not good because it's not production level - just fooling around* with the data flow so that the data get from one end to the other faster, with less code imprint. I've never had them run on anything beside my Q8200, and I'm wondering on the behavior of modern cpus. I'd appreciate if you (or anyone else) can run a benchmark (baseline) + these 2, and perhaps a time ./tests as a more real-world performance.

If I do a ./time tests, both run faster by a second (58.2 seconds baseline with endomorphism down to 57.2 seconds in my underclocked Q8200 @ 1.86gz), although the memory version seems faster in the benchmarks. I have a theory on
Neat.  You shouldn't benchmark using the tests: they're full of debugging instrumentation that distorts the performance and spend a lot of their time on random things.  Compile with --enable-benchmarks and use the benchmarks. Smiley

A quick check on i7-4600U doesn't give a really clear result:

field_sqr: min 0.0915us / avg 0.0917us / max 0.0928us
field_mul: min 0.116us / avg 0.116us / max 0.117us
field_inverse: min 25.2us / avg 25.7us / max 28.5us
field_inverse_var: min 13.8us / avg 13.9us / max 14.0us
field_sqrt: min 24.9us / avg 25.0us / max 25.2us
ecdsa_verify: min 238us / avg 238us / max 239us

After (v1):
field_sqr: min 0.0924us / avg 0.0924us / max 0.0928us
field_mul: min 0.117us / avg 0.117us / max 0.117us
field_inverse: min 25.4us / avg 25.5us / max 25.9us
field_inverse_var: min 13.7us / avg 13.7us / max 14.0us
field_sqrt: min 25.1us / avg 25.3us / max 26.1us
ecdsa_verify: min 237us / avg 237us / max 237us

After (v2):
field_sqr: min 0.0942us / avg 0.0942us / max 0.0944us
field_mul: min 0.118us / avg 0.118us / max 0.119us
field_inverse: min 25.9us / avg 26.0us / max 26.4us
field_inverse_var: min 13.6us / avg 13.7us / max 13.8us
field_sqrt: min 25.6us / avg 25.9us / max 27.8us
ecdsa_verify: min 243us / avg 244us / max 246us

10  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 14, 2017, 03:14:27 PM
Strange. I was under the impression that

allowed me a tiny little bit of moaning.
It's a trivial optimization that already existed in three other places in the code. Thanks for noticing that it hadn't been performed there, and providing code for one of the two places it needed to be improved, but come on-- you're just making yourself look foolish here with the rude attitude while you're clearly fairly ignorant about what you're talking about overall. Case in point:

The endormorphism makes verification and ECDH significantly faster. It doesn't do anything else beyond additional endomorphism related tests.  

It does not make pubkey generation faster and really can't (well it could be used with some effort to halve the size of the in-memory tables at a small performance penalty.).

It's a little absurd that you insult a ~27% performance increase to verification while bragging about a under-half-percent change to verification performance.  It seems to me that you're trying to compensate for ignorance by insulting a lot, it might fool a few people who just don't know much of anything-- but not anyone else.  And it prevents you from learning. I do think wanna-be applies, in spades, and if you keep with that attitude it will probably continue to do so.

I don't think the person who wrote this did really care for performance.

And again, I ask you-- suggest something else similar with is 1/5th as fast, 1/5th as well documented-- I could also add 1/5th as well tested?  I'm not aware of anything.

No I haven't, because reading the secp256k1 code is a major PITA, but thanks for the pointer.

Odd to hear that, since the libsecp256k1 code has received many positive comments about its clarity and origination. One researcher described it as the cleanest production ECC code that he'd read.

Style preferences differ, of course,  -- lets see what your own code looks like


 <blink> <blink>

It wasn't 'the collective'.
Sipa wrote the entire library,  from scratch,  all by himself.
That is far from an accurate history, but it doesn't matter-- Pieter did do the lionshare of the work but he didn't do it in isolation. But less fortunate, is where your internal imagination continues and you write:

Which means that if you want to change any bit there,  you have to play the bitcoin celebrities PR game, which is mostly about indulging a  big egos of a few funny characters.
Which is just something you're purely making up. And it's unfortunate because if other people read it and don't know that you're extrapolating based on your imagination and fears they might not contribute where they otherwise might. That does the world a disservice.

And you can't seriously expect from and coder  to work on his personal  hobby project and  then deliver it with the industry standards documentation.
What kind of documentation would you even expect from a lib that provides a simple EC math functions?  The function names are descriptive enough if you understand the operations they provide. And if you don't understand the math behind it then no document is going to help you anyway.

But what is strange is that it is _extensively_ documented.

E.g. above rico666 commented on secp256k1_fe_cmov -- even if someone were ignorant enough of the subject and the conventions used to not immediately know that this function performed a conditional move of a field element, or of programming enough to not know what a conditional move was; there is documentation (in this case apparently added by me):

/** If flag is true, set *r equal to *a; otherwise leave it. Constant-time. */
static void secp256k1_fe_cmov(secp256k1_fe *r, const secp256k1_fe *a, int flag);

... even though this is purely internal code and is not accessible to an end user of the library.
11  Bitcoin / Development & Technical Discussion / Re: Increasing blocksize dynamically w/economic safeguards - the ideal compromise? on: January 11, 2017, 01:40:00 AM
out of band fees can work in both directions, e.g. including rebates.  (and, in fact rebates can be done inband with coinjoins with no trust)

Also consider what your scheme does when a majority hashpower censors any transaction paying a high inband fee level.
12  Bitcoin / Development & Technical Discussion / Re: Post your SegWit questions here - open discussion - big week for Bitcoin! on: January 09, 2017, 12:16:49 PM
Were there not some Compact Blocks fixes in 13.1? Miners not signalling Segwit could make use of those
No, just segwit support for compact blocks; and more compact block tests.
13  Bitcoin / Development & Technical Discussion / Re: Increasing blocksize dynamically w/economic safeguards - the ideal compromise? on: January 09, 2017, 12:14:38 PM
101 = vote increase 1.35%, pay 10% of transaction fees to next block
111 = vote increase 2.7%, pay 25% of transaction fees to next block
There are no mandated fees in the Bitcoin protocol so the natural response to schemes like this is for miners to simply accept fees via other means (such as the direct txout method supported by eligius since 2011, or via outputs with empty scriptpubkeys, or via out of band fees) and give users a discount for using them. The expected result would be fees migrating out of the fee area, and protocols that depend on them (like your suggestion) becoming dysfunctional.  Sad

I previously tried to rescue this class of proposal by having the  change not be to fees but by modifying the lowness of the required hash (effective difficulty), but it's difficult to do that in the presence of subsidy.

Unrelated, as you note your proposal is no constraint if miners agree-- this is also why it fails to address the conflict of interest between miners (really mining pools), who are paid to include transactions, and everyone else-- who experiences them as an externalize except to the extent that they contribute to economic growth (not at all a necessity: e.g. many companies want to use the bitcoin blockchain without using the Bitcoin currency at all).  Still, better to solve one issue even if all can't be solved.

14  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 09, 2017, 11:50:01 AM
I am also interested in a faster secp256k1.

Unfortunately, it seems Pieter Wuille et al. were neither serious about providing a fast secp256k1 nor about documenting it well.  Roll Eyes

Can you show another library for the same application within a factor of _five_ of the performance?  Or with more than 1/5th the documentation?

I'm just ... beside myself at your comment, it's not even insulting: it's just too absurd.

- which alone made the LBC generator about 10% faster overall -
and later also changing the field_5x52 code in secp256k1_fe_set_b32 to
Thanks,  that function is not performance critical in anything the library was intended for ... e.g. not brainwallet cracking. But making it faster would be fine.

You might have noticed that the constant unpacking macro does effectively the same thing: but a word at a time.

If you open a PR on the function you should add it to the bench_internal benchmarks as you go.

Is there any place where R&D discussion about further development is ongoing? Before I'd start re-implementing that mess from scratch I'd prefer to participate in some "official endeavor".

same place it's always been

However, I have little hope that will make sense:
Signature verification isn't really the limiting factor in Bitcoin Core performance anymore in any case.
... just because something isn't a limiting factor isn't a reason to improve it.

Together with other statements from gmaxwell @ github ("this is alpha/, don't expect other doc shan source.." - something like that from memory) I see there seems not much motivation in further development from the "official side".
Disinterest in spoon feeding people who sound like wanna-be thieves who thefts would scare ordinary people away from Bitcoin and whom can't even be bothered to RTFM (there has been fairly detailed documentation for the library for years) is not evidence of a lack of interest in further development.

Any optimizations are very welcome - I'm sure sipa (the original author) would agree. He's a very nice guy and it's easy to contact him directly (probably best via IRC). Although, about the code used by the core,  he'll probably tell you that it's being maintained by other people now.
Uh, what?

In 5x52 asm I found 2-3% by simply doing what the cpu scheduler should be doing and issuing together the adds + muls (the cpu integer unit typically has one add and one mul unit and ideally we want to be using them at the same time).
Awesome! PRs welcome!

One question I have regarding secp256k1 is whether endomorphism is safe, and if yes, shouldn't it be enabled in bitcoin builds if it's faster (benchmarks show that it is)?
It is potentially patent encumbered for another couple years, restricting it to experimental use.

So the adcq+mulq are issued together to the mul and add units of the cpu respectively. I'm baffled on why the cpu scheduler wasn't already doing this but then again I do have an older cpu (core2 quad / 45nm) to play with - it might not be an issue with modern ones.
Generally newer cpus work better, but the performance is also more important on older (and in-order cpus like the atoms) since they're slower to begin with.
15  Bitcoin / Development & Technical Discussion / Re: Post your SegWit questions here - open discussion - big week for Bitcoin! on: January 09, 2017, 11:27:40 AM
0.13.1 was basically only segwit.
16  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 06, 2017, 11:36:11 AM
I thought signature verification was the biggest limiting factor in IBD process (obviously with slow CPU). 
On arm, which doesn't yet use the libsecp256k1 assembly optimizations by default-- perhaps.  Otherwise, no, not since libsecp256k1 made signature validation many times faster. Maybe on some really unbalanced system with a slow single core cpu and huge dbcache and fast network and SSD it might be a majority.
17  Bitcoin / Development & Technical Discussion / Re: secp256k1 library and Intel cpu on: January 06, 2017, 09:41:44 AM
The content in your first link would likely not be helpful.

The content in your second link-- the  would be beneficial, though potentially not by much because the dependency chain in the arithmetic is hand constructed to reduce conflicts.  If someone would like to try them out, it shouldn't be very hard.

In the context of Bitcoin development (thus achows' response), Signature verification isn't really the limiting factor in Bitcoin Core performance anymore in any case.
18  Bitcoin / Development & Technical Discussion / Re: Brain wallet, step-by-step guide (FIXED!)[Mod note: DO NOT USE BRAINWALLETS] on: January 04, 2017, 11:23:33 PM
Is this really the case? and how do you "correctly use" Electrum seeds? because you made a "if used correctly" remark.
Coming up with the string on your own rather than having the software do it or storing it only in your memory.
19  Bitcoin / Development & Technical Discussion / Re: Post your SegWit questions here - open discussion - big week for Bitcoin! on: January 04, 2017, 11:22:13 PM
If we aren't continually filling blocks then that is a disaster.
20  Bitcoin / Development & Technical Discussion / Re: Brain wallet, step-by-step guide (FIXED!)[Mod note: DO NOT USE BRAINWALLETS] on: January 03, 2017, 07:27:01 PM
question the technical aspects of what I'm saying, instead of trying to undermine my motives.  It's just pathetic, man. How old are you?

Because the mod is a type of person that prefers to run a forum for kids who he can impress and patronise all the time.


By this logic nobody should trust your expertise on cryptography because you know too much about the topic and your advice might be luring unconscious  people into using solutions that you claim are secured,  but personally know how to break.

How are you going to answer that?

Ask me again if you ever see me advocating solutions which are have resulted in lots of funds loss in practice... or selling wallet cracking tools.
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 ... 232 »
Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!