NXT :: descendant of Bitcoin - Updated Information

[ChuckOne]

Not, if the runtime for calculating a sig is dependent on the number of bits of the payload.

It is a common misconception that basic operations are in O(1), but that's wrong. Even addition and multiplication of larger numbers take longer than of smaller numbers. The same hodls for sig function.

I like the conceptual idea of having set-based transactions because set-based thinking is one of the principle of modern computer science.

What I was trying to explain to you was that you should not make not unproven statements.
Show me the runtime complexity of the sig function in terms of payload length and we'll see.

NRS signs SHA256 of a message.

1 signature takes 100 times more CPU ticks than SHA256(32_random_bytes). Signing of a transaction that is 100 times longer ~ 2 ordinary signatures.

Not sure, I got it.

The runtime complexity is still O(n)? But the factory in between is 100? (n = number of bits)

[ChuckOne]

Question: Is it possible to re-use the nxt_db directory when upgrading from 0.7.4 to 0.7.5  without downloading the whole blockchain again?

Yes, it is designed to be possible to upgrade without having to delete the nxt_db directory, that's the reason I bother with applying those SQL statements.

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

What exactly do you mean by performance?

1) startup is slow as always. Especially these migrations

2) running is as good as always. Does not go above 5%.

3) RAM consumption is 1% lower. Still 87% of total memory.

yes, which is why having the option of using the nxt_db is quite helpful

Why is that? Or maybe, I didn't understand what you mean.

[l8orre]

Question: Is it possible to re-use the nxt_db directory when upgrading from 0.7.4 to 0.7.5  without downloading the whole blockchain again?

Yes, it is designed to be possible to upgrade without having to delete the nxt_db directory, that's the reason I bother with applying those SQL statements.

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

What exactly do you mean by performance?

1) startup is slow as always. Especially these migrations

2) running is as good as always. Does not go above 5%.

3) RAM consumption is 1% lower.

yes, which is why having the option of using the nxt_db is quite helpful

Why is that? Or maybe, I didn't understand what you mean.

when you update, you just dump the old nxt_db directory into the nxt_db in the unzip directory of the new version, and the blockchain won't be downloaded -just as with before, when you could just dump the blocks.nrs and transactions.nrs into the new version.
only then it would crash very often, and now with sql it hopefully won't.

At least that's what I understand from it.

[ChuckOne]

when you update, you just dump the old nxt_db directory into the nxt_db in the unzip directory of the new version, and the blockchain won't be downloaded -just as with before, when you could just dump the blocks.nrs and transactions.nrs into the new version.
only then it would crash very often, and now with sql it hopefully won't.

At least that's what I understand from it.

Well, I see. I don't think we were talking about the 'manual performance of upgrading' but the runtime performance, though.

But, you are right. That's the idea of having a database which is able to handle transactions, rollbacks, caching, indexing etc. at once. Much better than inventing your own serialization format. +1

[mcjavar]

What does this mean?


[2014-02-14 10:39:40.346] DEBUG: Will pop block 5073332198957728923 at height 66148
[2014-02-14 10:39:40.347] DEBUG: Will pop block 8348856438008341233 at height 66147
[2014-02-14 10:39:40.348] DEBUG: Reversal of alias assignment not supported
[2014-02-14 10:39:40.348] DEBUG: Popping off last block not possible, will do a rescan
[2014-02-14 10:39:40.349] Re-scanning blockchain...
[2014-02-14 10:39:54.734] ...Done
[2014-02-14 10:39:54.737] DEBUG: Generate block failed: Previous block id doesn't match

This is all normal, works as intended. It is telling you exactly what is happening - popping off a block, doing a rescan because alias assignment cannot be simply undone in a pop-off. The block generation attempt failed because in the meantime another block arrived and was accepted.

Alias assignment being undone? I didn´t touch the client when this message came.

[Come-from-Beyond]

Not, if the runtime for calculating a sig is dependent on the number of bits of the payload.

It is a common misconception that basic operations are in O(1), but that's wrong. Even addition and multiplication of larger numbers take longer than of smaller numbers. The same hodls for sig function.

I like the conceptual idea of having set-based transactions because set-based thinking is one of the principle of modern computer science.

What I was trying to explain to you was that you should not make not unproven statements.
Show me the runtime complexity of the sig function in terms of payload length and we'll see.

NRS signs SHA256 of a message.

1 signature takes 100 times more CPU ticks than SHA256(32_random_bytes). Signing of a transaction that is 100 times longer ~ 2 ordinary signatures.

Not sure, I got it.

The runtime complexity is still O(n)? But the factory in between is 100? (n = number of bits)

Complexity of signing is O(1).

[jl777]

Question: Is it possible to re-use the nxt_db directory when upgrading from 0.7.4 to 0.7.5  without downloading the whole blockchain again?

Yes, it is designed to be possible to upgrade without having to delete the nxt_db directory, that's the reason I bother with applying those SQL statements.

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

Does this mean that we are actually generating checkpoint files on all the nodes all the time? If so, how much work would it be to have newly installed nodes query for the most recent db files and start with that, instead of redownloading the entire blockchain. Of course, must be peer verified checksums, etc.

How fast does the DB grow at 1000TPS?

[ChuckOne]

Not, if the runtime for calculating a sig is dependent on the number of bits of the payload.

It is a common misconception that basic operations are in O(1), but that's wrong. Even addition and multiplication of larger numbers take longer than of smaller numbers. The same hodls for sig function.

I like the conceptual idea of having set-based transactions because set-based thinking is one of the principle of modern computer science.

What I was trying to explain to you was that you should not make not unproven statements.
Show me the runtime complexity of the sig function in terms of payload length and we'll see.

NRS signs SHA256 of a message.

1 signature takes 100 times more CPU ticks than SHA256(32_random_bytes). Signing of a transaction that is 100 times longer ~ 2 ordinary signatures.

Not sure, I got it.

The runtime complexity is still O(n)? But the factory in between is 100? (n = number of bits)

Complexity of signing is O(1).

That is plain wrong. The more bits you have to sign the longer signing takes. The question is: how long in terms of the input length.

[ChuckOne]

Question: Is it possible to re-use the nxt_db directory when upgrading from 0.7.4 to 0.7.5  without downloading the whole blockchain again?

Yes, it is designed to be possible to upgrade without having to delete the nxt_db directory, that's the reason I bother with applying those SQL statements.

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

Does this mean that we are actually generating checkpoint files on all the nodes all the time? If so, how much work would it be to have newly installed nodes query for the most recent db files and start with that, instead of redownloading the entire blockchain. Of course, must be peer verified checksums, etc.

How fast does the DB grow at 1000TPS?

du says: 257480 for current nxt_db folder

[Jean-Luc]

Alias assignment being undone? I didn´t touch the client when this message came.

It is not because of something you did. The block that had to be popped off contained an alias assignment transaction. When a block is popped off, an attempt is made to undo all transactions in it. But this type of transaction cannot be undone, and in this case a full rescan of the blockchain is needed.

[Jean-Luc]

du says: 257480 for current nxt_db folder

What? Mine is 111M only.

[ChuckOne]

Alias assignment being undone? I didn´t touch the client when this message came.

It is not because of something you did. The block that had to be popped off contained an alias assignment transaction. When a block is popped off, an attempt is made to undo all transactions in it. But this type of transaction cannot be undone, and in this case a full rescan of the blockchain is needed.

Interesting. Why can't this be undone?

[Jean-Luc]

Does this mean that we are actually generating checkpoint files on all the nodes all the time? If so, how much work would it be to have newly installed nodes query for the most recent db files and start with that, instead of redownloading the entire blockchain. Of course, must be peer verified checksums, etc.

I don't understand what you mean by checkpoint file. The rest is a bad idea.

[ChuckOne]

du says: 257480 for current nxt_db folder

What? Mine is 111M only.

Err:

Code:

pi@raspberrypi ~/nxt $ du -h nxt_db/
252M	nxt_db/

Wrong usage of du?

[Jean-Luc]

Interesting. Why can't this be undone?

Because to find out the previous value of the alias URL, if any, you need to go back through all previous transactions. In effect, again scanning the blockchain.

[mikesbmw]

Running 0.7.5:

I get a non-stop list of this:

height 66516 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:24.534] DEBUG: Failed to accept block 11979910396361888590 at
height 66516 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:26.139] DEBUG: Failed to accept block 11979910396361888590 at
height 66516 received from node3.mynxtcoin.org, blacklisting
[2014-02-14 22:01:27.190] DEBUG: Failed to accept block 11979910396361888590 at
height 66516 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:28.239] DEBUG: Failed to accept block 11979910396361888590 at
height 66516 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:40.314] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from node3.mynxtcoin.org, blacklisting
[2014-02-14 22:01:41.923] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from node3.mynxtcoin.org, blacklisting
[2014-02-14 22:01:42.979] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:44.029] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:45.634] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from node3.mynxtcoin.org, blacklisting
[2014-02-14 22:01:46.697] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:47.748] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:48.810] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:50.821] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from node3.mynxtcoin.org, blacklisting
[2014-02-14 22:01:51.923] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from vps2.nxtcrypto.org, blacklisting
[2014-02-14 22:01:53.745] DEBUG: Failed to accept block 12715248329747037589 at
height 66517 received from node90.nxtbase.com, blacklisting


EDIT: okay, seems to have stopped now after several hundred lines of this.

[ChuckOne]

Interesting. Why can't this be undone?

Because to find out the previous value of the alias URL, if any, you need to go back through all previous transactions. In effect, again scanning the blockchain.

Why not caching it?

[greyw00lf]

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

copying the db from the old version is definitely faster, because my Raspberry takes a long time to download the whole blockchain (maybe 1h? haven't really watched at the time precisely).

Scanning the blockchain now takes about 9 minutes on my Raspberry.
No issues at the moment with my Pi even unlocking my account via the GUI is fast now! Thanks Jean-Luc!

[ChuckOne]

If too slow for Raspberries, do whichever is faster, upgrade or download from scratch.

Did I manage to improve the Raspberry performance with the last release?

copying the db from the old version is definitely faster, because my Raspberry takes a long time to download the whole blockchain (maybe 1h? haven't really watched at the time precisely).

Scanning the blockchain now takes about 9 minutes on my Raspberry.
No issues at the moment with my Pi even unlocking my account via the GUI is fast now! Thanks Jean-Luc!

Have you overclocked your Raspi?

[Come-from-Beyond]

Complexity of signing is O(1).

That is plain wrong. The more bits you have to sign the longer signing takes. The question is: how long in terms of the input length.

We would have O(n) if Sign(32_bytes) == Sign(64_bytes) / 2. Actually we have Sign(32_bytes) == Sign(3200_bytes) / 2, so it's more like O(1). Right?