Come-from-Beyond (OP)
Legendary
 Offline
Activity: 2142
Merit: 1010
Newbie
|
 |
November 29, 2013, 08:01:53 PM |
|
I wrote this a couple of times but noone seems to paid attention.
BCNext offered to reveal any part of the code for review. He doesn't want to publish the complete code till the 3rd of January to protect Nxt against copycats.
So, if there are Java programmers who would like to view (and possibly improve) the source code, please, post ur requests here.
|
|
|
|
|
|
bybitcoin
|
|
November 29, 2013, 08:09:52 PM |
|
I wrote this a couple of times but noone seems to paid attention.
BCNext offered to reveal any part of the code for review. He doesn't want to publish the complete code till the 3rd of January to protect Nxt against copycats.
So, if there are Java programmers who would like to view (and possibly improve) the source code, please, post ur requests here.
Good clever move! Perhaps you can post it on the announcement board too, for getting wider attention. |
|
|
|
|
arafel71
Newbie
Offline
Activity: 51
Merit: 0
|
|
November 29, 2013, 08:17:26 PM |
|
Hi come-from-beyond
That's interesting,
can you ''package'' and share an Eclipse workspace with somes java source ? we don't need all source from now because we need to protect nxt coin but some java source can help us for understanding algorithm.
A Java J2EE project director.
|
|
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
November 29, 2013, 08:20:07 PM |
|
Hi come-from-beyond
That's interesting,
can you ''package'' and share an Eclipse workspace with somes java source ? we don't need all source from now because we need to protect nxt coin but some java source can help us for understanding algorithm.
A Java J2EE project director.
I can't do it. Post here what part (mining algo, crypto, peer connection, etc.) u need and I'll post it here after I get it from BCNext. |
|
|
|
|
arafel71
Newbie
Offline
Activity: 51
Merit: 0
|
|
November 29, 2013, 08:27:42 PM |
|
I see many question on forum about mining process, maybe you can post some part of this code if BCnext allow it.
I let other java developper ask other part (crypto, etc...) ,
many thanks for help us see ''threw the fog'' .
|
|
|
|
|
|
Jean-Luc
|
|
November 29, 2013, 10:34:59 PM |
|
Post here what part (mining algo, crypto, peer connection, etc.) u need and I'll post it here after I get it from BCNext.
The part that seems to need most work at the moment is the networking and the loading of the block chain, which is still not stable, isn't it? It is also probably not Nxt specific, so disclosing it is not likely to help someone clone Nxt. I could try to help debugging it. |
|
|
|
|
kwest
|
|
November 29, 2013, 10:54:35 PM |
|
I really like this move. A little community help never hurt.
|
|
|
|
|
|
bob131313
|
|
November 30, 2013, 03:16:25 AM |
|
I would like to review the peer connection code.
Thanks
|
|
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
November 30, 2013, 07:33:29 AM |
|
I see many question on forum about mining process, maybe you can post some part of this code if BCnext allow it.
Part of main() method: (new Thread() {
@Override
public void run() {
HashMap<Account, Block> lastBlocks = new HashMap<>();
HashMap<Account, BigInteger> hits = new HashMap<>();
while (state == 0) {
try {
if (bootstrappingState >= 0) {
HashMap<Account, User> unlockedAccounts = new HashMap<>();
for (User user : users.values()) {
if (user.secretPhrase != null) {
Account account = accounts.get(Account.getId(Crypto.getPublicKey(user.secretPhrase)));
if (account != null && account.getEffectiveBalance() > 0) {
unlockedAccounts.put(account, user);
}
}
}
for (Map.Entry<Account, User> unlockedAccountEntry : unlockedAccounts.entrySet()) {
Account account = unlockedAccountEntry.getKey();
User user = unlockedAccountEntry.getValue();
Block lastBlock = Block.getLastBlock();
if (lastBlocks.get(account) != lastBlock) {
byte[] generationSignature = Crypto.sign(lastBlock.generationSignature, user.secretPhrase);
byte[] generationSignatureHash = MessageDigest.getInstance("SHA-256").digest(generationSignature);
BigInteger hit = new BigInteger(1, new byte[] {generationSignatureHash[7], generationSignatureHash[6], generationSignatureHash[5], generationSignatureHash[4], generationSignatureHash[3], generationSignatureHash[2], generationSignatureHash[1], generationSignatureHash[0]});
lastBlocks.put(account, lastBlock);
hits.put(account, hit);
JSONObject response = new JSONObject();
response.put("response", "setBlockGenerationDeadline");
response.put("deadline", hit.divide(BigInteger.valueOf(Block.getBaseTarget()).multiply(BigInteger.valueOf(account.getEffectiveBalance()))).longValue() - (getEpochTime(System.currentTimeMillis()) - lastBlock.timestamp));
user.send(response);
}
int elapsedTime = getEpochTime(System.currentTimeMillis()) - lastBlock.timestamp;
if (elapsedTime > 0) {
BigInteger target = BigInteger.valueOf(Block.getBaseTarget()).multiply(BigInteger.valueOf(account.getEffectiveBalance())).multiply(BigInteger.valueOf(elapsedTime));
if (hits.get(account).compareTo(target) < 0) {
account.generateBlock(user.secretPhrase);
}
}
}
}
Thread.sleep(1000);
} catch (Exception e) {
log("9: " + e.toString());
}
}
}
}).start();
Part of Account class: int getEffectiveBalance() {
if (height == 0) {
return (int)(balance / 100);
}
if (Block.getLastBlock().height - height < 1440) {
return 0;
}
int amount = 0;
for (long transactionId : Block.getLastBlock().transactions) {
Transaction transaction = transactions.get(transactionId);
if (transaction.recipient == id) {
amount += transaction.amount;
}
}
return (int)(balance / 100) - amount;
}
Part of Account class: void generateBlock(String secretPhrase) throws Exception {
HashMap<Long, Transaction> newTransactions;
synchronized (transactions) {
Transaction[] sortedTransactions = unconfirmedTransactions.values().toArray(new Transaction[0]);
while (sortedTransactions.length > 0) {
int i;
for (i = 0; i < sortedTransactions.length; i++) {
Transaction transaction = sortedTransactions[i];
if (transaction.referencedTransaction != 0 && transactions.get(transaction.referencedTransaction) == null) {
sortedTransactions[i] = sortedTransactions[sortedTransactions.length - 1];
Transaction[] tmp = new Transaction[sortedTransactions.length - 1];
System.arraycopy(sortedTransactions, 0, tmp, 0, tmp.length);
sortedTransactions = tmp;
break;
}
}
if (i == sortedTransactions.length) {
break;
}
}
Arrays.sort(sortedTransactions);
newTransactions = new HashMap<>();
for (int i = 0; i < 255 && i < sortedTransactions.length; i++) {
long id = sortedTransactions[i].getId();
newTransactions.put(id, sortedTransactions[i]);
}
}
Block block = new Block(1, getEpochTime(System.currentTimeMillis()), lastBlock, newTransactions.size(), 0, 0, 0, null, Crypto.getPublicKey(secretPhrase), null, new byte[64]);
block.transactions = new long[block.numberOfTransactions];
int i = 0;
for (Map.Entry<Long, Transaction> transactionEntry : newTransactions.entrySet()) {
Transaction transaction = transactionEntry.getValue();
if (transaction.type == Transaction.TYPE_PAYMENT && transaction.subtype == Transaction.SUBTYPE_PAYMENT_ORDINARY_PAYMENT) {
block.totalAmount += transaction.amount;
}
block.totalFee += transaction.fee;
block.payloadLength += transaction.getBytes().length;
block.transactions[i++] = transactionEntry.getKey();
}
Arrays.sort(block.transactions);
MessageDigest digest = MessageDigest.getInstance("SHA-256");
for (i = 0; i < block.numberOfTransactions; i++) {
digest.update(newTransactions.get(block.transactions[i]).getBytes());
}
block.payloadHash = digest.digest();
block.generationSignature = Crypto.sign(Block.getLastBlock().generationSignature, secretPhrase);
byte[] data = block.getBytes();
byte[] data2 = new byte[data.length - 64];
System.arraycopy(data, 0, data2, 0, data2.length);
block.blockSignature = Crypto.sign(data2, secretPhrase);
ByteBuffer outputBuffer = Peer.getOutputBuffer(224 + block.payloadLength, Peer.PACKET_TYPE_NEW_BLOCK_REQUEST);
outputBuffer.put(block.getBytes());
for (i = 0; i < block.numberOfTransactions; i++) {
Transaction transaction = newTransactions.get(block.transactions[i]);
outputBuffer.put(transaction.getBytes());
}
Peer.sendToAllPeers(outputBuffer);
}
Part of Transaction class (used by Arrays.sort()): public int compareTo(Transaction o) {
if (height < o.height) {
return -1;
} else if (height > o.height) {
return 1;
} else {
if (fee * 32768L / getBytes().length > o.fee * 32768L / o.getBytes().length) {
return -1;
} else if (fee * 32768L / getBytes().length < o.fee * 32768L / o.getBytes().length) {
return 1;
} else {
if (timestamp < o.timestamp) {
return -1;
} else if (timestamp > o.timestamp) {
return 1;
} else {
if (index < o.index) {
return -1;
} else if (index > o.index) {
return 1;
} else {
return 0;
}
}
}
}
}
|
|
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
November 30, 2013, 07:41:59 AM |
|
The part that seems to need most work at the moment is the networking and the loading of the block chain, which is still not stable, isn't it? It is also probably not Nxt specific, so disclosing it is not likely to help someone clone Nxt. I could try to help debugging it.
Networking is being fixed now, it will be published later. Loading the blockchainPart of main() method: if (bootstrappingState >= 0) {
peer = Peer.getAnyConnectedPeer();
if (peer != null) {
setBootstrappingState(0);
ByteBuffer outputBuffer = Peer.getOutputBuffer(0, Peer.PACKET_TYPE_CUMULATIVE_DIFFICULTY_REQUEST);
peer.send(outputBuffer);
}
}
Part of Peer class: case PACKET_TYPE_CUMULATIVE_DIFFICULTY_REQUEST:
{
byte[] cumulativeDifficulty = Block.getLastBlock().cumulativeDifficulty.toByteArray();
ByteBuffer outputBuffer = getOutputBuffer(2 + cumulativeDifficulty.length, PACKET_TYPE_CUMULATIVE_DIFFICULTY_RESPONSE);
outputBuffer.putShort((short)cumulativeDifficulty.length);
outputBuffer.put(cumulativeDifficulty);
send(outputBuffer);
}
break;
case PACKET_TYPE_CUMULATIVE_DIFFICULTY_RESPONSE:
{
if (bootstrappingState == 0) {
int cumulativeDifficultyLength = inputBuffer.getShort();
byte[] cumulativeDifficulty = new byte[cumulativeDifficultyLength];
inputBuffer.get(cumulativeDifficulty);
betterCumulativeDifficulty = new BigInteger(cumulativeDifficulty);
synchronized (blocks) {
if (betterCumulativeDifficulty.compareTo(Block.getLastBlock().cumulativeDifficulty) > 0) {
setBootstrappingState(-1);
bootstrappingPeer = this;
ByteBuffer outputBuffer = Peer.getOutputBuffer(0, PACKET_TYPE_MILESTONE_BLOCKS_REQUEST);
send(outputBuffer);
} else {
setBootstrappingState(1);
}
}
}
}
break;
case PACKET_TYPE_MILESTONE_BLOCKS_REQUEST:
{
try {
LinkedList<Long> milestoneBlocks = new LinkedList<>();
Block block = Block.getLastBlock();
int jumpLength = block.height / 4093 + 1;
while (block.height > 0) {
milestoneBlocks.add(block.getId());
for (int i = 0; i < jumpLength && block.height > 0; i++) {
block = blocks.get(block.previousBlock);
}
}
ByteBuffer outputBuffer = Peer.getOutputBuffer(2 + milestoneBlocks.size() * 8, PACKET_TYPE_MILESTONE_BLOCKS_RESPONSE);
outputBuffer.putShort((short)milestoneBlocks.size());
for (int i = 0; i < milestoneBlocks.size(); i++) {
outputBuffer.putLong(milestoneBlocks.get(i));
}
send(outputBuffer);
} catch (Exception e) {
log("12: " + e.toString());
}
}
break;
case PACKET_TYPE_MILESTONE_BLOCKS_RESPONSE:
{
if (bootstrappingState < 0) {
Block commonBlock = blocks.get(genesisBlock);
int numberOfBlocks = inputBuffer.getShort();
while (numberOfBlocks-- > 0) {
long blockId = inputBuffer.getLong();
Block block = blocks.get(blockId);
if (block != null) {
commonBlock = block;
break;
}
}
try {
ByteBuffer outputBuffer = Peer.getOutputBuffer(8, PACKET_TYPE_NEXT_BLOCK_IDS_REQUEST);
lastCommonBlock = commonBlock.getId();
outputBuffer.putLong(lastCommonBlock);
send(outputBuffer);
} catch (Exception e) {
log("13: " + e.toString());
}
}
}
break;
case PACKET_TYPE_NEXT_BLOCK_IDS_REQUEST:
{
try {
LinkedList<Long> nextBlocks = new LinkedList<>();
Block block = blocks.get(inputBuffer.getLong());
while (block != null && nextBlocks.size() < 4093) {
block = blocks.get(block.nextBlock);
if (block != null) {
nextBlocks.add(block.getId());
}
}
ByteBuffer outputBuffer = Peer.getOutputBuffer(2 + nextBlocks.size() * 8, PACKET_TYPE_NEXT_BLOCK_IDS_RESPONSE);
outputBuffer.putShort((short)nextBlocks.size());
for (int i = 0; i < nextBlocks.size(); i++) {
outputBuffer.putLong(nextBlocks.get(i));
}
send(outputBuffer);
} catch (Exception e) {
log("14: " + e.toString());
}
}
break;
case PACKET_TYPE_NEXT_BLOCK_IDS_RESPONSE:
{
if (bootstrappingState < 0) {
int numberOfBlocks = inputBuffer.getShort();
if (numberOfBlocks == 0) {
setBootstrappingState(1);
break;
}
long blockId;
while (numberOfBlocks-- > 0) {
blockId = inputBuffer.getLong();
if (blocks.get(blockId) == null) {
break;
}
lastCommonBlock = blockId;
}
if (numberOfBlocks < 0) {
ByteBuffer outputBuffer = Peer.getOutputBuffer(8, PACKET_TYPE_NEXT_BLOCK_IDS_REQUEST);
outputBuffer.putLong(lastCommonBlock);
send(outputBuffer);
} else {
if (Block.getLastBlock().height - blocks.get(lastCommonBlock).height > 720) {
setBootstrappingState(1);
} else {
while (lastBlock != lastCommonBlock && Block.popLastBlock()) { }
if (lastBlock == lastCommonBlock) {
ByteBuffer outputBuffer = Peer.getOutputBuffer(8, PACKET_TYPE_NEXT_BLOCKS_REQUEST);
outputBuffer.putLong(lastBlock);
send(outputBuffer);
} else {
setBootstrappingState(1);
}
}
}
}
}
break;
case PACKET_TYPE_NEXT_BLOCKS_REQUEST:
{
LinkedList<Block> nextBlocks = new LinkedList<>();
int totalLength = 0;
Block block = blocks.get(inputBuffer.getLong());
while (block != null) {
block = blocks.get(block.nextBlock);
if (block != null) {
int length = 224 + block.payloadLength;
if (totalLength + length > 32768 - (8 + 2)) {
break;
}
nextBlocks.add(block);
totalLength += length;
}
}
ByteBuffer outputBuffer = Peer.getOutputBuffer(2 + totalLength, PACKET_TYPE_NEXT_BLOCKS_RESPONSE);
outputBuffer.putShort((short)nextBlocks.size());
for (int i = 0; i < nextBlocks.size(); i++) {
block = nextBlocks.get(i);
outputBuffer.put(block.getBytes());
for (int j = 0; j < block.numberOfTransactions; j++) {
outputBuffer.put(transactions.get(block.transactions[j]).getBytes());
}
}
send(outputBuffer);
}
break;
case PACKET_TYPE_NEXT_BLOCKS_RESPONSE:
{
if (bootstrappingState < 0) {
int numberOfBlocks = inputBuffer.getShort();
if (numberOfBlocks == 0) {
setBootstrappingState(1);
break;
}
while (numberOfBlocks-- > 0) {
if (!Block.pushBlock(inputBuffer)) {
break;
}
}
ByteBuffer outputBuffer = Peer.getOutputBuffer(8, PACKET_TYPE_NEXT_BLOCKS_REQUEST);
outputBuffer.putLong(lastBlock);
send(outputBuffer);
}
}
break;
|
|
|
|
|
|
|
arafel71
Newbie
Offline
Activity: 51
Merit: 0
|
|
November 30, 2013, 07:49:16 AM |
|
I see many question on forum about mining process, maybe you can post some part of this code if BCnext allow it.
thanks |
|
|
|
|
inkadnb
Member
Offline
Activity: 126
Merit: 10
|
|
December 01, 2013, 10:15:09 AM |
|
Where is the block and transaction verification done? This is the part I'm most interested in. For example, I could, in theory have a few NXT coins, and simply by changing my clock ahead a several days, could I not in theory generate a new block? As per, int elapsedTime = getEpochTime(System.currentTimeMillis()) - lastBlock.timestamp;
if (elapsedTime > 0) {
BigInteger target = BigInteger.valueOf(Block.getBaseTarget()).multiply(BigInteger.valueOf(account.getEffectiveBalance())).multiply(BigInteger.valueOf(elapsedTime));
if (hits.get(account).compareTo(target) < 0) {
account.generateBlock(user.secretPhrase);
}
}
you need at least one coin, but the entire value of the base target and balance is multiplied by the difference between the current time (in milliseconds) and the lastBlock. How is this enforced on other clients? How do they know that the time I'm generating is in fact valid? I realize there's time synchronization/rejection going on, but I'd like some clarification. Is the time verification done separately from transaction and block verification? |
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
December 01, 2013, 10:28:32 AM |
|
Where is the block and transaction verification done?
Asked for that code. For example, I could, in theory have a few NXT coins, and simply by changing my clock ahead a several days, could I not in theory generate a new block?
All blocks with timestamps in the future r ignored. |
|
|
|
|
inkadnb
Member
Offline
Activity: 126
Merit: 10
|
|
December 01, 2013, 10:53:04 AM |
|
Where is the block and transaction verification done?
Asked for that code. For example, I could, in theory have a few NXT coins, and simply by changing my clock ahead a several days, could I not in theory generate a new block?
All blocks with timestamps in the future r ignored. Right, I was implying that you set the time ahead enough so that generateBlock() is called, however, once in this method, revert the time back so when
Block block = new Block(1, getEpochTime(System.currentTimeMillis()), lastBlock, newTransactions.size(), 0, 0, 0, null, Crypto.getPublicKey(secretPhrase), null, new byte[64]);
is called, the system time is correct. This falsely creates a "valid" block. This would be trivial to do in any modern debugger. |
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
December 01, 2013, 10:58:17 AM |
|
Right, I was implying that you set the time ahead enough so that generateBlock() is called, however, once in this method, revert the time back so when
Block block = new Block(1, getEpochTime(System.currentTimeMillis()), lastBlock, newTransactions.size(), 0, 0, 0, null, Crypto.getPublicKey(secretPhrase), null, new byte[64]);
is called, the system time is correct. This falsely creates a "valid" block. This would be trivial to do in any modern debugger. Such a block won't pass validation. |
|
|
|
|
|
SlyWax
|
|
December 02, 2013, 01:46:10 AM |
|
There is not a single comment in the code,
this is very bad for the future ...
|
|
|
|
|
|
extortion
|
|
December 02, 2013, 04:11:26 AM |
|
non existent cryptocurrencies, because hey, why not?
|
Extortion. We are Anonymous. We are legion. We do not forgive. We Do Not Forget. Expect Us.
|
|
|
Come-from-Beyond (OP)
Legendary
Offline
Activity: 2142
Merit: 1010
Newbie
|
|
December 02, 2013, 07:54:02 AM |
|
There is not a single comment in the code,
this is very bad for the future ...
Hardcore programmers don't bother with commenting.  |
|
|
|
|
klee
Legendary
Offline
Activity: 1498
Merit: 1000
|
|
December 02, 2013, 08:44:41 AM |
|
There is not a single comment in the code,
this is very bad for the future ...
Hardcore programmers don't bother with commenting. LOL |
|
|
|
|
|
