Technical questions about blockchain

[redigaffi]

Hello People,

I'm developing an basic blockchain using nodeJS and TypeScript, basically to learn and understand better how a Blockchain/cryptocurrency work, this project uses PoW algorithm, And consensus catch the longest node available.

My Questions:

1. A block have it just one transaction or can it have more transactions?

2. Miners: So, just imagine I launch my coin, and there are no transactions yet, so miners have no work, if miners dont can do work, no new coins are supplied, so my questions, even if there are no transactions, can Block's without any transaction merged to the Blockchain? also "Empty Blocks" ?

3. About wallet System, I have no clue how this works, I know that public key is generated from Private key,  but how can the server know that the transaction comes from the wallet owner?

Imagine I have an endpoint in the node, /tx, there you send a json with from, to, and amount:

- With an wallet application it would be easy, you just verify public key with private and send transaction to /tx, but anyway, what would preventing users to send transaction to /tx, the verification must be done in the server, so i have to send public and private key to the server, right?

But that can lead to security issues, what happens if there's an middleman listening to your requests?

4. Storage
How do other coins store the Blockchain, Plain Files, MySQL, Or in memory? Redis? what is the normal way to store the Blockchain?

5. What would be better, treat every node as an public API that can be consumed with HTTP or use a socket communication (I think this one is the best, fastest, and securest).

6. Does every node perform mining tasks?

I Appreciate a lot your reading, and  I Hope i get some light from you!

Best Regards

[1714065842]

1714065842

[1714065842]

1714065842

[redigaffi]

Ok, about how to verify i'm the wallet owner, I've read that I have to send a signature with the transaction
 so the server can check the public key against the signature. Can I use RSA for this? I know that bitcoin uses
 RSA and ECDSA, but well I think for my basic coin RSA would be enough, is this correct or would using ony
 RSA incorrect?

[ranochigo]

1. A block have it just one transaction or can it have more transactions?

Blocks will have transactions up to the size limit (ie. 1MB for Bitcoin).

2. Miners: So, just imagine I launch my coin, and there are no transactions yet, so miners have no work, if miners dont can do work, no new coins are supplied, so my questions, even if there are no transactions, can Block's without any transaction merged to the Blockchain? also "Empty Blocks" ?

That is a misconception. Miners are not exactly mining transactions, they are basically hashing the block header until they find a hash that meets the target. The transactions are included in the merkle root of the block header. There must be at least the coinbase transaction in the block.

3. About wallet System, I have no clue how this works, I know that public key is generated from Private key,  but how can the server know that the transaction comes from the wallet owner?

Imagine I have an endpoint in the node, /tx, there you send a json with from, to, and amount:

- With an wallet application it would be easy, you just verify public key with private and send transaction to /tx, but anyway, what would preventing users to send transaction to /tx, the verification must be done in the server, so i have to send public and private key to the server, right?

But that can lead to security issues, what happens if there's an middleman listening to your requests?

Using ECDSA. In a transaction, the user provides the public key and also the signature of the transaction. Using the public key, nodes can use the public key and the signature to verify that it corresponds to the SHA256 of the TXID.

6. Does every node perform mining tasks?

No. Full nodes usually just verify transactions and blocks.

[redigaffi]

1. A block have it just one transaction or can it have more transactions?

Blocks will have transactions up to the size limit (ie. 1MB for Bitcoin).

2. Miners: So, just imagine I launch my coin, and there are no transactions yet, so miners have no work, if miners dont can do work, no new coins are supplied, so my questions, even if there are no transactions, can Block's without any transaction merged to the Blockchain? also "Empty Blocks" ?

That is a misconception. Miners are not exactly mining transactions, they are basically hashing the block header until they find a hash that meets the target. The transactions are included in the merkle root of the block header. There must be at least the coinbase transaction in the block.

3. About wallet System, I have no clue how this works, I know that public key is generated from Private key,  but how can the server know that the transaction comes from the wallet owner?

Imagine I have an endpoint in the node, /tx, there you send a json with from, to, and amount:

- With an wallet application it would be easy, you just verify public key with private and send transaction to /tx, but anyway, what would preventing users to send transaction to /tx, the verification must be done in the server, so i have to send public and private key to the server, right?

But that can lead to security issues, what happens if there's an middleman listening to your requests?

Using ECDSA. In a transaction, the user provides the public key and also the signature of the transaction. Using the public key, nodes can use the public key and the signature to verify that it corresponds to the SHA256 of the TXID.

6. Does every node perform mining tasks?

No. Full nodes usually just verify transactions and blocks.

 Hello, thanks for your answer. I don't understand that, could you explain that?, My block structure is simple and i Haven't a
 "Merkle Root" My block structure is the following:

 private index: number;
 private timestamp: number;
 private data: Data;
 private proofOfWork: number;
 private difficulty: number;
 private previousHash: string;
 private hash: string;
 
 Where data is a List of "Transactions", I don't want to replicate Bitcoin, so I think i dont need a merkle Root, I know
 that miners just do some "hard work" that in reality doesn't have relation to the block, actually my mining algorithm does the following:
 
 sha256(lastDifficulty + difficulty) and search for an amount of 0 at the end of the string.
 
 But my question is not resolved, what would happen if there aren't any transactions to "mine" would i merge Block Without transactions
 to the blockchain? Or not, but if not, how new coins can be supplied?
 
 Best Regard

[DannyHamilton]

But my question is not resolved, what would happen if there aren't any transactions to "mine" would i merge Block Without transactions
to the blockchain? Or not, but if not, how new coins can be supplied?

If you are designing a new coin, then you will need to decide how your coin will work.

If you are asking how bitcoin handles the situation...

Bitcoin requires that every block has AT LEAST 1 TRANSACTION.  There is a special transaction often called the "coinbase transaction" or the "generation transaction" which every block is required to include.  This is the transaction that pays the miner their reward.  If there is more than one transaction in the block, then it is required to be the first transaction in the list. A block without this transaction is considered to be invalid and is rejected by everyone else.

[redigaffi]

Oh thanks a lot, now i understand.

So the "reward" itself is included in the new Block, so for example:

I launch a new coin, and miners start mining, but no pending transaction are available for a loong time.

So everytime a miner start mining, new blocks are created with only miners reward, in this case how I could handle supply? with difficulty?

In my coin difficulty is measured in this way:

difficulty 2:
Means finding a proof when hashing it with last proof finds a hash with 2 0's at the end of the hash,

At the start it would be very easy to mine blocks, but every 10 Block's I increment difficulty (which means more time). Would this be correct?

And another question:
When new transactions arrives to my node, I check the transaction signature with the public key (to know if the sender of the tx is the owner of the address), then I add this transaction to a "pending transaction pool" this is a pool of valid transactions that have to be mined, would the node forward this valid pending transactions to other nodes too?

Because it can be possible that a miner is not connected to all nodes, and if the miner mine's the block the node would forward the valid block to all nodes.

And another question:
What is the storage system? Just hold it in memory or MySQL for example? or mongodb?

And another question:
Everytime someone does a transaction I have to iterate over all blocks and transactions to check if the public address has enough funds?

Sorry for my ignorance, I'm not doing this to launch a new coin, just for learning and to publish it!

Thanks!!

[ranochigo]

So everytime a miner start mining, new blocks are created with only miners reward, in this case how I could handle supply? with difficulty?

Bitcoin handles the total amount through block halving. To ensure that it is distributed over a spread period, the difficulty ensures that blocks are only generated as close as ten minutes as possible. This ensures that about 12.5BTC (right now) gets distributed per 10 minutes.

In my coin difficulty is measured in this way:

difficulty 2:
Means finding a proof when hashing it with last proof finds a hash with 2 0's at the end of the hash,

At the start it would be very easy to mine blocks, but every 10 Block's I increment difficulty (which means more time). Would this be correct?

No! It doesn't make sense at all. If the difficulty doesn't change when the block frequency gets higher or lower, the supply won't be spread out.

When new transactions arrives to my node, I check the transaction signature with the public key (to know if the sender of the tx is the owner of the address), then I add this transaction to a "pending transaction pool" this is a pool of valid transactions that have to be mined, would the node forward this valid pending transactions to other nodes too?

Because it can be possible that a miner is not connected to all nodes, and if the miner mine's the block the node would forward the valid block to all nodes.

When you can check that the transaction is valid and the inputs are in your UTXO, you will relay it to your peers.

And another question:
Everytime someone does a transaction I have to iterate over all blocks and transactions to check if the public address has enough funds?

No. Bitcoin does not work with public key or addresses. When a transaction is seen by the node, they will remove and create new unspent outputs in their UTXO set. When someone intiates a transaction, they are effectively spending an input. For that reason, nodes do not keep track of the balances in addresses.

[DannyHamilton]

Oh thanks a lot, now i understand.

So the "reward" itself is included in the new Block, so for example:

In bitcoin?  Yes, that is how it works.

I launch a new coin, and miners start mining, but no pending transaction are available for a loong time.

So everytime a miner start mining, new blocks are created with only miners reward, in this case how I could handle supply?

That's up to you.  You are designing the coin, you need to decide how you want it to work.

with difficulty?

With Bitcoin, difficulty is used to control what the average time is between solved blocks.  If you want your coin to work the same way, then you need to find a way for all nodes to agree about how fast blocks are being solved, and then to all adjust the difficulty in the same way to speed up or slow down the blocks as needed.

If you don't want your coin to work the same way as Bitcoin, then you get to decide how you want to use difficulty (if any).

In my coin difficulty is measured in this way:

difficulty 2:
Means finding a proof when hashing it with last proof finds a hash with 2 0's at the end of the hash,

Are you talking about the hash as represented in binary?  If so, that's not going to give you much control over difficulty.  For example, if you use SHA256, then you'll only be able to make a MAXIMUM of 256 adjustements, and then you'll run out of zeros.  Also, adding a zero is a DOUBLING of the difficulty.  You won't be able to adjust difficulty by 10% or 5% or anything like that.  You'll only be able to make it DOUBLE as difficult!

At the start it would be very easy to mine blocks, but every 10 Block's I increment difficulty (which means more time). Would this be correct?

So, assuming you are using SHA256 and binary representation, after 2560 blocks you will have used up all your zeros, and have the maximum possible difficulty.  Blocks will be solved once every few trillion years.  After just 720 blocks you will already have higher difficulty than Bitcoin, and after 760 blocks your coin will be more then 10 times more difficult to mine than Bitcoin.

You really might want to think about the mathematics and incentive structures behind your coin if you want it to work, otherwise it will be completely unusable.

And another question:
When new transactions arrives to my node, I check the transaction signature with the public key (to know if the sender of the tx is the owner of the address), then I add this transaction to a "pending transaction pool" this is a pool of valid transactions that have to be mined, would the node forward this valid pending transactions to other nodes too?

This is your coin.  You can make it work however you want.  If that's what you want it to do, then that is what it should do.

If you are asking about how bitcoin handles transactions:

Each node maintains a list of unspent outputs.  When it receives a transaction, it verifies that all of the transaction inputs are in its list of unspent outputs. If there are any inputs that are NOT in that list, then the transaction is invalid.  The node then executes the script formed by concatenating the Txout-script (also called the scriptPubKey) with the Txin-script (also called the scriptSig).  If that script does not execute without error, or does not have a valid result, then the transaction is invalid.

As long as the transaction is valid, then the Bitcoin node adds it to its own mempool (list of unconfirmed transactions) and shares that transaction with all the peers it is connected to which have not yet received it.

Because it can be possible that a miner is not connected to all nodes, and if the miner mine's the block the node would forward the valid block to all nodes.

This is your coin, you can design it to work however you want. If you want ALL nodes to have the blocks, then you need to find a way to share that block with everyone.  One possible method (the method that Bitcoin uses) is to forward the blocks (relay them to all connected peers that haven't received it yet)

And another question:
What is the storage system? Just hold it in memory or MySQL for example? or mongodb?

This is your coin.  You can use whatever database you want.  Bitcoin holds some data in memory structures, and other data it stores in levelDB.

And another question:
Everytime someone does a transaction I have to iterate over all blocks and transactions to check if the public address has enough funds?

This is your coin.  You can make it work however you want

If you are asking how Bitcoin does it...

Bitcoin nodes do not keep track of balances at all.  Transactions consists of two lists.  The first list is called "inputs". The "inputs list" provides value to the transaction.  It is a list of some of the outputs that are not currently seen as inputs in any other valid transactions.  The second list is called "outputs".  Each output in the "outputs list" is assigned value from the transaction.  If the sum of the value of the outputs exceeds the sum of the value of the inputs, then the transaction is invalid.  When an output is not yet used as an input it is called an "unspent" output.  When it is used as an input, it is called a "spent" output.  The node only needs to keep track of the unspent outputs.  When it adds a transaction to its memory pool, it removes all the transaction's inputs from its list of unspent outputs (since they would then be considered "Spent".  It also adds all the new outputs to it's list of unspent outputs.

[redigaffi]

So... I think i'm really confused, if I can do thinks like I want, what makes other coins a "crypto currency", so what should have/rules follow every crypto currency to be even a cryptocurrency?

Is it the Blockchain? The decentralized block structure?

What makes a cryptocurrency a cryptocurrency, i thought this was a protocol that i should to follow so my coin can be a "real" cryptocurrency.

I've read that bitcoin uses ECC for generating public address/private key. What alternatives could I use for my basic Coin?

[DannyHamilton]

So... I think i'm really confused, if I can do thinks like I want, what makes other coins a "crypto currency", so what should have/rules follow every crypto currency to be even a cryptocurrency?

Is it the Blockchain? The decentralized block structure?

What makes a cryptocurrency a cryptocurrency, i thought this was a protocol that i should to follow so my coin can be a "real" cryptocurrency.

I've read that bitcoin uses ECC for generating public address/private key. What alternatives could I use for my basic Coin?

The term "cryptocurrency" is a generic term that refers to any system that uses cryptographic functions (such as hashes, digital signatures, and/or encryption) as part of a framework that allows users reliably exchange value.

Each cryptocurrency has its own protocol.  In may cases, those protocols are created by lazy individuals that start by simply copying an existing protocol and then make minor changes without any consideration for the overall effects on their system.

There are a few have introduced innovative ideas, but it is not yet clear if those ideas are better or worse than they pre-existing systems.

Prior to Satoshi's introduction of the blockchain as a method of decentralized consensus, all cryptocurrencies needed a trusted entity to determine the ordering of transactions.  The blockchain was a significant innovation that allowed cryptocurrencies to operate in a decentralized way.

Generally a cryptocurrency would likely need a method for the user to prove that they are authorized to spend some value.  Digital signatures are a common way to accomplish this.

They also need a method to prevent double-spending and to determine transaction order.  The blockchain appears to be the best way to accomplish this if you want the system to be decentralized and trustless.  Let us know if you come up with a better way.

[redigaffi]

Hello, thanks again for the great reply.

I have another question, about miners:

What would exactly happen when 2 miners find the next Block at the same time?

I already read about orphane blocks, or miners 51% ... but i don't really understand what really is happening, some basic explanation?

When this happens, can i just catch the longest available valid BlockChain and replace it with all other nodes?

Best Regards!

[DannyHamilton]

Hello, thanks again for the great reply.

I have another question, about miners:

What would exactly happen when 2 miners find the next Block at the same time?

I already read about orphane blocks, or miners 51% ... but i don't really understand what really is happening, some basic explanation?

Are you asking how Bitcoin handles this? Or are you asking what YOU should do to handle it?

In Bitcoin, a node will accept the FIRST valid block it receives at a given height.  It will only switch to a different block at that height if it is part of a longer chain.  The abandoned block is then called an "orphaned" block.

Lets imagine there is a blockchain with 3 blocks in it (blocks A, B, and C):

A -> B -> C

Now imagine that you are a miner and you mine a new block (block D1) on top of block C.
At the exact same time someone else mines a new block (block D2) on top of block C.

You both broadcast your blocks to the peers that you are connected to.  The blocks are relayed throughout the network and eventually I receive block D1 (your block) from one of my peers.  My node verifies that your block is valid. It is, so I add it to my blockchain, and relay it to all the peers that I am connected to. A millisecond later, one of the peers that I'm connected to sends me block D2 (the one you did NOT create).  I see that I already have a block at that height, and that both chains are the same length, so I just keep block D1.

Meanwhile someone else receives block D2 first from one of their peers.  Their node verifies that block D2 is valid. It is, so they add it to their blockchain, and relay it to all the peers that they are connected to.  A millisecond later, they receive block D1. They see that they already have a block at that height, and that both chains are the same length, so they just keep block D2.

Since, on average if the blocks were broadcast at EXACTLY the same time, approximately half of the nodes and miners on the network will hear about block D1 first, and the other half will hear about block D2 first, the network will disagree on what the blockchain looks like right now.  They will NOT have consensus.

Those that hear about block D1 first will have the following blockchain:
A -> B -> C -> D1

Those that hear about block D2 first will have the following blockchain:
A -> B -> C -> D2

Eventually a miner will solve a new block. Lets imagine that it is a miner that was working with the D2 chain. We'll call his block E2. He will broadcast his block to all the peers he is connected to.  If they were working with the D2 chain, then they will just add the E2 block to their chain after they validate it.  If they were working with the D1 chain, then they will see that the D2 chain is now longer than their chain.  They will abandon their chain and switch to the now longer D2 chain.  As block E2 spreads through the network eventually all nodes will switch to the, now longer, D2 chain.  All nodes will have a chain that looks like this:
A -> B -> C -> D2 -> E2

The D1 block will be abandoned as an "orphaned" block.

When this happens, can i just catch the longest available valid BlockChain and replace it with all other nodes?

Here's a great video that explains how bitcoin works, and why it works the way that it does.  It generalizes some of the technical details about how the block hashing and transactions work, so don't take it as a technical explanation of exactly what is happening in bitcoin, but it clarifies some of the general concepts and helps explain why bitcoin is structured the way that it is:

https://www.youtube.com/watch?v=bBC-nXj3Ng4

[redigaffi]

Nice vid, it helped a lot.

When I have a Beta of the project could you check it out? I'm going to publish the code at github.

Thank you!

[redigaffi]

How could I calculate the probability of finding X consecutives '0's in a sha string?

[DannyHamilton]

How could I calculate the probability of finding X consecutives '0's in a sha string?

Note that the concept of requiring "consecutive 0's" is a generalization that is often used when introducing the "proof-of-work" concept to people.

In reality, bitcoin doesn't require a specific number of 0's.

Instead, bitcoin sets a target value, and requires that the value of the hash be less than the target.

The result of SHA256 is NOT a string.  It is a number. That number can be represented in binary, hexadecimal, decimal, or any other base that you like.  When it is represented as a hexadecimal number, it looks a bit like a "string" because it has the letters A, B, C, D, E, and F in it.

Because it is actually a number, which is just being displayed to you in hexadecimal (or base₁₆), it can be easily compared to a target value to determine if it is greater than, equal to, or less than that target.

If you use a binary representation of a 256 bit hash, and you require a specific number of 0's then each increase in difficulty is DOUBLE the previous difficulty.  This isn't very useful when you only want to increase the difficulty by 10% or 15%.

Calculating the probability is as simple as dividing the quantity of valid values by the quantity of possible values.

Here's an example...
Lets imagine instead of using a 256 bit hash, we use an 8 bit hash.
With 8 bits you can represent a total of 256 "possible values" (0 through 255).
If you set the difficulty target at a value of 128, then any value less than 128 is a "valid value".
Therefore, the probability of a hash being less than the target is:
128/256 = 0.5 = 50%

Now lets imagine that we increase the difficulty by lowering the target value to 95.
In that case, the probability of a hash being less than the target is:
95 / 256 = 0.371 = 37.1%

Notice that if you choose to require that the binary representation of the hash start with a number of consecutive zeros, then you ONLY have 8 possible target values:
Zero 0's: xxxxxxxx = 256 valid values out of 256 possible values = 100%
One 0: 0xxxxxxx = 128 valid values out of 256 possible values = 50%
Two 0's: 00xxxxxx = 64 valid values out of 256 possible values = 25%
Three 0's: 000xxxxx = 32 valid values out of 256 possible values = 12.5%
Four 0's: 0000xxxx = 16 valid values out of 256 possible values = 6.25%
Five 0's: 00000xxx = 8 valid values out of 256 possible values = 3.125%
Six 0's: 000000xx = 4 valid values out of 256 possible values = 1.5625%
Seven 0's: 0000000x = 2 valid values out of 256 possible values = 0.78125%
Eight 0's: 00000000 = 1 valid value out of 256 possible values = 0.390625%

Notice that if you want a probability of 10%, you can't do it with "consecutive leading zeros".  However, if you use a "target value", and an 8 bit hash then you can increase or decrease difficulty by as little as 0.390625% if you simply increment or decrement the target value by 1.


A 256 bit hash with a "target value" gives you even more control over the probability. Each time you increase or decrease the target value by 1, you increase or decrease the probability by:
1 / 2²⁵⁶ = 0.00000000000000000000000000000000000000000000000000000000000000000000000000086%

[redigaffi]

So, bitcoin basically says to the miner,

this is the target: 5000 (between is it a random number?? that the node generates?)

And the miner has to find a hash also:

if sha256(prevHash + randNumber) < target
   proof = randNumber

Ok,  where do target comes? is it a random number or has it some relation with previous blocks?

[DannyHamilton]

So, bitcoin basically says to the miner,

this is the target: 5000

Effectively, yes.

Technically, bitcoin uses a 32 bit number to store the target. So, it doesn't have a full range of 2²⁵⁶ target values.  Each miner and node uses the same formula for calculating what the target value is from the 32 bit representation.

You can read the technical details at the following link to learn about the conversion from the 32 bit "nbits" value stored in the block header to the actual target value:
https://bitcoin.org/en/developer-reference#target-nbits

(between is it a random number?? that the node generates?)

The target is not "random".  The target started a a minimum value set by Satoshi.  At that point, there wasn't enough hash power on the network yet for blocks to be completed every 10 minutes, so mining was a bit slower in the beginning.

Once every 2016 blocks, EVERY node, EVERY solo miner, and EVER mining pool performs the exact same difficulty re-target calculation for them selves.  Since they are all working with the same data, and they are all doing the exact same calculation, they all end up at the exact same answer.

The process is effectively to do the following:

• Look at the timestamp of the current block.
• Look at the timestamp of the block that was created 2016 blocks ago.
• Calculate the difference between the two timestamps.
• If that difference is less than 20160 minutes, then the blocks are being created too quickly.  Calculate the ratio between the actual difference and the intended 20160 minutes.  Adjust the difficulty target to be more difficult by that same ratio to get the average time between blocks back closer to 10 minutes.
• If that difference is more than 20160 minutes, then the blocks are being created too slowly.  Calculate the ratio between the actual difference and the intended 20160 minutes.  Adjust the difficulty target to be less difficult by that same ratio to get the average time between blocks back closer to 10 minutes.
• If the calculated adjustment ratio is more than 400% then only adjust by 400% this time, and check again in 2016 blocks.

And the miner has to find a hash also:

if sha256(prevHash + randNumber) < target
   proof = randNumber

No.

Miner has to find hash:

If SHA256(currentBlockHeader) < target
  proof = currentBlockHeader

The block header for bitcoin contains 6 pieces of information:

• 4 byte version number
• 32 byte hash of previous block
• 32 byte merkle root (commitment to the block transaction list)
• 4 byte timestamp
• 4 byte nbits (difficulty target representation)
• 4 byte nonce

Since each nonce value has the same probability of success, typically the miner would try all 4,294,967,296 possible nonce values in order rather than trying random nonce values. If they try all possible nonce values and they haven't found a solution, then they need to change something else in the header (which would allow them to try all 4,294,967,296 possible nonce values again with the new, different, header.  An easy thing to change is the timestamp, however they could also change the transaction list (which would result in a new merkle root), or the version number.

Ok,  where do target comes? is it a random number or has it some relation with previous blocks?

It has a relation to the amount of time that it takes to create 2016 blocks.  The difficulty is not adjsuted after every block.  It is only adjusted once every 2016 blocks.  It is adjusted to keep the average amount of time between blocks close to 10 minutes.

[redigaffi]

Hey! Thanks again a lot!

I understand better now, so this would be a basic implementation of Bitcoin's HashCash adaption:




But I doesn't understand how to calculate the difficulty, so for example, I start with this target (it's the start ):


target = 9999999999999999999999999999999999999999999999999999999999999999999999

How could i calculate the difficulty/probability? I dont know the max integer outcome of:
let responseAsInt = parseInt(response, 16);

Between, to do it more difficultier to mine, would i just remove nine's at the end, or just rest, for example:

999999999999999999999999999999999999999999999999999999999999999999 (removed for 4 nine's)

or

9999999999999999999999999999999999999999999999999999999999999999990001 (removed 9998 )


Should have the same effect, but what is better?

Thanks a lot really!

[DannyHamilton]

Hey! Thanks again a lot!

I understand better now, so this would be a basic implementation of Bitcoin's HashCash adaption:

- snip -

But I doesn't understand how to calculate the difficulty, so for example, I start with this target (it's the start ):

target = 9999999999999999999999999999999999999999999999999999999999999999999999

How could i calculate the difficulty/probability? I dont know the max integer outcome of:
let responseAsInt = parseInt(response, 16);

Between, to do it more difficultier to mine, would i just remove nine's at the end, or just rest, for example:

999999999999999999999999999999999999999999999999999999999999999999 (removed for 4 nine's)

or

9999999999999999999999999999999999999999999999999999999999999999990001 (removed 9998 )


Should have the same effect, but what is better?

Thanks a lot really!

You can't use nodejs integers (such as returned from the parseInt method) with SHA256, because the maximum integer value that nodejs can safely hold natively is 9007199254740991:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/MAX_SAFE_INTEGER

You'll need to include some big integer module if you want to literally compare values as decimal integers.

That being said, SHA256 has a maximum value of 115792089237316195423570985008687907853269984665640564039457584007913129639936

So, a target of 57896044618658097711785492504343953926634992332820282019728792003956564819968 would give every hash a 50% chance of being successful.

A target of 11579208923731619542357098500868790785326998466564056403945758400791312963994 would give every hash a 10% chance of being successful.

A target of 1157920892373161954235709850086879078532699846656405640394575840079131296399 would give every hash a 1% chance of being successful.

A target of 1157920892373161954235709850086879078532699846656405640394575840079131296 would give every hash a 0.001% chance of being successful.

Perhaps you can figure out how to do that math?

[redigaffi]

Hello! Thanks again,

So I think I understood now, but i found this article:

https://samsclass.info/141/proj/BitDiff.htm

where it says:

"Bitcoin miners must find a SHA-256 hash that starts with many zeroes"

So I'm confused now, i though you told me that the server sends you a challenge (a hash), and the target,  and you (the miner) have to find:

if (sha256(challenge + randomStuff) < target)

Best regards!

[DannyHamilton]

Hello! Thanks again,

So I think I understood now, but i found this article:

https://samsclass.info/141/proj/BitDiff.htm

where it says:

"Bitcoin miners must find a SHA-256 hash that starts with many zeroes"

So I'm confused now,

That website is wrong.

There is a LOT of incorrect information about bitcoin on the internet.  You should be careful about what information you trust.

i though you told me that the server sends you a challenge (a hash), and the target,  and you (the miner) have to find:

if (sha256(challenge + randomStuff) < target)

If you are mining in a mining POOL, then you will configure your software so that the pool server and your client software agree on the difficulty target that you will work with.

If you are solo mining, then there is no SERVER.  Bitcoin doesn't have any servers, bitcoin is a peer-to-peer system.  In the case of Bitcoin, each node calculates the difficulty target themselves.  They are all using the blockchain as the source of their information, and they are all performing exactly the same calculations, so they all end up calculating the same difficulty.

The difficulty of a current block can be calculated from the "bits" field in the block header.

Here is a more reliable source of information about the difficulty target:
https://en.bitcoin.it/wiki/Difficulty

[redigaffi]

Hello! Thanks again,

So I think I understood now, but i found this article:

https://samsclass.info/141/proj/BitDiff.htm

where it says:

"Bitcoin miners must find a SHA-256 hash that starts with many zeroes"

So I'm confused now,

That website is wrong.

There is a LOT of incorrect information about bitcoin on the internet.  You should be careful about what information you trust.

i though you told me that the server sends you a challenge (a hash), and the target,  and you (the miner) have to find:

if (sha256(challenge + randomStuff) < target)

If you are mining in a mining POOL, then you will configure your software so that the pool server and your client software agree on the difficulty target that you will work with.

If you are solo mining, then there is no SERVER.  Bitcoin doesn't have any servers, bitcoin is a peer-to-peer system.  In the case of Bitcoin, each node calculates the difficulty target themselves.  They are all using the blockchain as the source of their information, and they are all performing exactly the same calculations, so they all end up calculating the same difficulty.

The difficulty of a current block can be calculated from the "bits" field in the block header.

Here is a more reliable source of information about the difficulty target:
https://en.bitcoin.it/wiki/Difficulty

Ok, right, yes there are no servers, but in solo mining, the mining software (the client) must connect to any available bitcoin node to ask for difficulty and a challenge, the miner search a hash that meets the requirements and send it back to a node or many with the hash and his address.

Between, I cant figure out how to adjust the difficulty, right now I implemented the way you told me,

A node sends the miner a target, and the miner has to find a hash below the target.

So just for testing purpose my current target is:

target = 115792089237316195423570985008687907853269984665640564039457584007913129639936 (2^256)

Every 2 block i adjust difficulty, for every 2 block it should need 2 mins, so my ratio would be:

ratio = 2/120 (0,016666667)

So, right now with my current target, 2^256 block are created instantanly, the time what i need to press "mine" button, 2 block are created in 7~ seconds, so actual ratio would be:

actualRatio = 2/7 (0,285714286) ( a lot easier )

At this step i can't figure out how to relation ratio and actualRatio and target to set target to my ratio, to adjust the difficulty.

What do i need to adjust the target with ratio and actualRatio?

Any suggestion?

Many thanks!

[achow101]

Ok, right, yes there are no servers, but in solo mining, the mining software (the client) must connect to any available bitcoin node

Usually mining software connects to a miners own node, not just any node. Any node likely does not have the required interfaces exposed to the general internet so you can't just connect to any node to get the information.

to ask for difficulty and a challenge, the miner search a hash that meets the requirements and send it back to a node or many with the hash and his address.

No, that's not how mining works.

The mining software will ask the node for the information necessary to construct a block. This is usually the block version, current difficulty, previous block hash, and branches of the merkle tree (including leafs, i.e. transactions themselves). The mining software then constructs the block by assembling the block header and filling in the details that it needs to, e.g. the coinbase transaction with the payout to his own address. When the mining software has completed a block, it then submits the entire block (header and transactions) to the node. It does not give it a hash or many hashes and an address; it gives it the full block usually.

Between, I cant figure out how to adjust the difficulty, right now I implemented the way you told me,

A node sends the miner a target, and the miner has to find a hash below the target.

So just for testing purpose my current target is:

target = 115792089237316195423570985008687907853269984665640564039457584007913129639936 (2^256)

Every 2 block i adjust difficulty, for every 2 block it should need 2 mins, so my ratio would be:

ratio = 2/120 (0,016666667)

So, right now with my current target, 2^256 block are created instantanly, the time what i need to press "mine" button, 2 block are created in 7~ seconds, so actual ratio would be:

actualRatio = 2/7 (0,285714286) ( a lot easier )

At this step i can't figure out how to relation ratio and actualRatio and target to set target to my ratio, to adjust the difficulty.

What do i need to adjust the target with ratio and actualRatio?

I highly suggest that you read the source code in Bitcoin Core that calculates the difficulty for each block: https://github.com/bitcoin/bitcoin/blob/master/src/pow.cpp#L13

To calculate the next proof of work, you multiple the last block's target by the amount of time it actually took, then divide it by the time you want it to take. You have a formula that looks like this:

Code:

(Last block's target)/X = (wanted time)/(actual time)

Then you just solve for X. In your example, you would do

Code:

(2^256)/X=7/120
X=(2^256)*120/7

Because we are dealing with integers here and not decimals, this is just the floor of that, so your target would be 1985007244068277635832645457291792706056056879982409669247844297278510793827474 .

[redigaffi]

Thanks a lot to all.

So, as I understand from this post:
https://bitcointalk.org/index.php?topic=1734235.0

Miners are nodes but they do additional tasks.

But how can this possible, I have mined bitcoins using a client that has less then 20mb of installation size, the blockchain is around 165.30GB

Between, as it is my coin, i can do it the way I like, or just try other methods, right?

There is no "cryptocurrency standard" or is there any?

And about "miners usually construct the block and send it back" Why this way? and not:

The node sends the miner a hash (target) and difficulty and the miner finds a hash that fits the difficulty and send it back to the node, and the node then constructs the block.

Best regards!

[DannyHamilton]

Thanks a lot to all.

So, as I understand from this post:
https://bitcointalk.org/index.php?topic=1734235.0

Miners are nodes but they do additional tasks.

But how can this possible, I have mined bitcoins using a client that has less then 20mb of installation size, the blockchain is around 165.30GB

In that case you were a participant in a mining pool.  The pool operated the node for you.


Technically, the pool was mining, they were just using your hash power (with your permission) to do it. You were just renting the hash power of your computer to the pool.  The pool then paid you for the use of your hash power.  This is why you got paid even though you never solved a block yourself.

Between, as it is my coin, i can do it the way I like, or just try other methods, right?

It's your software.  You can do whatever you want with it.

It's possible that your ideas will be better than anything anybody else has ever thought of.

It's also possible that your ideas will be much worse than existing coins such as Bitcoin.

There is no "cryptocurrency standard" or is there any?

In my personal opinion, Bitcoin is the ONLY viable cryptocurrency.  The rest are all either scams or interesting experiments.

However, there are many people that have different opinions than mine.  There is no "official" business or agency in charge of setting standards. There are some common design patterns, but if you can come up with an idea that you want to implement, then you can implement it.

And about "miners usually construct the block and send it back" Why this way? and not:

The node sends the miner a hash (target) and difficulty and the miner finds a hash that fits the difficulty and send it back to the node, and the node then constructs the block.

The data that is being hashed is the block header.  Therefore, the header MUST be constructed before you can start hashing.

The header includes a merkle root that commits the transaction list to the header. Therefore, the merkle root MUST be constructed before you can construct the header.

The merkle root is constructed from the block transaction list. Therefore the block transaction list MUST be constructed before you can construct the merkle root.

Once you have constructed the transaction list, merkle root, and block header...  You have constructed the block header.  There is nothing left to do.

[redigaffi]

Thanks man, sorry for my stupid questions... I thought that i knew how bitcoin architecture is, but every time I've got more questions, I'm software developer since many years, and this project is for pure learning purpose, so I just want to try to make this the most "correct" possible.

I have problems with difficulty adjustments


Following what achow101 has said to calculate time for target:

"you multiple the last block's target by the amount of time it actually took, then divide it by the time you want it to take."

Following this, I have:

lastTarget = 2^256 = 115792089237316195423570985008687907853269984665640564039457584007913129639936

newTarget=(lastTarget*7)/120 = 67545385388434447330416407921734612914407491054956995689683590671282658956 (where the 7 is the time it took to mine 2 blocks)

difficulty = newTarget/LastTarget = 0.00058%

I'm getting a new target, but it's too easy... I can mine with this target in less than 2s, it should be atleast 1m or so...

Best regards

[redigaffi]

Hello Again,

So Danny, you said that miners don't search hashes with 0 at the start, but why then all block hash have 0's before? for example:

https://blockexplorer.com/block/0000000000000000008664c500e67e68ecc775868eaad375183a9b92b121abf1

And:

When a node receives a transaction, if the transaction is correct, does the node broadcast the transaction? before being added to a block? or the node just wait till resolving the block and broadcast the block ?

[DannyHamilton]

Hello Again,

So Danny, you said that miners don't search hashes with 0 at the start, but why then all block hash have 0's before? for example:

https://blockexplorer.com/block/0000000000000000008664c500e67e68ecc775868eaad375183a9b92b121abf1

Because the hash has to be lower than the target value.

During the time in which that block was mined, the target was:
0x000000000000000000c1bd000000000000000000000000000000000000000000

And this value:
0x0000000000000000008664c500e67e68ecc775868eaad375183a9b92b121abf1

is less than that target.

Note that the following value:
0x000000000000000000e673b61ff56f79fbd866759f9be284292b8ca3a230bce0

has the same number of zeros, but is NOT valid, because it is higher than the target value.

And:

When a node receives a transaction, if the transaction is correct, does the node broadcast the transaction? before being added to a block?

Full nodes validate transactions and then send the transaction to all connected peers that have not yet received the transaction.

This is why you see a transaction that was sent to you immediately, before it is confirmed.  The sender didn't necessarily connect directly to your node (how would they know where to connect to?).  Instead they sent the transaction to the peers they were connected to, and those peers sent it to the peers they were connected to, and those peers sent it to the peers they were connected to, and so on until it got sent to a peer that you are connected to, and that peer sent it to you.

Note that this is also how the miners and mining pools find out about the transactions so that they can add them to their blocks.  Nodes and wallets don't need to know the IP address of EVERY miner on the system.  For that matter, they don't need to know the IP address of ANY miners on the system. They just send the transaction to the peers they happen to be connected to.  The transaction gets relayed throughout the network and eventually miners hear about the transaction from the peers that they are connected to.  Then they can add the transaction to the block they are working on if they want to.

Thanks man, sorry for my stupid questions...

No stupid questions.  We aren't born understanding the Bitcoin protocol.  It's something that we learn.  Sometime we learn it through reading documentation or the source code itself, and sometimes we have others clarify and explain the pieces that we don't understand.

I would suggest, however, that you take some time to read about the protocol.  If you ONLY learn from the things that you think to ask, then you are NEVER going to learn the things that you aren't aware that you need to ask about.

Since you are a "software developer since many years", then I STRONGLY suggest you take a look at the source code of a full node implementation.  You'll get a much better understanding of exactly what it is doing.

I have problems with difficulty adjustments
. . .

I'm getting a new target, but it's too easy... I can mine with this target in less than 2s, it should be atleast 1m or so...

You are saying that it took 7 seconds to mine a block, and then after increasing the difficulty you were able to mine the blocks even faster (2 seconds)?

There are a couple of possibilities that come immediately to mind here...

1.  Perhaps you are miscalculating the amount of time that your software spent actually calculating hashes?  If you are including the amount of time that it takes for your software to load up into RAM and to build the block, and to output that block to you, then you are including a LOT of variable time that could be different every time you run the program.  On such a small timescale those variable times have a larger influence on total time than the block hashing has.  Possible solutions would be:

• Try timing how long it takes to solve 2000 blocks, and then adjust your difficulty
• Start with a MUCH lower target so that hashing is a larger percentage of the processing time
• Iterate multiple adjustments to see if it eventually starts growing appropriately as hashing becomes an ever larger percentage of processing time

2. You are not properly comparing target to hash?  It is possible that you have a bug in your code that is causing it to exit your hashing loop before it has found a low enough hash value. Possible method to determine this would be:

• Print out both the target and the solution hash when your software finishes processing a block so you can compare the values yourself and see if they are correct