Transparent mining 2, or What part of Legacy should be left behind

[bitcoinpaul]

I said the other day and I'll say it again: Nxt is ultimately a low-level protocol.  Like the pipes that run under the streets, it serves as a necessary foundation for everything built above it.  What can be built is limitless, but in order for that limitlessness to exist, the foundation has to be simple.

+1

[ChuckOne]

- has to have a client
     - this costs nothing

 - risks all his stake if the client is malicious
     - this can be prevented with some forethought... which costs nothing

 - risks his hardware if the client is not secured against malicious attacks
     - risks his hardware?  This is quite far-fetched and highly unlikely.  It reminds me of the StuxNet virus, which made Iranian uranium-enrichment centrifuges spin until they burned themselves out.  But I know of very few "malicious attacks" that can fry my home PC.

- has to pay the electricity bill
     - this is a sunk cost.  The forger already has an electricity bill, and as has been stated countless times, the cost of electricity for running a Nxt node is very low.

- has to pay for the hardware/renewal of such
     - this is generally also a sunk cost, since people can forge on existing hardware... except for folks who have bought a Raspberry Pi: they're out $50.  Anyone who has spent $5000 on a spiffy Nxt forging rig should be a little embarrassed.

- has to pay for a place that hardware can exist in
     - This is also a sunk cost.  You already have a place to exist in, which you generally share with your hardware.  And your hardware isn't that big.

- has to pay for the bandwidth he contributes to the network
     - This is also a sunk cost, since you already pay for bandwidth. And you can still forge if you don't advertise your node address, which minimizes bandwidth usage.

- has to implement security measures (attacks, heat etc.)
     - "heat security"?  I don't even know what that means.  As for other security: most people already have an edge firewall protecting their local network.  People who don't use any form of security for their home setup are far more exposed already, for their existing hardware (PCs, etc.).  So we're at "sunk cost" again.

As your reply shows, you have no idea how precious time and other resources are and how computing works nowadays. Nothing of my previous points is far-fetched or completely impossible.

Furthermore, you just provide many relative arguments: of course forging is cheaper than mining. So what? Cheaper means not: no time, no hardware, no place etc. It still costs something. Compensation is therefore legit.

I said the other day and I'll say it again: Nxt is ultimately a low-level protocol.  Like the pipes that run under the streets, it serves as a necessary foundation for everything built above it.  What can be built is limitless, but in order for that limitlessness to exist, the foundation has to be simple.

Again? What's your point and what is the difference to what all we said already?

The current banking system enables us to do many things, too. It has flaws and I hope with NXT we create a system that has none of them but enables us to do the same things if not even more.

Btw. I like your analogy with the pipes. The infrastructure is NXT, of course. However, like in the real world how many companies support the pipes/cables/etc. underneath the ground? Very few, I guess. Most of them PAY for their maintenance.

Let me ask it right out: What exactly do you want to accomplish: no fees? low fees? different forging mechanism?

[Zahlen]

I'm pretty sure that when TF is fully activated, blocks will be 1 minute apart, no matter what. C-f-b mentioned every block would have a "group" of forgers (I believe it was 3) eligible to forge the next block. So they would all have to drop out at the same time to simply delay a single block (unlikely). This also has the side effect of being unable to guess who will truly release the next block within the group, thus make it hard to game your own transactions.

Ah thanks, that makes more sense. And that means they can check on each other and make sure all transactions are processed (instead of selectively including them).

[sepehr]

I know the Internet is a wild place and Bitcoin/Doge/Maxcoin has put dollar signs in lots of peoples' eyes... but you don't earn the right to get compensated for doing nothing.  Some people get lucky (just like lottery players do, or people who manage to sell an old phone with Flappy Bird on it on eBay for $60,000), but that's just not the norm.  Sorry to disappoint you... but hey, Santa Claus isn't real, either.

I completely agree with my wiki Boss!

I said the other day and I'll say it again: Nxt is ultimately a low-level protocol.  Like the pipes that run under the streets, it serves as a necessary foundation for everything built above it.  What can be built is limitless, but in order for that limitlessness to exist, the foundation has to be simple.

But there are some questions!
Who should pay for the pipes to go around the world? Server operators do so!
So they should paid because of their efforts to keep the network globalized.
My point is this!

[Ix]

Whatever the case may be, TF should avoid 15+ min time to next block situations. CfB already hinted at this, the high variance from willy-nilly needle in a haystack random hashing, and using base target to compensate, eventually leads to potentially catastrophic situations like multiple quick blocks followed by an insanely long one.

I'm not sure how Nxt avoids 15+ min blocks either. My info is a little hazy and the OP of this thread and the other linked do not clarify much, but forgers who miss a block are penalized for 1,440 blocks or something I think. That's not a big penalty, especially considering that there may be millions or billions of accounts and your odds of being selected again in those next 1,440 blocks are low. A choice of 3 forgers does not really improve things because all it does is take a big number and divide by 3. The number of potential forgers can't really scale up because then you have many multiple accounts competing for the same block which would be pretty disorganized and would not lend much weight to early transaction confirmations. So if you keep it low, you are hoping that some few specific accounts are all online and ready to forge. Needle in a haystack here too? Maybe.

[pandaisftw]

Whatever the case may be, TF should avoid 15+ min time to next block situations. CfB already hinted at this, the high variance from willy-nilly needle in a haystack random hashing, and using base target to compensate, eventually leads to potentially catastrophic situations like multiple quick blocks followed by an insanely long one.

I'm not sure how Nxt avoids 15+ min blocks either. My info is a little hazy and the OP of this thread and the other linked do not clarify much, but forgers who miss a block are penalized for 1,440 blocks or something I think. That's not a big penalty, especially considering that there may be millions or billions of accounts and your odds of being selected again in those next 1,440 blocks are low. A choice of 3 forgers does not really improve things because all it does is take a big number and divide by 3. The number of potential forgers can't really scale up because then you have many multiple accounts competing for the same block which would be pretty disorganized and would not lend much weight to early transaction confirmations. So if you keep it low, you are hoping that some few specific accounts are all online and ready to forge. Needle in a haystack here too? Maybe.

NXT forging is deterministic, we are not searching for a needle in a haystack. Even with 1 billion forging nodes, you will know with 99.99% (or something close to that) accuracy the next X people that will forge. In the future, base target (difficulty) will simply be a legacy from Bitcoin - which c-f-b mentioned in OP - because why do you need to adjust difficulty when the forging order is already determined? If the order is determined, you simply say "X, broadcast your block at 60s, otherwise you will be penalized." 60 seconds later, "Y, broadcast your block at 60s, otherwise you will be penalized." And so on. In this scenario, X and Y are groups of forgers.

Anyways, I won't claim to understand how any of this works, but the important concept to understand is that forging is deterministic, thus you can have blocks come out in exactly 60 second intervals.

[Ix]

Even with 1 billion forging nodes, you will know with 99.99% (or something close to that) accuracy the next X people that will forge.

The issue that I raised was that you do not know that they will be available and paying attention. If they are not, no block gets made and there is a delay to transaction confirmation.

[marek3ball]

Let me summarize:

We want:
 - passive forging (that what we do right now)
 - active forging (passive forging + additional service on top of NXT)
 - supporting the family
 - slow change

ok you're gonna have to explain what the differences are between active/passive forging.  I do not get how you are making the distinction

I would like to know the answer too.

[2Kool4Skewl]

Let me summarize:

We want:
 - passive forging (that what we do right now)
 - active forging (passive forging + additional service on top of NXT)
 - supporting the family
 - slow change

ok you're gonna have to explain what the differences are between active/passive forging.  I do not get how you are making the distinction

I would like to know the answer too.

In passive forging, you lease your forging power to another node.  You will not have to run your own node to collect transaction fees.

In active forging, you generate transaction fess by running your own node.

[Zahlen]

A choice of 3 forgers does not really improve things because all it does is take a big number and divide by 3.

Not divide. Suppose each forger has a small chance e of failing to forge (whether by d/c or some other reason. Assume this e is the same for all forgers to keep things simple). Then the chance of all 3 not forging is e^3, which is much smaller than e. Consequently the chance that the block gets generated on time is 1-e^3, this is much more reliable than 1-e, which is if there were only one candidate to generate the next block.

I'm guessing actual TF would choose more than 3 candidates, coz if you end up with only 2 candidates, no one can check against the other. With more candidates, idea then is that the chance of too many candidates dropping out from a pool is still much lower than that of any one specific candidate dropping out (math is no longer as simple as the above). And you can scale up the number of candidates as network connectivity and speed increases.

So if you keep [the number of accounts] low, you are hoping that some few specific accounts are all online and ready to forge.

That's not how Nxt works. The account that forges the next block is selected from only online accounts with an effective stake, not from all existing accounts.


Yeah it's tough to understand, even if you manage to read through all of this, and related threads! Helps if you can clear away existing assumptions from previous crypto experience, Nxt is designed from scratch. The wiki (link in my sig) may also help.

[CIYAM]

That's not how Nxt works. The account that forges the next block is selected from only online accounts with an effective stake, not from all existing accounts.

Whilst I do *get* the idea behind how the TF "choice" works I don't quite follow this part.

How can every node know which accounts are "online" (in order to consider for TF) when nodes would only be keeping connections to a fraction of all online nodes?

[Ix]

Not divide.

I was presuming a different system and referring to the high probability of failure in either scenario.

So if you keep [the number of accounts the number of candidates] low, you are hoping that some few specific accounts are all online and ready to forge.

That's not how Nxt works. The account that forges the next block is selected from only online accounts with an effective stake, not from all existing accounts.

I corrected your edit. But yes I see there were some misunderstandings and I get the gist of a pretty interesting system.

Yeah it's tough to understand, even if you manage to read through all of this, and related threads!

It'd be nice if the OP was less proselytizing and more detail. Not everyone has time to jump around hundreds of forum pages to garner details.

The wiki (link in my sig) may also help.

I may have to check it out at some point.

[ZeroTheGreat]

There is no "profit" in forging because merely forging adds no value.

Completely disagree! Only secure system'll be popular and for that forging'd be popular activity too, here works network effect: more independent forgers with their own goals, more secure system is for everybody else. So it adds value in security aspect of payment system and whole ecosystem.

Selfish miners (those who mine only to earn fees) should be "removed" from the system, they r not interested in success of Nxt and only want to cash-out.

"Selfish"? It like a communist's word. Selfish is the nature of human being. It's not a bad thing.

Agreed. And Nxt solves great problem: forgers selfishness won't bring so powerful arms race as exists in PoW-cryptos. Cos, especially if decentalized Internet'll rise up, end-user don't need infinite bandwith of node. Any imaginable system'll have boundaries of demand (and spam already prevented by minfees) and forgers after achieving it'll compete only in reputational field, which is not only normal, but wished too.

In passive forging, you lease your forging power to another node.  You will not have to run your own node to collect transaction fees.

Mgm, in my point of vie it can only strengthen up active forging, cos there'll be a possibility for each forger to increase his own power (which'll not increase EV, but'll decrease dispersion), and with that to ensure landlords the safety of their coins from being stolen.

[ZeroTheGreat]

That's not how Nxt works. The account that forges the next block is selected from only online accounts with an effective stake, not from all existing accounts.

Whilst I do *get* the idea behind how the TF "choice" works I don't quite follow this part.

How can every node know which accounts are "online" (in order to consider for TF) when nodes would only be keeping connections to a fraction of all online nodes?

Transparent Forging called transparent for a reason: in next phases of implementation TF (it's not complete for now) each tx's broadcaster'll scan network and calculate 100%-s of block success by working nodes in next 15 secs, asking them for their effectiveBalance and looking for network's BaseTarget (how many coins're forging now), then send tx info directly to them. Ofc, there still be orphans, some network dynamic dividing (nodes on/off, providers problems, geography and so on) and need to rebroadcast. This's my understanding, hot it'll work.

But bigger Nxt'll be, stronger network effect'll be. Compare it to torrents: when u're starting to download popular file, u're reaching cap speed almost instantly, cos exactly of network effect: more peers are, easier to set up complete map of interconnections and create optimal set of seeders for u.

[bitcoinpaul]

There is no "profit" in forging because merely forging adds no value.

Completely disagree! Only secure system'll be popular and for that forging'd be popular activity too, here works network effect: more independent forgers with their own goals, more secure system is for everybody else. So it adds value in security aspect of payment system and whole ecosystem.

I wonder if the idea is doomed because of Diffusion of Responsibility.

[CIYAM]

...each tx's broadcaster'll scan network and calculate 100%-s of block success by working nodes in next 15 secs...

Hmm... if your ping times are anything like they are in China (often 300+ms) then even in 15 secs you certainly won't be able to determine all active nodes so I do question this idea.

Can you give me some "concrete" examples of just how many nodes this can possibly scale to?

For example - you will never work out all connections if you have connections to 1M+ nodes (and will be consuming vast amounts of memory even trying to) - so has BCNext purposely limited the total network size to something quite paltry like a few thousand nodes?

[murraypaul]

Mining reward is another obsolete part. True reward for supporting Nxt network comes from services that use Nxt. Someone owns a currency exchange and mine blocks to keep his business running. Another one owns a shop and mine blocks to keep his business running. The 3rd person owns a software company that develops programs for Nxt-based services and mine blocks to keep his business running. Selfish miners (those who mine only to earn fees) should be "removed" from the system, they r not interested in success of Nxt and only want to cash-out. If a clone appears such the people will likely jump to another ship, they add very little value to Nxt. All this doesn't mean that we should get rid of fees completely, we still need them as a countermeasure against spamming.

Perhaps I am misunderstanding something, but this seems to idealistically ignore a different sort of selfishness.
If I'm running a Nxt-based business, I could either invest in forging myself, or just not bother, and expect someone else to do it.
A business that forges for itself, with minimal mining rewards, will be less efficient than one that just expects someone else to do it.
The free-riding company will out-compete the 'honest' company.

[Zahlen]

That's not how Nxt works. The account that forges the next block is selected from only online accounts with an effective stake, not from all existing accounts.

Whilst I do *get* the idea behind how the TF "choice" works I don't quite follow this part.

How can every node know which accounts are "online" (in order to consider for TF) when nodes would only be keeping connections to a fraction of all online nodes?

Some preliminaries:

I don't know how it'll be implemented eventually. Were I to try to design an algo to satisfy the properties of TF I've gathered so far, I might do

1. During the time between blocks, gather info on as many online forging nodes as you can, through the nodes you're connected to. Call these "seen nodes"

2. Strictly order these nodes, say by increasing account number. Insert new nodes/delete d/ced nodes as they are discovered.

3. Compute Hash(public_key_of_previous_forger + N)/(MAX_HASH + 1) for N = 1 to # of candidates. Hash could be say SHA256. Think of these as dice rolls in [0,1) that will determine the candidate accounts.

4. Just before block generation, calculate (your own view of) the candidate accounts for forging. The candidate accounts are precisely those accounts for which [ total_stake_of_previous_accounts / total_stake_of_all_seen_nodes, (total_stake_of_previous_accounts + own_stake) / total_stake_of_all_seen_nodes ) contains a die roll. These intervals are mutually disjoint, and their union is [0,1). Each interval represents each accounts stake in [0,1)

5. Some algo that determines which candidate is chosen, not relevant to our discussion here.

Finally getting to your question:

During the time between blocks, each node builds up their own picture of what the whole network will look like. Different nodes will have slightly different pictures due to different connections and connectivity (assume the overall network condition isn't so bad as to have radically different pictures). But even with slight differences, the die rolls are still likely to land in the same accounts in 4). (picture a [0,1) line segment broken up into very many separate account stakes. Assume no very big stakes. If you remove a few of them and insert a few others, while maintaining stake ratios, the picture doesn't change by much (math can provide bounds hmm, better to simulate it I think.)). So all nodes can achieve consensus.


Again, just my thinking, and coming from general math.

EDIT: No, this particular way doesn't work I think, need another way to determine candidates. But I think the general idea holds: you don't need all nodes to see the same picture of the network, you only need that their pictures mostly agree, and consequently mostly agree on the candidates.

[mthcl]

- has to have a client
     - this costs nothing

 - risks all his stake if the client is malicious
     - this can be prevented with some forethought... which costs nothing

 - risks his hardware if the client is not secured against malicious attacks
     - risks his hardware?  This is quite far-fetched and highly unlikely.  It reminds me of the StuxNet virus, which made Iranian uranium-enrichment centrifuges spin until they burned themselves out.  But I know of very few "malicious attacks" that can fry my home PC.

- has to pay the electricity bill
     - this is a sunk cost.  The forger already has an electricity bill, and as has been stated countless times, the cost of electricity for running a Nxt node is very low.

- has to pay for the hardware/renewal of such
     - this is generally also a sunk cost, since people can forge on existing hardware... except for folks who have bought a Raspberry Pi: they're out $50.  Anyone who has spent $5000 on a spiffy Nxt forging rig should be a little embarrassed.

- has to pay for a place that hardware can exist in
     - This is also a sunk cost.  You already have a place to exist in, which you generally share with your hardware.  And your hardware isn't that big.

- has to pay for the bandwidth he contributes to the network
     - This is also a sunk cost, since you already pay for bandwidth. And you can still forge if you don't advertise your node address, which minimizes bandwidth usage.

- has to implement security measures (attacks, heat etc.)
     - "heat security"?  I don't even know what that means.  As for other security: most people already have an edge firewall protecting their local network.  People who don't use any form of security for their home setup are far more exposed already, for their existing hardware (PCs, etc.).  So we're at "sunk cost" again.

As your reply shows, you have no idea how precious time and other resources are and how computing works nowadays. Nothing of my previous points is far-fetched or completely impossible.

Furthermore, you just provide many relative arguments: of course forging is cheaper than mining. So what? Cheaper means not: no time, no hardware, no place etc. It still costs something. Compensation is therefore legit.

Agree with ChuckOne. Look also at the example of bittorrent: the torrent trackers had to use rating systems and such, just because otherwise there would be too many leechers for too few seeders. That's not because it costs anything (at least in free countries like Brasil, where there is no risk that you could get prosecuted for that) just to leave the client open, it's because people just don't care about it. There are so many things to care about in this life, so it's better to offer at least some reward if we want that people invest some attention to the network maintenance.  

[Come-from-Beyond]

...each tx's broadcaster'll scan network and calculate 100%-s of block success by working nodes in next 15 secs...

Hmm... if your ping times are anything like they are in China (often 300+ms) then even in 15 secs you certainly won't be able to determine all active nodes so I do question this idea.

Can you give me some "concrete" examples of just how many nodes this can possibly scale to?

For example - you will never work out all connections if you have connections to 1M+ nodes (and will be consuming vast amounts of memory even trying to) - so has BCNext purposely limited the total network size to something quite paltry like a few thousand nodes?

BCNext limited nothing. We don't need to "ping" nodes.