[ANN][CLAM] CLAMs, Proof-Of-Chain, Proof-Of-Working-Stake, a.k.a. "Clamcoin"

[SuperClam]

Awesome work with the stake data chilly2k - we're happy to have you as part of the family

Deb calls it "the clamily".

Interesting about the mask, that is the part of the code I was not processing correctly. Ok so I think we have the wait 16 seconds part of it cleared.

No, because look at the lines before that:

   if (nSearchTime > nLastCoinStakeSearchTime)
    {
        int64_t nSearchInterval = IsProtocolV2(nBestHeight+1) ? 1 : nSearchTime - nLastCoinStakeSearchTime;

It only gets into that block if nSearchTime is greater than the last search time. And nSearchTime has already been rounded down to a multiple of 16 seconds. So it will only get into this block at most once per 16 seconds.

After 16 seconds have passed this code continues, and nSearchInterval becomes 1. I still can't follow exactly where it iterates 16 different timestamps in one hashing session? I might have to add a couple of log prints with timestamps into the code to visualize this better.

dooglus can verify and expound on this; but, the concept is that it doesn't iterate all 16 One second increment timestamps.  In fact, if you look at the timestamps of on-chain blocks (approved by network consensus) - all of them have 16 second interval timestamps.

CheckCoinStakeTimestamp() mandates that block timestamps must follow the 16 second mask.
You could hash all 16 seconds if you pleased, but peers would reject the block as a timestamp violation.

[presstab]

Awesome work with the stake data chilly2k - we're happy to have you as part of the family

Deb calls it "the clamily".

Interesting about the mask, that is the part of the code I was not processing correctly. Ok so I think we have the wait 16 seconds part of it cleared.

No, because look at the lines before that:

   if (nSearchTime > nLastCoinStakeSearchTime)
    {
        int64_t nSearchInterval = IsProtocolV2(nBestHeight+1) ? 1 : nSearchTime - nLastCoinStakeSearchTime;

It only gets into that block if nSearchTime is greater than the last search time. And nSearchTime has already been rounded down to a multiple of 16 seconds. So it will only get into this block at most once per 16 seconds.

After 16 seconds have passed this code continues, and nSearchInterval becomes 1. I still can't follow exactly where it iterates 16 different timestamps in one hashing session? I might have to add a couple of log prints with timestamps into the code to visualize this better.

dooglus can verify and expound on this; but, the concept is that it doesn't iterate all 16 One second increment timestamps.  In fact, if you look at the timestamps of on-chain blocks (approved by network consensus) - all of them have 16 second interval timestamps.

CheckCoinStakeTimestamp() mandates that block timestamps must follow the 16 second mask.
You could hash all 16 seconds if you pleased, but peers would reject the block as a timestamp violation.

Wow! Ok now that you say that I see the same 16 second mask in the check stake code. Now this all makes sense in the context dooglus put it in. This is a pretty cool way to hash for proof of stake. Now I know why I was so lost :/

[dooglus]

Wow! Ok now that you say that I see the same 16 second mask in the check stake code. Now this all makes sense in the context dooglus put it in. This is a pretty cool way to hash for proof of stake. Now I know why I was so lost :/

The nSearchInterval variable used to (before v2 of the protocol) be set to how many seconds since the last time we tried hashing, and the hashing function would then iterate through all those missed seconds.

In v2 we don't want it doing that, so we set the variable to 1 so we get a single iteration of the loop.

The 16 second mask thing makes it 16 times easier to find a block than it was before, since we get to try 16 times less often, but still want the same frequency of blocks being solved. This makes it 16 times more likely that two peers will find a block at the same time, and so we end up with 16 times as many orphans as before. Other than that it's cool though.

Oh, and as for SuperClam's "dooglus can verify and expound on this; but", I don't need to. She pretty much nailed it.

[presstab]

Its a very clever concept. Essentially you were able to reduce the amount of work involved in the proof of stake hashing, while doing it in a manner that does not compromise solving blocks because the difficulty will adjust to make the statistics approximately correct.

I have always wondered why the CLAM difficulty chart moves in such smooth waves compared to other coins, I suppose this could have something to do with that. Although now I wouldn't be surprised if you clammers redesigned the difficulty adjustment code as well.

[dooglus]

Its a very clever concept. Essentially you were able to reduce the amount of work involved in the proof of stake hashing, while doing it in a manner that does not compromise solving blocks because the difficulty will adjust to make the statistics approximately correct.

I have always wondered why the CLAM difficulty chart moves in such smooth waves compared to other coins, I suppose this could have something to do with that. Although now I wouldn't be surprised if you clammers redesigned the difficulty adjustment code as well.

Three things:

1) When the time granularity changed from 1 second to 16 seconds, another accidental change went in at the same time which missed out a factor of 10^8 in the difficulty calculation. It made blocks 100 million times easier to solve, but gave us 16 times less chances to try, so the two factors combined made it 6.25 million times easier than before to stake. As a result everyone was staking every 16 seconds, everyone was orphaning everyone else for a few hours, the network difficulty adjusted upwards very quickly, and after 3 or 4 hours everything was back to normal. Is was kind of amazing to watch the difficulty adjustment code do its thing and deal with the accidental 10^8 error. Do you have long-term difficulty charts? If so you'll easily see the time I'm talking about. You need a log scale, or it will look like the difficulty was 0 before the steep rise.

Edit: I don't see how to adjust the date range on your chart, but http://blocktree.io/charts/CLAM shows this:



The difficulty wasn't really 0 before November 2014, it just looks that way when you don't use a log y-axis.

2) I don't think the CLAM project is responsible for the 16 second granularity idea or code. I think it is from one of the upstream projects. I've never paid attention to any of them, but I think it was dark coin, black coin, or something like that.

3) The difficulty adjustment code was written by me specifically for CLAM, and went live at the same time as the v2 protocol and the 10^8 error. It's not ideal how it waves up and down several times per day, but it's better than the previous system of difficulty adjustment, which was far too reactive to how long the last 1 block took to stake.

[SuperClam]

...
3) The difficulty adjustment code was written by me specifically for CLAM, and went live at the same time as the v2 protocol and the 10^8 error. It's not ideal how it waves up and down several times per day, but it's better than the previous system of difficulty adjustment, which was far too reactive to how long the last 1 block took to stake.

The network has grown a great deal and achieved a certain measure of stability.

If we ever fork again, what would be the downside of increasing the window and further smoothing difficulty?

[dooglus]

If we ever fork again, what would be the downside of increasing the window and further smoothing difficulty?

I just wonder if there isn't a better way of doing it that doesn't cause this constant swinging up and down.

Is there a chart of the DOGE difficulty over time I wonder. They use some difficulty adjustment system with a fancy name that I forgot. Turbo lightning. Black vortex. You know, something silly like that...

[presstab]

Interesting, thanks for filling me in a bit on the history. I might do some searching into which coins also have that granularity in it. I have never seen it or heard about it before. But then again I tend to stay in my own little corner, so why would I know

My long range chart for diff is here. My charts aren't anything fantastic at the moment, hope to make them better in the future. My focus has been more behind the scenes this last two weeks. I just launched a new indexing method that should keep the addresses fairly precise, as it will not DB any transactions until they are deep enough in the chain.

Also, preparing to stake clams I think I may add my 'splitblock' feature to my own build of the GUI that allows splitting one output into multiple outputs in one easy transaction, unless this is something clams has built in somewehre? I think the best strategy is more outputs = more chances to stake, instead of the theory that higher value outputs will stake easier?

[SuperClam]

Interesting, thanks for filling me in a bit on the history. I might do some searching into which coins also have that granularity in it. I have never seen it or heard about it before. But then again I tend to stay in my own little corner, so why would I know
My long range chart for diff is here. My charts aren't anything fantastic at the moment, hope to make them better in the future. My focus has been more behind the scenes this last two weeks. I just launched a new indexing method that should keep the addresses fairly precise, as it will not DB any transactions until they are deep enough in the chain.
Also, preparing to stake clams I think I may add my 'splitblock' feature to my own build of the GUI that allows splitting one output into multiple outputs in one easy transaction, unless this is something clams has built in somewehre? I think the best strategy is more outputs = more chances to stake, instead of the theory that higher value outputs will stake easier?

We have quite a bit of additional functionality around the stake system, such as:

Flags:
-maxstakevalue, -splitsize, -combinelimit

Commands "sendtoaddress", "sendfrom", and "rawtransaction" allow JSON style specification, like {"amount":aaa,"count":ccc} to cause the send amount to be split into ccc outputs each of size aaa CLAMs.

[SebastianJu]

Given a difficulty leading to 16 seconds these 4 seconds would be huge. I mean the difference to orphan blocks or get block orphaned. Or isnt it needed to calculate a block?

I don't think you're understanding still. "Difficulty leading to 16 seconds" isn't what's happening. The difficulty adjusts how hard it is to stake a block. It adjusts such that we find around one block per minute. But blocks can only be found when the time (in seconds since some date in 1970) is a multiple of 16. That only happens every 16 seconds. That's fixed by the protocol (until the developers change the protocol again, of course), and isn't related to the difficulty.

Yes, but what i wanted to ask, just dice has the chance to find a block each 16 seconds. When jd really needs 4 seconds to go through all outputs then this is huge. Most other staking wallets dont have so many outputs so that they most probably need way less time to search for hashes. This sounds, at least, like the chance of find blocks that get orphaned are high. Not taken the time into consideration that other miners need to propage their block to jd.

When you think about it... clam can be lucky that you are the owner of jd. You easily could orphan all other blocks, where jd finds a block too, and only find blocks yourself, or raise that chance only so slight that its not found out. Nobody could proof how long jd needs to proof a block another miner found.

So the 4 seconds are real and you meant it goes through each of that outputs (shouldnt it be inputs as long as they arent sent out?) and checks if it finds a hash? Does the output amount matter here? I mean you described rounding the amount of clams down to an integer. Does this apply to the address these outputs are on, so a big amount of clams or only to the single output? If the latter then one could get an advantage by sending the clams in amounts of 1 to a new address. The chance to find a block would be maximized?

It's a real 4 seconds. 4 seconds out of every 16 seconds the CPU on one core of JD's staking wallet server is pegged at 100%. They're outputs of the transactions that created them. They're not the inputs of any transactions yet, or they wouldn't be unspent. They're potential inputs, if you like, but actual outputs. When they stake they become inputs of the staking transaction.

I get now how you come to name them outputs. You see them from a protocol perspective. I saw them from the wallet perspective. They are inputs into the wallet. Though outputs transactionwise.

The rounding down to an integer was related to how the age of an output affected its staking power in an older version of CLAM. I think it used to multiply the value by the age and round down to an integer. I don't think it does that rounding any more, or the multiplication by the age. These days the staking power (called the "weight") is just the same as the value in CLAMs. Each output is considered separately. It doesn't matter if you have lots of 1 CLAMs outputs on a single address, or in lots of different addresses. They each get their own individual chance of staking, with a probability proportional to their own individual value in CLAMs.
There is a benefit to splitting your outputs up into several smaller outputs. Suppose you have 1000 CLAMs. It will stake very quickly, and become 1001 CLAMs. But then it will take 8 hours to mature before it can stake again. The best you could hope for it that it will stake 3 times per day (since that's how many 8 hour maturation periods you can fit into a day).

If instead you split it into 1000 outputs of size 1, each one tries to stake independently. Each one has a 1000 times lower chance of staking than the 1000 CLAM output did, but there are 1000 of them, so it takes roughly the same time for one of them to stake, and turn from 1 CLAM to 2 CLAMs. Then, however, only the 2 CLAM output is frozen for 8 hours while it matures. The other 999 CLAMs continue trying to stake. So you have saved yourself an 8 hour wait for 99.9% of your value.

If you split your value up into *too* many outputs, you'll have too much hashing to do every 16 seconds that you won't be able to get through it all. And if you ever want to spent your outputs, having them split up into millions of tiny pieces makes the transaction which spends them very big (and so very expensive in tx fees).

So there's a tradeoff - split enough, but not too much.

Ah ok. I thought about this. So i guess you could calculate the optimum amount of clams in an output in order to get the most stakes and consider the 8 h wait time. Then add the average withdraw amount over time with the staked clams and calculate the additional fees (for taking many outputs as inputs) into consideration too. Then use a server that brings down calculation time to < 1 second and you would have the optimum setup. Right?

[Bitcoiner2015]

If we ever fork again, what would be the downside of increasing the window and further smoothing difficulty?

I just wonder if there isn't a better way of doing it that doesn't cause this constant swinging up and down.

Is there a chart of the DOGE difficulty over time I wonder. They use some difficulty adjustment system with a fancy name that I forgot. Turbo lightning. Black vortex. You know, something silly like that...

Do you mean kimoto gravity well?

[tspacepilot]

Interesting, thanks for filling me in a bit on the history. I might do some searching into which coins also have that granularity in it. I have never seen it or heard about it before. But then again I tend to stay in my own little corner, so why would I know
My long range chart for diff is here. My charts aren't anything fantastic at the moment, hope to make them better in the future. My focus has been more behind the scenes this last two weeks. I just launched a new indexing method that should keep the addresses fairly precise, as it will not DB any transactions until they are deep enough in the chain.
Also, preparing to stake clams I think I may add my 'splitblock' feature to my own build of the GUI that allows splitting one output into multiple outputs in one easy transaction, unless this is something clams has built in somewehre? I think the best strategy is more outputs = more chances to stake, instead of the theory that higher value outputs will stake easier?

We have quite a bit of additional functionality around the stake system, such as:

Flags:
-maxstakevalue, -splitsize, -combinelimit

Commands "sendtoaddress", "sendfrom", and "rawtransaction" allow JSON style specification, like {"amount":aaa,"count":ccc} to cause the send amount to be split into ccc outputs each of size aaa CLAMs.

If I have 1 clam and I split it as far as possible down to 100,000,000 "clamtoshis" in order to have the best chance to stake is this really the right thing to do?  And if everyone else does it wouldn't it make your UTXO set astronomically huge?  And wouldn't it make spending any reasonable amount of clams extremely expensive (because of all the nano-outputs you'd have to reference)?

I'm sure I'm missing something.  Thanks in advance!

[dooglus]

Black vortex. You know, something silly like that...

Do you mean kimoto gravity well?

Doesn't sound right to me. I think it was cooler sounding.

I guess I'm thinking of this, from https://en.wikipedia.org/wiki/Dogecoin:

On March 12, 2014, version 1.6 of the Dogecoin client was announced. Along with allowing for there to be a fixed reward per block, the new client update also introduced a new difficulty algorithm called DigiShield

But KGW looks interesting too:

http://bitcoin.stackexchange.com/questions/21730/how-does-the-kimoto-gravity-well-regulate-difficulty

My point is that there are probably better ways to retarget the difficulty than the naive way we're currently doing it (which in turn is better than the way we were doing it before...).

[dooglus]

If I have 1 clam and I split it as far as possible down to 100,000,000 "clamtoshis" in order to have the best chance to stake is this really the right thing to do?  And if everyone else does it wouldn't it make your UTXO set astronomically huge?  And wouldn't it make spending any reasonable amount of clams extremely expensive (because of all the nano-outputs you'd have to reference)?

I'm sure I'm missing something.  Thanks in advance!

Splitting a CLAM into 10^8 separate outputs is going to cost you a lot in transaction fees.

Splitting is a case of diminishing returns. The gains you get from splitting 1 CLAM into two halves are far bigger than any gain from splitting those two halves into 4 quarters.

And yes, merging those outputs back into a single output will also cost you a fortune in transaction fees.

So it's a trade-off. There's pressure in both directions (too little splitting and your coins spend too much of their life "maturing"; too much splitting and you pay a lot of transaction fees both when splitting and when re-joining, and use too much CPU time checking all your outputs for staking opportunities), and so you can solve some equations to find the optimal splitting level based on how long you anticipate staking for, the speed of your staking machine, the price of CLAM, etc. (Or you can just set all your outputs to be 13.37 CLAM each because you think the number is 'kewl').

[SuperClam]

Black vortex. You know, something silly like that...

Do you mean kimoto gravity well?

Doesn't sound right to me. I think it was cooler sounding.
I guess I'm thinking of this, from https://en.wikipedia.org/wiki/Dogecoin:

On March 12, 2014, version 1.6 of the Dogecoin client was announced. Along with allowing for there to be a fixed reward per block, the new client update also introduced a new difficulty algorithm called DigiShield

But KGW looks interesting too:
http://bitcoin.stackexchange.com/questions/21730/how-does-the-kimoto-gravity-well-regulate-difficulty
My point is that there are probably better ways to retarget the difficulty than the naive way we're currently doing it (which in turn is better than the way we were doing it before...).

The main concept behind the the DigiShield and Kimoto Gravity Well was resistance to multi-pool hash "attacks".

If I understand Kimoto Gravity Well correctly, it worked by applying a curve such that more recent blocks have a higher weight in the equation that figures the next difficulty target.  This is of course similar to simple one block re-targets which were also a popular "solution".

DigiShield had implementation and exploit-ability issues.  I know because I was closely involved in the project when that feature was launched.  It functioned by making the difficulty adjustment non-symmetric.  In testing, this seemed to show a reduction in the type of "wave" oscillation you see as adjustments over-shoot.  In practice, it didn't work all that well.

xploited might be able to offer some insight into DigiShield's problems; as he was even more closely involved in it's inception.

[dooglus]

The main concept behind the the DigiShield and Kimoto Gravity Well was resistance to multi-pool hash "attacks".

If I understand Kimoto Gravity Well correctly, it worked by applying a curve such that more recent blocks have a higher weight in the equation that figures the next difficulty target.  This is of course similar to simple one block re-targets which were also a popular "solution".

DigiShield had implementation and exploit-ability issues.  I know because I was closely involved in the project when that feature was launched.  It functioned by making the difficulty adjustment non-symmetric.  In testing, this seemed to show a reduction in the type of "wave" oscillation you see as adjustments over-shoot.  In practice, it didn't work all that well.

xploited might be able to offer some insight into DigiShield's problems; as he was even more closely involved in it's inception.

I didn't realise you guys were at all involved with DigiShield, or I probably wouldn't have been so flippant about it. Which coin were you involved with which first added DigiShield? Was it a DOGE thing? I really don't pay much attention to other altcoins.

While we don't have to worry about multi-pools we do have to be able to cope with sudden changes to the total network stake weight. A big staker could go offline temporarily leaving the difficulty too high for a while, or someone could dig up a bunch of distribution CLAMs and start staking them, leaving the difficulty too low for a while. The effect is similar to a big multi-pool visiting.

With the first (single block) difficulty retargetting what would happen that due to variance we would see 20 second gap between blocks occasionally, and the retargetting algorithm would overreact, setting the difficulty far too high, and it would take quite a while for things to recover. That would leave us with >2 minute blocks for a while, when all that had happened was one staker got lucky once.

Are there any plans for a hard fork any time soon? If so, it would be worth experimenting with the various adjustment algorithms and seeing what we can do to improve things. I imagine running a simulation, plotting the difficulty over time with various different retarget algorithms, etc. If not, I don't think it's a big enough problem to warrant a hard fork on its own.

[daxxog]

Hey guys. Just created an instant exchange. It allows you to permanently link a bitcoin deposit address to a CLAM address. So you could use it to buy CLAMS automatically with coinbase or something. Also included is a simple API. If you have any questions, shoot me an email. Thanks!

https://clamgate.com/

[BayAreaCoins]

Hey guys. Just created an instant exchange. It allows you to permanently link a bitcoin deposit address to a CLAM address. So you could use it to buy CLAMS automatically with coinbase or something. Also included is a simple API. If you have any questions, shoot me an email. Thanks!

https://clamgate.com/

If it could scroll down on its own (the text box.)

[daxxog]

Hey guys. Just created an instant exchange. It allows you to permanently link a bitcoin deposit address to a CLAM address. So you could use it to buy CLAMS automatically with coinbase or something. Also included is a simple API. If you have any questions, shoot me an email. Thanks!

https://clamgate.com/

If it could scroll down on its own (the text box.)

Yeah I suppose I could add that feature. I mainly put the text box in so developers could easily see how the API works.

[BayAreaCoins]

Hey guys. Just created an instant exchange. It allows you to permanently link a bitcoin deposit address to a CLAM address. So you could use it to buy CLAMS automatically with coinbase or something. Also included is a simple API. If you have any questions, shoot me an email. Thanks!

https://clamgate.com/

If it could scroll down on its own (the text box.)

Yeah I suppose I could add that feature. I mainly put the text box in so developers could easily see how the API works.

ahhh no worries man! I just like chipping my 2 cents at shit lol