Technical question regarding POS coins and CPU time

[allcrypt]

Ok. So then. Lets back this up a bit. I know a far amount about mining coins - I was looking into setting AllCrypt up as a pool/exchange and trying to figure out how to merge mine coins, and when doing that, I learned a lot about how mining works. Super short: There is a target, which is based off the difficulty. Miners slam random numbers through the algo, and see if it beats the target number. If so, you solved the block, submit it.

How the hell does POS work then? All POS coins are "advertised" as you get a % return per <time period> but this is obviously done by internal mining.

What's the process? It's based on the coins you have... how? It cant be CPU based (CPU mine as normal, if you hit the block your reward is percentage based) because then it could be cheated and someone with more CPU could mine more POS blocks.

So it's the coins themselves that "mine". How? What exactly is being hashed and compared to the target? Every coin you have? Every unspent input? Every address that has coins? What determines what is hashed? Because since you have finite coins in your wallet, you only have a finite number of tries to mine a block. If it's not finite, then it's CPU based and more CPU = more mining = If I have 10,000 coins and you have 10,000 coins and you have a faster CPU you'll mine more than me. So, can't be that. (Or is it that, but the rewards are throttled?)

I want to fix this issue. At least in the wallets on my site. And if it's an elegant solution, hell, maybe it's the next big thing. Lots of coins feature KGW or DigiShield. Not bad advertising to feature something our exchange came up with.

Again - I've googled everything I can think of trying to find a whitepaper or reference, and nothing gives me the info I'm looking for. Combing the code is ungodly painful when I dont even know what I'm looking for. Thats why I tried profiling... see what's chewing up the CPU. But, the results were worthless (see my post a few back).

[allcrypt]

New issue: Trying to clean up another wallet it keeps failing on the send. Thought it was a fee issue, started looking at the code. Had this in the debug:

ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629)

I went into the code and see that that 4ad39 is a substr of the first 10 characters. So I had it show the whole thing. It's the same length as a transaction ID, so I wrote all the txids (and the output from listunspent) before it tried to send. I thought maybe in the 2 seconds between listing the unspent, and the send, that it got used somehow.

Well that full string (middle part removed just in case it's part of a private key or god knows what else) 4ad396f7f0b3bdfc30f5f71ab21bxxxxxxxxxxxxxxxxxxxxaa36840c08fe8c93 isnt anywhere in the listunspent output. its not a block hash. Its not a txid in the wallet. No clue what it is. The whole nBlockPos and nTxPos isnt helpful either.

Anyone have a clue what the heck that is referring to? Maybe the wallet is broken and it's listing already-spent inputs erroneously in the listunspent? If I drop the # of transactions batched together down to a smaller number I have some luck.

Another edit: I'm pretty sure this coin is just broken. I reduced the max # of TX that can be batched together to 10, and now, the first 10 go, but when it tries the second 10, it bombs with a 'TX rejected' and there's absolutely nothing in the debug log. Not a size issue (it was 2342 bytes), not a fee issue (fee correctly added). I even added a 3 second wait before starting the next one.

Next thing I'm gonna try is grabbing 10 random inputs to use instead of the 10 smallest value inputs and see what happens.

ANOTHER edit: I'm thinking maybe its because the TX is being used for staking? I know if a wallet is staking coins you can't send them... I isolated one of the 'problem' transactions. Just have no idea how to see which coins the wallet is using for staking at the moment. Hrm

[allcrypt]

I've no clue. The blocks that show under 'Stake' in getinfo are all the immature staked blocks. I'm not sure how to see which blocks are being used FOR staking. I turned the reservebalance on but it's apparently not let go of that transaction, because I still cannot use it in a send. It's still showing in listunspent - so it hasnt been spent. But I cannot actually spend it, as it's said it's "used" already. Every time I try and send it, I get a DIFFERENT hash in the debug.log (this line: ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629) - the blockpos and txpos stay the same, but the hash is different each time. So I've no idea what it's referring to, at all.

Pulling out hair.

[bee7]

Ok. So then. Lets back this up a bit. I know a far amount about mining coins - I was looking into setting AllCrypt up as a pool/exchange and trying to figure out how to merge mine coins, and when doing that, I learned a lot about how mining works. Super short: There is a target, which is based off the difficulty. Miners slam random numbers through the algo, and see if it beats the target number. If so, you solved the block, submit it.

How the hell does POS work then? All POS coins are "advertised" as you get a % return per <time period> but this is obviously done by internal mining.

What's the process? It's based on the coins you have... how? It cant be CPU based (CPU mine as normal, if you hit the block your reward is percentage based) because then it could be cheated and someone with more CPU could mine more POS blocks.

So it's the coins themselves that "mine". How? What exactly is being hashed and compared to the target? Every coin you have? Every unspent input? Every address that has coins? What determines what is hashed? Because since you have finite coins in your wallet, you only have a finite number of tries to mine a block. If it's not finite, then it's CPU based and more CPU = more mining = If I have 10,000 coins and you have 10,000 coins and you have a faster CPU you'll mine more than me. So, can't be that. (Or is it that, but the rewards are throttled?)

I want to fix this issue. At least in the wallets on my site. And if it's an elegant solution, hell, maybe it's the next big thing. Lots of coins feature KGW or DigiShield. Not bad advertising to feature something our exchange came up with.

Again - I've googled everything I can think of trying to find a whitepaper or reference, and nothing gives me the info I'm looking for. Combing the code is ungodly painful when I dont even know what I'm looking for. Thats why I tried profiling... see what's chewing up the CPU. But, the results were worthless (see my post a few back).

Nope. Not just advertised. They also "mined", but the number of hashes to calculate per second is limited by the number of inputs eligible for staking. The reward (may be subject of upper limiting) for the PoS block is calculated (usually) as a multiple of percentage, input amount and input age. When the input is spent (by any means - regular payment tx or stake mint tx) then it's age reset to zero. The small set of data, 28 bytes length, the is a concatenation of the following data elements:

Code:

    ss << nStakeModifier;
    ss << nTimeBlockFrom << nTxPrevOffset << txPrev.nTime << prevout.n << nTimeTx;

is sha256d'ed. The resulting hash is multiplied by time weight of the input (the number of full days multiplied by the input amount in full coins) and is compared to the current proof-of-stake target. If the condition is met then the stake kernel is found. All of the above parameters but nTimeTx are constant for the same input since it becomes eligible for staking. Only nTimeTx is changed. Thus each input has a chance to find a block only once in a second. The probability of that chance depends on its amount, age, current difficulty and current time since Jan 1, 1970 in seconds. In a sense, if both of us have same set of 1000 inputs (we copied a wallet over), you have a powerful computer that is capable to check all 1000 inputs every second and I have a computer that is capable to iterate only through the half of inputs, you will find the stakes twice often than me. but then, when the input is spent it becomes ineligible for some time. In a long run each input in my wallet would also produce a kernel. Provided that the reward is a percentage of multiple of input age and it's value, the overall income would be the same for you and me.

Initially all this code have been implemented for PPC that uses sha256d as a block header hash, so the overhead to calculate were not significant. For these who has no many inputs even with scrypt or x11, or even x999 block hash function, this implementation would be just fine (the devices like raspberry are out of scope).

Side note: KGW or DGS has nothing to do with hashing - it is just a formula to determine the next difficulty.

Again - I've googled everything I can think of trying to find a whitepaper or reference, and nothing gives me the info I'm looking for. Combing the code is ungodly painful when I dont even know what I'm looking for.

Linus said once (I am not sure if he is the original author and I am not sure if I reproduce it precisely):

"... and you always may read the code"

[bee7]

New issue: Trying to clean up another wallet it keeps failing on the send. Thought it was a fee issue, started looking at the code. Had this in the debug:

ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629)

I went into the code and see that that 4ad39 is a substr of the first 10 characters. So I had it show the whole thing. It's the same length as a transaction ID, so I wrote all the txids (and the output from listunspent) before it tried to send. I thought maybe in the 2 seconds between listing the unspent, and the send, that it got used somehow.

Well that full string (middle part removed just in case it's part of a private key or god knows what else) 4ad396f7f0b3bdfc30f5f71ab21bxxxxxxxxxxxxxxxxxxxxaa36840c08fe8c93 isnt anywhere in the listunspent output. its not a block hash. Its not a txid in the wallet. No clue what it is. The whole nBlockPos and nTxPos isnt helpful either.

Anyone have a clue what the heck that is referring to? Maybe the wallet is broken and it's listing already-spent inputs erroneously in the listunspent? If I drop the # of transactions batched together down to a smaller number I have some luck.

Another edit: I'm pretty sure this coin is just broken. I reduced the max # of TX that can be batched together to 10, and now, the first 10 go, but when it tries the second 10, it bombs with a 'TX rejected' and there's absolutely nothing in the debug log. Not a size issue (it was 2342 bytes), not a fee issue (fee correctly added). I even added a 3 second wait before starting the next one.

Next thing I'm gonna try is grabbing 10 random inputs to use instead of the 10 smallest value inputs and see what happens.

ANOTHER edit: I'm thinking maybe its because the TX is being used for staking? I know if a wallet is staking coins you can't send them... I isolated one of the 'problem' transactions. Just have no idea how to see which coins the wallet is using for staking at the moment. Hrm

It is a txid.

try 'walletd gettransaction 4ad396f7f0b3bdfc30f5f71ab21bxxxxxxxxxxxxxxxxxxxxaa36840c08fe8c93'


It might be broken, sometimes shit happens.

If the input is spent during PoS block generation it is marked as such the same way as any other input: the block is created, signed and passed to ProcessBlock fn. Then if all checks are fine and the block becomes best chain the ConnectInputs marks everything spent by the block as spent in the tx db.

[bee7]

I've no clue. The blocks that show under 'Stake' in getinfo are all the immature staked blocks. I'm not sure how to see which blocks are being used FOR staking. I turned the reservebalance on but it's apparently not let go of that transaction, because I still cannot use it in a send. It's still showing in listunspent - so it hasnt been spent. But I cannot actually spend it, as it's said it's "used" already. Every time I try and send it, I get a DIFFERENT hash in the debug.log (this line: ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629) - the blockpos and txpos stay the same, but the hash is different each time. So I've no idea what it's referring to, at all.

Pulling out hair.

Then it definitely broken - once the block is written to the file it may not change its location and thus all transactions included into the block as well. If the wallet reports different tx id each time then it must be a bug.

I looked into some coin code, the tx id printed is the tx id of the transaction being submitted. The file positions printed are positions of the block and tx that already used this input.

The way the listunspent picks usnpent inputs is different to the one used in ConnectInputs. Try to start a wallet with -rescan option. it seems there is an inconsistency.

[allcrypt]

Real tired and read over the last few posts... so going to try and make sense here....

So that TXID I was seeing is the TXID of the send I'm attempting to do, correct? Thats some poor code then because the way it reads is that THAT  is the txid was already spent. I screwed with the code trying to figure this out, and added 'prevout.hash.ToString().c_str()' to that line thats printed to the debug.log, ran a createrawtransaction using a TXID that absolutely, for sure, was reported in listunspent, it failed and bombed out, and in the debug.log, the new  prevout.hash.ToString().c_str() I added to the code reported the "bad" txid that it's saying is already spent.

So I tried this with a few other CPU hogging wallets. Two others I had the same issue. It bombed out, reporting an input was already spent when listunspent was reporting it was available.

One of the coins I tried it on (cornerstonecoin CCX) had not flushed it's debug.log in ages (file size was 200+ megs), and I ran my code against it. First try collecting 876 inputs into a 100kb transaction failed. So I kept halving it, getting successes, then halving again on failures, until I had a 4 input transaction that failed, and upon failure, I had the script dump me the txids of the transactions it was trying to use.

I then grep'ed the debug.log for them. One was not in there. Two were the same txid, with different vouts, and the 3rd was unique. So I had 2 left. Both hit in there with this line (different txids obviously)

WalletUpdateSpent found spent coin 43.974794CCX fc1e5b508d66bc4c7f964ff0df195ed949d6a43e82b795b78a3dff195a6d30e2

Which is exactly the line that appears when I successfully SEND using sendrawtransaction. (Or any send for that matter).

Unfortunately they are not timestamped so I cannot see WHEN that was sent, but it appears that listunspent is for sure showing me inputs that HAVE already been spent. Unfortunately I've already restarted the CCX wallet and it wiped the debug log, so I cant estimate how far back the inputs had been spent. I'm now trying to restart litecoinplus with -rescan to see if that transaction that was hanging me up earlier is now recognized as spent.

What is a major pain in the ass is there is no way that I'm aware of to use the RPC to see if a previous input has been spent. My script may need to just bomb out and keep trying, getting smaller and smaller until it can flag a small subset of txids as already spent.

Just checked it - the coin started up, and one of the txids which had shown as unspent before a -rescan is now NOT showing. Going to try this on that coin again and see if it bombs out. It for sure seems like some of these wallets are not correctly marking some inputs spent.