Found this thread looking for some pywallet help and figure since it's still active I may as well ask.

I'm a developer with AllCrypt.com, a crypto exchange. We've been using the internal accounting on most wallets as a secondary/backup failsafe (user hacks site, starts withdrawing coins the hacked/corrupted/overwritten database says is there, but the wallet stops the send as it knows, internally, that the user does not have enough coins). We're moving away from that as MANY altcoins have horrendous internal accounting, and larger coins like BTC and DOGE are now taking a performance hit because of the bloat. We're implementing a new system that handles the accounting in a second layer, to free that load off the wallets themselves. (Plus as a dandy little bonus, we implemented a frontend to the wallet's RPC calls, thus extending the calls, or creating new ones as need be. Pre-caching the address keypool so no waiting for an address? Hallelujah! Who else hates walletpassphrase throwing an exception just because the wallet is already unlocked? Can I get an A-men?!)

Anyway (Sorry, it's late, I'm exhausted) We were going to just export and reimport all private keys into a new wallet, but, then we lose all transaction info (we can no longer do a gettransaction to get details of an older deposit, for instance).

Is there a way, with pywallet, to export all transactions, let us clean up the list (remove all "moves" to/from internal accounts) and re-import them into a clean wallet without any accounting?

I see on the import page the ability to import a single transaction, but it wants a "Txk" and "Txv"? Key and... value? That seems like it's missing info it would need. I assume to mass import them I'd need to script it?

Just wondering if there is a simple answer before starting to mess directly with the software. Thanks all.

Aaaaand... gone.

No blocks in 10 days. Gone.

Last night I deleted the blockchain and started re-downloading it.

It would randomly stop syncing. A stop and restart would fix it.

Then I noticed something - it kept dying at xxx57 blocks. I'm not sure if it was just a "We ran x,000 blocks now do something that causes us to freeze" or if xx,x57 was a magic number.

Anyway. Now, I start it, and a while later it completely crashes, with the debug.log full of:
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input
ERROR: mempool transaction missing input

I've spent about 6 hours of time on this coin over the weekend. It's offline until someone provides a solution that works, out of the box, without us wasting more development time on a dead coin. Sorry folks.

Oh good. I'm out. Until a dev provides a SOLUTION CCX is de-listed from AllCrypt. Anyone who has coins on deposit with us, we'd love to let you withdraw them, but we cannot get a blockchain to sync. Without the blockchain on our end moving, coins can't leave. Coins can't confirm coming in.

We make a new wallet and import all addresses - the chain moves. However the wallet is so cluttered with garbage transactions, noting will send out without a transaction creation failed error.

We consolidate them, and the instant the coins LEAVE our wallet (IE: they stop staking) we stop getting blocks.

I'm going to delete the blockchain, once. If that doesn't fix the issue (not sure why it would, this will be the third time), the coin is off AllCrypt.

And a better question: With no blockchain movement, how the hell are we still getting tiny 0 confirmation deposits from a bunch of users? We're talking like 100-200 CCX deposits, that appear to be coming from a mining pool?

You can't mine if the blockchain isnt moving.

What is going on?

And we've been stuck at 251668 since I made the last 70million of sends out of the wallet.

The block explorer posted is completely broken.

I have a user on my site who barely speaks english submitting a new support ticket every 30 minutes asking me where his withdrawal is - the latest post is linking to something 7 pages back about downloading the blockchain, all the while ignoring every reply our support staff sends him that the problem is on HIS end.

*IF* we are on the right chain, we are the only ones moving it. And now that we have 0 coin age, and a huge number of coins not fully confirmed, we wont be moving the blockchain anytime soon.

Missing coin dev, dead blockchain... Not trying to be negative, but this is frustrating as heck.

On the plus side, I think we just discovered the POS version of a multipool pushing your difficulty through the roof then taking off. All coins that can mine are unconfirmed, thus, cannot mine themselves to push themselves to confirming.

Is no one actually mining this coin? Or rather, since it's pure POS - does no one have enough of this coin to actually move the blockchain?!

Here's why I ask. And I'm starting to get annoyed at this situation.

CCX was cluttered with thousands of micro unspent inputs. It was causing sends to take forever, and CCX was slamming 100% CPU all the time.

So we wrote a script to collect all those microtransactions and batch them up and 'clean up' the wallet. The INSTANT we did this, we died. None of the transactions we sent got any confirmations, and our 500million+ CCX wallet was left with about 2 million coins in it, and no new blocks for a few days.

So tonight, I waste 2 hours of my time deleting all the 0 confirmation transactions. I then dump all the private keys. I then make a new wallet, import the keys, and scan the blockchain.

Suddenly, we're getting new blocks again. Woohoo!

Ok, so, not wanting to lose all the transaction info in the OLD wallet, I load up the old wallet, create a new deposit address, and shut down. Start the new wallet, and in batches, start moving the CCX back into an address in the OLD wallet. After it's all moved out, I shut down, load the old wallet up. Run our auditing, and we're short about 70 million CCX. WTF? When I ran the audit after importing the keys, we were good.

So, shut down.

Bring up the newer, just sent all the coins out wallet.

About 70 million in 0 confirmation sends.

IT STOPPED CONFIRMING BECAUSE I MOVED THE COINS OUT.

So, Apparently, AllCrypt.com owns, I'd imagine, about 95% of the network/POS hash of CCX. Because - the instant I 'sent' all our coins - the blockchain stopped moving.

Best part? The coins that DID make it through? Some of it confirmed. Some is halfway there. NONE OF IT has any coin age so that it's worth crap to stake with.

Moneysupply says there 3.5 billion CCX out there. Whoever has it has their wallet turned off. Because the chain dies when I move the coins. Sure hope the minimum coin age isnt too high, because the amount of confirmed, CAN STAKE coins in our wallet is now miniscule, until someone turns a wallet back on and gets the blockchain moving.

Thats astounding.
 

No blocks in 24 hours. I'm gonna go with 'its dead'.

Real tired and read over the last few posts... so going to try and make sense here....

So that TXID I was seeing is the TXID of the send I'm attempting to do, correct? Thats some poor code then because the way it reads is that THAT  is the txid was already spent. I screwed with the code trying to figure this out, and added 'prevout.hash.ToString().c_str()' to that line thats printed to the debug.log, ran a createrawtransaction using a TXID that absolutely, for sure, was reported in listunspent, it failed and bombed out, and in the debug.log, the new  prevout.hash.ToString().c_str() I added to the code reported the "bad" txid that it's saying is already spent.

So I tried this with a few other CPU hogging wallets. Two others I had the same issue. It bombed out, reporting an input was already spent when listunspent was reporting it was available.

One of the coins I tried it on (cornerstonecoin CCX) had not flushed it's debug.log in ages (file size was 200+ megs), and I ran my code against it. First try collecting 876 inputs into a 100kb transaction failed. So I kept halving it, getting successes, then halving again on failures, until I had a 4 input transaction that failed, and upon failure, I had the script dump me the txids of the transactions it was trying to use.

I then grep'ed the debug.log for them. One was not in there. Two were the same txid, with different vouts, and the 3rd was unique. So I had 2 left. Both hit in there with this line (different txids obviously)

WalletUpdateSpent found spent coin 43.974794CCX fc1e5b508d66bc4c7f964ff0df195ed949d6a43e82b795b78a3dff195a6d30e2

Which is exactly the line that appears when I successfully SEND using sendrawtransaction. (Or any send for that matter).

Unfortunately they are not timestamped so I cannot see WHEN that was sent, but it appears that listunspent is for sure showing me inputs that HAVE already been spent. Unfortunately I've already restarted the CCX wallet and it wiped the debug log, so I cant estimate how far back the inputs had been spent. I'm now trying to restart litecoinplus with -rescan to see if that transaction that was hanging me up earlier is now recognized as spent.

What is a major pain in the ass is there is no way that I'm aware of to use the RPC to see if a previous input has been spent. My script may need to just bomb out and keep trying, getting smaller and smaller until it can flag a small subset of txids as already spent.

Just checked it - the coin started up, and one of the txids which had shown as unspent before a -rescan is now NOT showing. Going to try this on that coin again and see if it bombs out. It for sure seems like some of these wallets are not correctly marking some inputs spent.

Is there a working block explorer somewhere?

I was testing consolidating all unspent inputs to reduce CPU time spent POS staking, and I think I somehow got banned for a double spend attempt (bug in the script didn't clear the TX array before building the array for the second send, so the same inputs were used again). We havent had a block in over an hour and have 1 connection. Everything else is timing out (no ban messages, just a ton of connection attempts then disconnects). Is there a way to see the ban list?

And I'd like to see a block explorer if there is one... one transaction I'm working with is problematic. It's listing that it's already been spent, but, listunspent is showing it just fine.

I've no clue. The blocks that show under 'Stake' in getinfo are all the immature staked blocks. I'm not sure how to see which blocks are being used FOR staking. I turned the reservebalance on but it's apparently not let go of that transaction, because I still cannot use it in a send. It's still showing in listunspent - so it hasnt been spent. But I cannot actually spend it, as it's said it's "used" already. Every time I try and send it, I get a DIFFERENT hash in the debug.log (this line: ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629) - the blockpos and txpos stay the same, but the hash is different each time. So I've no idea what it's referring to, at all.

Pulling out hair.

New issue: Trying to clean up another wallet it keeps failing on the send. Thought it was a fee issue, started looking at the code. Had this in the debug:

ERROR: ConnectInputs() : 4ad396f7f0 prev tx already used at (nFile=1, nBlockPos=53949471, nTxPos=53949629)

I went into the code and see that that 4ad39 is a substr of the first 10 characters. So I had it show the whole thing. It's the same length as a transaction ID, so I wrote all the txids (and the output from listunspent) before it tried to send. I thought maybe in the 2 seconds between listing the unspent, and the send, that it got used somehow.

Well that full string (middle part removed just in case it's part of a private key or god knows what else) 4ad396f7f0b3bdfc30f5f71ab21bxxxxxxxxxxxxxxxxxxxxaa36840c08fe8c93 isnt anywhere in the listunspent output. its not a block hash. Its not a txid in the wallet. No clue what it is. The whole nBlockPos and nTxPos isnt helpful either.

Anyone have a clue what the heck that is referring to? Maybe the wallet is broken and it's listing already-spent inputs erroneously in the listunspent? If I drop the # of transactions batched together down to a smaller number I have some luck.

Another edit: I'm pretty sure this coin is just broken. I reduced the max # of TX that can be batched together to 10, and now, the first 10 go, but when it tries the second 10, it bombs with a 'TX rejected' and there's absolutely nothing in the debug log. Not a size issue (it was 2342 bytes), not a fee issue (fee correctly added). I even added a 3 second wait before starting the next one.

Next thing I'm gonna try is grabbing 10 random inputs to use instead of the 10 smallest value inputs and see what happens.

ANOTHER edit: I'm thinking maybe its because the TX is being used for staking? I know if a wallet is staking coins you can't send them... I isolated one of the 'problem' transactions. Just have no idea how to see which coins the wallet is using for staking at the moment. Hrm

Ok. So then. Lets back this up a bit. I know a far amount about mining coins - I was looking into setting AllCrypt up as a pool/exchange and trying to figure out how to merge mine coins, and when doing that, I learned a lot about how mining works. Super short: There is a target, which is based off the difficulty. Miners slam random numbers through the algo, and see if it beats the target number. If so, you solved the block, submit it.

How the hell does POS work then? All POS coins are "advertised" as you get a % return per <time period> but this is obviously done by internal mining.

What's the process? It's based on the coins you have... how? It cant be CPU based (CPU mine as normal, if you hit the block your reward is percentage based) because then it could be cheated and someone with more CPU could mine more POS blocks.

So it's the coins themselves that "mine". How? What exactly is being hashed and compared to the target? Every coin you have? Every unspent input? Every address that has coins? What determines what is hashed? Because since you have finite coins in your wallet, you only have a finite number of tries to mine a block. If it's not finite, then it's CPU based and more CPU = more mining = If I have 10,000 coins and you have 10,000 coins and you have a faster CPU you'll mine more than me. So, can't be that. (Or is it that, but the rewards are throttled?)

I want to fix this issue. At least in the wallets on my site. And if it's an elegant solution, hell, maybe it's the next big thing. Lots of coins feature KGW or DigiShield. Not bad advertising to feature something our exchange came up with.

Again - I've googled everything I can think of trying to find a whitepaper or reference, and nothing gives me the info I'm looking for. Combing the code is ungodly painful when I dont even know what I'm looking for. Thats why I tried profiling... see what's chewing up the CPU. But, the results were worthless (see my post a few back).

Prelim findings: Success (sort of)

So I took one of our worst offenders, CoolCoin. I spent all afternoon learning a hell of a lot about creating raw transactions, signing them, transaction size, required fees, etc, I wrote an admit tool that checks a coin and groups up all the tiny unspent inputs, batches them into huge transactions, and sends them back to a new address in the wallet.

I reduced the # of unspent inputs on CoolCoin from 7667 to 85. The CPU usage instantly dropped from 99.8% to 32.9%.

I consider it a partial victory though, because as it continues to stake and we get deposits, the inputs will grow again. In addition, JackpotCoin, a coin that currently has 2432 unspent inputs, is using 34.9% CPU. 30x the number of inputs to check for staking, and using the same amount of CPU time. LOCK has twice the number of unspent inputs, and is almost always below JPC's percentage by at least 5-10%.

So there is absolutely something "wrong" with some of these coins. Going to keep searching. Collecting all the inputs together isn't the best solution. Going to read through the links posted here when I have some more time. Didn't expect to spend my afternoon fixing this mess (but, I learned a hell of a lot of cool new stuff. Might even help AllCrypt actually calculate real transaction fees if we implement internal transaction creation and batching - Not in a hurry to get your withdrawal? Let us batch it up with others, save on fees! Or something similar).

But I'd love to hear further ideas of the causes (and where to look in the code) of where the staking is happening. Just from my own coding experience, it seems like a runaway thread. Like JPC is well coded and only stakes when it needs to, and CoolCoin is a spaztic kid running around screaming "GIMMIE POS COINS WOOWOO!!!1!1!!one!eleven!1!"

Going to take one of the offenders and try consolidating all the unspent outputs. One of the coins, litecoinplus, is using 99.8% CPU right now, and is one of our low volume coins. It has 5609 unspent outputs. Not sure if thats an absurdly high number - we've never bothered to look at things like that.

That being said, Jackpotcoin, one of our most popular coins, is on the same server as litecoinplus, is using 30% CPU, and has... wow. 2411 unspent. I would have sworn it was higher, given it's a higher traffic coin. But, I guess with more withdrawals those unspent outputs are being cleaned up a lot more often.

Going to clean up XLC and post the results. If it works, maybe the key is to just run some cleanup maintenance whenever the unspent inputs reach a certain threshold.

It's a bunch of them. Some we turned POS on work nicely (Lock, JPC, some others) but many just kill the CPU.

Havent checked the unspent outputs, and its 11:22pm here, so, not at the servers to check easily.

Nope. Linux box. No VMS. Here's a snippet of the gprof output we got after running it against the gmon.out:
Flat profile:

Each sample counts as 0.01 seconds.
 no time accumulated

  %   cumulative   self              self     total          
 time   seconds   seconds    calls  Ts/call  Ts/call  name    
  0.00      0.00     0.00 2733398524     0.00     0.00  xor_salsa8(unsigned int
  0.00      0.00     0.00 354052679     0.00     0.00  operator<(uint256 const&
  0.00      0.00     0.00 41734266     0.00     0.00  GetStakeModifierSelection
  0.00      0.00     0.00 28480211     0.00     0.00  void WriteCompactSize<CHa
  0.00      0.00     0.00 19838644     0.00     0.00  Shutdown(void*)
  0.00      0.00     0.00 13992220     0.00     0.00  std::_Rb_tree<uint256, st
  0.00      0.00     0.00 13990945     0.00     0.00  CMerkleTx::GetDepthInMain
  0.00      0.00     0.00 13990786     0.00     0.00  CMerkleTx::GetDepthInMain
  0.00      0.00     0.00 12933773     0.00     0.00  CScript::GetOp2(__gnu_cxx
  0.00      0.00     0.00 11657139     0.00     0.00  CDataStream::read(char*,
  0.00      0.00     0.00 10263946     0.00     0.00  std::vector<unsigned char
  0.00      0.00     0.00  9092240     0.00     0.00  CWalletTx::IsSpent(unsign
  0.00      0.00     0.00  8314437     0.00     0.00  void std::vector<char, ze
  0.00      0.00     0.00  7166960     0.00     0.00  operator<(uint160 const&,
  0.00      0.00     0.00  5848572     0.00     0.00  GetTime()
  0.00      0.00     0.00  4991290     0.00     0.00  CTransaction::IsFinal(int

It's all 0% (And we let the wallet run for 30 mins before shutting it down and analyzing the results)

Very odd. Spent an hour googling why gmon would give 0% on everything.

So, if that is the case, it might behoove us to collect all those small transactions and group them into one huge send back to the wallet to clean them up? I can see how that would get exponentially worse... 300 small transactions lead to 300 small POS rewards which then leads to 600 small transactions leading to 600 smaller POS rewards, which turn into 1200 small POS rewards, etc...

Given we are an exchange with tons of small deposits (from mining pools) we have a much larger than usual number of transactions/addresses in the wallet than 'normal' wallets do. Tomorrow I'll take one of the offending coins and try consolidating all of the transactions into one address and see what that does to the CPU time for the wallet.

Thanks for the tip.

That being said, JPC has been one of our most active coins. It's still one of the most polite ones as far as CPU time and staking.

We actually recompiled one wallet with profiling and analyzed it with gprof, and for some reason all the time %'s were listed as 0. 0% time for everything. Tried it with another tool and it ran so slow, after an hour it still hadn't finished scanning/validating the blockchain. Figured we'd ask here and see if we could find what the issue was in the code itself.