You're reminding me how spoilt I am
Well, your namesake certainly was  about 40 minutes to scan
I haven't timed it exactly, but it's somewhere around there even with my HDD. I'll pay closer attention next time I do a scan. |
|
|
|
Another little observation is: we're running to stand still, in performance terms.
For me anyways, 0.94 allows me to place the Armory DB on an SSD whereas before both it and the Bitcoin DB were on an HDD. Practically speaking, this has hugely improved the initial "reading blocks" and "building database" steps, down to about 14 minutes. The tx scanning phase is only slightly faster IIRC (possibly because the Bitcoin DB block files are still on HDD). In short, when 0.94 works, it's brilliant! |
|
|
|
Not typical of the first crashes, but there's probably something useful in here.
That stack trace looks exactly the same as mine on Windows (some of the line #s are off, but I think that's just a symptom of using different debuggers). Even the most recent calls are showing a similar out-of-bounds pointer being passed. FYI I've had one additional crash, and the stack trace was the same (with pb.fmp_.current_->filemap_ and pb.fmp_.current_->mapsize_ in GrabThreadData::pullBlockAtIter() corrupted). |
|
|
|
First-chance exception at 0x000007FEDFD63450 (_CppBlockUtils.pyd) in python.exe: 0xC0000005: Access violation reading location 0x0000000088067A8D.
On Win 7 64-bit, I'm also getting an access violation while scanning txs. Running the latest in the ffreeze branch (578c072). Here's a stack trace (via VS 2013) from a debug build: _CppBlockUtils.pyd!memcpy() Line 356 Unknown
_CppBlockUtils.pyd!BinaryData::copyFrom(const unsigned char * inData=0x0000000088067a8d, unsigned __int64 sz=80) Line 198 C++
_CppBlockUtils.pyd!BlockHeader::unserialize(const unsigned char * ptr=0x0000000088067a8d, unsigned int size=80) Line 31 C++
_CppBlockUtils.pyd!BlockHeader::unserialize(const BinaryDataRef & str={...}) Line 48 C++
_CppBlockUtils.pyd!BlockHeader::unserialize(BinaryRefReader & brr={...}) Line 54 C++
_CppBlockUtils.pyd!BlockHeader::BlockHeader(BinaryRefReader & brr={...}) Line 52 C++
_CppBlockUtils.pyd!PulledBlock::unserializeFullBlock(BinaryRefReader brr={...}, bool doFrag=true, bool withPrefix=false) Line 230 C++
> _CppBlockUtils.pyd!GrabThreadData::pullBlockAtIter(PulledBlock & pb={...}, LDBIter & iter={...}, LMDBBlockDatabase * db=0x00000000003baa10, BlockFileAccessor & bfa={...}) Line 490 C++
_CppBlockUtils.pyd!GrabThreadData::grabBlocksFromDB(std::shared_ptr<LoadedBlockData> blockData={...}, unsigned int threadId=2) Line 279 C++
_CppBlockUtils.pyd!std::_Bind<1,void,void (__cdecl*const)(std::shared_ptr<LoadedBlockData>,unsigned int),std::shared_ptr<LoadedBlockData>,unsigned int>::_Do_call<,0,1>(std::tuple<> _Myfargs={...}, std::_Arg_idx<0,1> __formal={...}) Line 1150 C++
_CppBlockUtils.pyd!std::_Bind<1,void,void (__cdecl*const)(std::shared_ptr<LoadedBlockData>,unsigned int),std::shared_ptr<LoadedBlockData>,unsigned int>::operator()<>() Line 1138 C++
_CppBlockUtils.pyd!std::_LaunchPad<std::_Bind<1,void,void (__cdecl*const)(std::shared_ptr<LoadedBlockData>,unsigned int),std::shared_ptr<LoadedBlockData>,unsigned int> >::_Run(std::_LaunchPad<std::_Bind<1,void,void (__cdecl*const)(std::shared_ptr<LoadedBlockData>,unsigned int),std::shared_ptr<LoadedBlockData>,unsigned int> > * _Ln=0x0000000007d1d060) Line 196 C++
_CppBlockUtils.pyd!std::_LaunchPad<std::_Bind<1,void,void (__cdecl*const)(std::shared_ptr<LoadedBlockData>,unsigned int),std::shared_ptr<LoadedBlockData>,unsigned int> >::_Go() Line 188 C++
_CppBlockUtils.pyd!_Call_func(void * _Data=0x000000000f8b18b0) Line 28 C++
_CppBlockUtils.pyd!_callthreadstartex() Line 376 C
_CppBlockUtils.pyd!_threadstartex(void * ptd=0x000000000f8b18b0) Line 354 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown
I think there's something going on near GrabThreadData::pullBlockAtIter(); here's a locals dump after switching to that frame: - bdr {ptr_=0x0000000088067a8d "" nBytes_=155478 } BinaryDataRef
+ ptr_ 0x0000000088067a8d "" const unsigned char *
nBytes_ 155478 unsigned __int64
- pb {stxMap_={ size=0 } nextBlock_=empty fmp_={current_=shared_ptr {lastSeenCumulated_={...} fetch_=FETCH_NONE (0) filemap_=0xec27b9f8692f2700 <Error reading characters of string.> ...} [1 strong ref] [default] ...} } PulledBlock &
+ DBBlock {dataCopy_={data_={ size=0 } } thisHash_={data_={ size=0 } } numTx_=4294967295 ...} DBBlock
+ stxMap_ { size=0 } std::map<unsigned short,PulledTx,std::less<unsigned short>,std::allocator<std::pair<unsigned short const ,PulledTx> > >
+ nextBlock_ empty std::shared_ptr<PulledBlock>
- fmp_ {current_=shared_ptr {lastSeenCumulated_={...} fetch_=FETCH_NONE (0) filemap_=0xec27b9f8692f2700 <Error reading characters of string.> ...} [1 strong ref] [default] ...} FileMapContainer
- current_ shared_ptr {lastSeenCumulated_={...} fetch_=FETCH_NONE (0) filemap_=0xec27b9f8692f2700 <Error reading characters of string.> ...} [1 strong ref] [default] std::shared_ptr<FileMap>
- [ptr] 0x000000005ad12f50 {lastSeenCumulated_={...} fetch_=FETCH_NONE (0) filemap_=0xec27b9f8692f2700 <Error reading characters of string.> ...} FileMap *
- lastSeenCumulated_ {...} std::atomic<unsigned __int64>
fetch_ FETCH_NONE (0) FILEMAP_FETCH
+ filemap_ 0xec27b9f8692f2700 <Error reading characters of string.> unsigned char *
mapsize_ 15458524216405398507 unsigned __int64
fnum_ 82 unsigned short
+ [deleter and allocator] default std::_Ref_count_base
+ [Raw View] 0x000000004186a2f8 {...} std::shared_ptr<FileMap> *
- prev_ 0x00000000154bf8e0 shared_ptr {lastSeenCumulated_={...} fetch_=FETCH_ACCESSED (2) filemap_=0x0000000021b70040 "ù¾´Ù\x1ef\x2" ...} [941 strong refs] [default] std::shared_ptr<FileMap> *
- [ptr] 0x0000000069977950 {lastSeenCumulated_={...} fetch_=FETCH_ACCESSED (2) filemap_=0x0000000021b70040 "ù¾´Ù\x1ef\x2" ...} FileMap *
+ lastSeenCumulated_ {...} std::atomic<unsigned __int64>
fetch_ FETCH_ACCESSED (2) FILEMAP_FETCH
- filemap_ 0x0000000021b70040 "ù¾´Ù\x1ef\x2" unsigned char *
mapsize_ 134020078 unsigned __int64
fnum_ 81 unsigned short
+ [deleter and allocator] default std::_Ref_count_base
+ [Raw View] 0x00000000154bf8e0 {...} std::shared_ptr<FileMap> *
+ iter {iter_={db_=0x00000000003baaa0 {env=0x000000000038f000 {dbenv=0x000000000581ed80 {...} threadTxMutex_=...} ...} ...} ...} LDBIter &
+ db 0x00000000003baa10 {baseDir_="C:\\Users\\Chris\\AppData\\Roaming\\Armory\\databases" genesisBlkHash_=...} LMDBBlockDatabase *
+ bfa {blkFiles_=shared_ptr { size=270 } [3 strong refs] [default] blkMaps_={ size=2 } lastSeenCumulative_=...} BlockFileAccessor &
fnum 82 unsigned short
+ brr {bdRef_={ptr_=0x0000000065ebbb90 "R" nBytes_=16 } totalSize_=16 pos_=16 } BinaryRefReader
offset 555597 unsigned __int64
size 155478 unsigned int
First off, fnum, offset, and size look good. I verified that they point to the beginning of block 258860, and that the block file doesn't appear corrupted. At the very least, the merkle root is correct. Here are two things that pop out at me though. pb.fmp_.current_->filemap_ and pb.fmp_.current_->mapsize_ are clearly invalid, perhaps something is overwriting them? IIUC, bdr.ptr_ is calculated from filemap_ and offset, and so bdr.ptr_ is invalid, and it's what eventually gets passed to memcpy() which causes the access violation. Another thing that looks strange is that I thought (I could be wrong) that bdr.ptr_ == filemap_ + offset, but in the locals above it's not (or multiple things are getting overwritten?). I'm not quite sure where to go from here.... whatever happened, it may be too late at this point in the debug to find. |
|
|
|
Anyhow, I had about 9 HD accounts, lost my Trezor device, so tried: 1. Recovering with Electrum using 24-word seed. Result: All 9 accounts showed in Electrum, however, I could not send any of the BTC - it kept asking me to plug in my Trezor, which is gone. (FAIL). 2. Recovering with MultiBitHD using 24-word seed. Result: Only HD Account #2 showed up in GUI, the acccount with least BTC of course. Rest of the HD accounts showed, but all with zero balances (Yes I let it fullly sync). (FAIL) 3. New install of Mycelium on Android using 24-word seed. Result: Only HD Account 1 was available (FAIL). So - Luckily, after physically FINDING my Trezor (THANK GOD)!, I was able to move the BTC to a temp wallet on Mycelium, wipe the trezor, now I just use one HD account the trezor (Account 1), and hope my new seed is recoverable if this happens again - it was a somewhat sleepless night to be honest. I may buy a spare Trezor now, I love it, but that was ridiculous, I could not recover all my HD accounts no matter what I tried , on multiple PC's.  Glad you found it! Electrum is definitely not compatible (and as jim618 already mentioned, MultiBit HD only works with the first account). The three wallets which MZ mentioned above should have worked, even with multiple accounts. If you're missing an account in one of those wallets, you could have tried manually adding new accounts and it should pick up the transactions in the accounts as you add them. If it doesn't, I'd consider it a bug... |
|
|
|
Conservationism here is justified especially so in that one can simply set the key-pool to an arbitrarily high value and obtain most of the values of the fancier schemes; without many of of their costs.
I think a reasonable compromise would be to have wallets refuse to use new keys without explicit authorization. Do you mean as a stop-gap prior to implementing an HD wallet (specifically in Bitcoin Core), or in lieu of implementing it [HD]? |
|
|
|
Cheers Chris for all the help you've provided - I've sent a little something over to the wallet address on github as a donation (0.06btc). Its not much and it certainly won't compensate you for all the time you've put into developing that fantastic tool, but I hope it goes some way to saying thanks.
That's very generous of you (especially considering I haven't actually helped yet  )! The amount doesn't matter (much  ), the sentiment absolutely does! Thank you!! I've resorted to using password lists. Your program seemed to manage that a bit better than using the token programme and six full willcards (90^6 possibilities there or thereabouts) where it naturally made my desktop collapse.
If you're dealing with really big sets, there are a few things that can help (if you haven't noticed them already). - the "--no-dupchecks" command-line option: saves a lot of RAM. Doing duplicate checking isn't very effective for MultiBit Classic .key files anyways.
- the "--max-tokens #" command-line option: if you limit the max # of tokens per guess, it can drastically reduce the # of permutations.
- the "--threads #" command-line option: the # defaults to the number of cores your CPU has, but sometimes using number_of_cores + 1 is a bit faster.
I was thinking that next for me might be just exploring all possibilities for 8 lower case letters - although i think thats still 200 billion which is beyond what my desktop seems to take
That number sounds right. My Desktop (an c. 2012 quad-core) would take about a week to run through that. Painful, but not impossible IMO. (I presume a memory problem??). With password lists I seem to be able to do about 50 or 60 million before I get a crash. If you have any suggestions for optimisation, I'd welcome them  If you're running into memory issues or crashing, definitely use the "--no-dupchecks" command-line option I mentioned above. If there's some other sort of crash, could you post the full error message you're getting? |
|
|
|
Thanks for the suggestion but that doesn't help if my copy of electrum is compromised. The program might ignore the entropy I provide and make a predictable seed instead.
I'm looking for a way like I described in my first post where you don't even need to use a computer to generate the seed (use the physical world only).
Got it. You can actually use the same method as in the first post you referenced, with a few small changes. 1. Use at least 13 words (more is OK too). 2. Enter the result into Electrum 2.x to see if the result is valid (via a wallet restore). If it's invalid, the button to continue will be grayed out. 3. Use one of these two methods to modify your seed, and return to step 2 to see if it's valid: - Add a 1 digit to the end (with or without a space, doesn't matter), or increment that digit.
- Re-roll the last word.
On average, you need to perform step 3 about 130 times before before you end up with a valid Electrum 2.x seed, however it's possible that you could need fewer or many more. For example, there's about a 14% chance you'll need to do step 3 500 or more times.... Also, be sure to take note of my response in that other thread, which remains important for generating an Electrum 2.x seed: you need a deterministic way to decide which die is #1, which is #2, etc. For example, you could roll them each one at a time, or you could use six different colored dice with each color always representing the same die #, or you could just always read the dice from left-most to right-most however they happen to fall (easy to do objectively if you have Travel Yahtzee). If you don't have some such deterministic method, you will almost certainly introduce bias as you read off the dice in your own personal order. |
|
|
|
Mycelium does not allow to export single keys from HD accounts, if you really really want to do that and perfectly understand what you are doing, you will need to use an external tool for that.
As I'm sure trasla and the other Mycelium devs know, exporting a single private key has hidden dangers which can lead to a loss of some or even all funds stored in the account. The conscience decision to not support this feature is a very wise one. EstefanTT, if you're not familiar with using external tools as trasla mentioned, you're likely not aware of the dangers involved (so just don't do it ) |
|
|
|
Some of you guys are mean and have no feelings Grow up. This is a forum for adults. The first mildly offensive thing fabiola! has posted amongst a sea of puerile adolescent testosterone, and you think fabiola! is out of line?! Wow.... Welcome to the boy's club, fabiola. I didn't post she was out of line. I didn't think what she posted was offensive. I'm simply telling her this is not a PG forum, and if she can't handle it, she shouldn't come here. Fair enough, but "Grow up" is about as offensive as "you guys are mean". IOW, it looked like shots returned. You could have just said "don't feed the adolescents" |
|
|
|
Some of you guys are mean and have no feelings Grow up. This is a forum for adults. The first mildly offensive thing fabiola! has posted amongst a sea of puerile adolescent testosterone, and you think fabiola! is out of line?! Wow.... Welcome to the boy's club, fabiola. |
|
|
|
Shorter validation time for one transaction blocks is expected for miners
It is unclear to me what you are talking about. I believe one thing valiron is talking about is the practice (which I recall reading about, I've no idea if it's still in use, or ever has been) of pool operators sending out work for empty blocks to their miners immediately after a new block is found, to get the miners working on the new chain ASAP, and then creating a non-empty block & merkle tree at their leisure and updating miners once it's ready. |
|
|
|
So I finally had a chance to give btcrecover a go tonight and all I can say is wow wow wow. I tested it with an encrypted empty multibit wallet password 123jonnyboy456 and the token %0,3djonnyboy%0,3d and the script found that password in a flash, so I know that your program clearly works, and damn quickly too.
Yeah... unfortunately, MultiBit Classic is one of only two popular wallets that uses no key stretching to improve brute-force resistance (well, I suppose that's "fortunately" for you ). Electrum is the other one. MultiBit HD (still in Beta) for the most part fixes this. It also helps that btcrecover by default spawns one worker process per CPU core on your machine. One thing I can't understand is how you do better than openssl. If the characters of the private key are (as I believe they should be) uniformly distributed with the exception of the padding on the end, how can you ever tell an honest decryption from a random decryption?
Encoded in binary, 32 out of the 33 bytes of a private key are uniformly distributed (the first is typically a flag byte to indicate whether or not the corresponding public key is compressed). However, MultiBit chooses to write the private keys into a .key file not in a binary format, but in a WIF-encoded format (which is then stored in plain ASCII). (This is to maintain compatibility with Bitcoin Core's import/export format.) It's easy to check for such a key, and extremely unlikely for such a correctly formatted key to appear at the beginning of an incorrectly decrypted .key file. It's so unlikely, that's why btcrecover only bothers decrypting the first 16-byte block. I'll be around tonight (it's now early evening where I live), so let me know if you have any questions! -Chris |
|
|
|
but it [Electrum] is not the safest because you have to trust someone else to give you the right data.
FYI Electrum implements SPV—Electrum servers cannot "invent" non-existent transactions. Electrum servers could withhold one more transactions, however because there are multiple servers operated by (allegedly) independent parties, multiple parties would have to collude to pull this off (or someone would have to control your Internet connection). |
|
|
|
In all the replies to the OP, I seem to not see anything mentioned about Xapo or Coinbase....hmmmm. Are these not good choices?
No. You can't access private keys when using Xapo and Coinbase but I think you can access private keys if use Coinbase vault. +1 for MZ's answer. However, Coinbase does have a new multisig option (which isn't the default, you have to go looking for it). I haven't checked it out yet, but if it's what it says it is, I'd consider it a good option if you use their multisig wallet, and only that. |
|
|
|
I'm not certain, but I don't think you can adjust the merkle root without needing to recompute the midstate as well?
You're right, the merkle root spans both SHA-256 (512-bit) blocks, so you must recalc the entire SHA-256 from scratch. | Byte len | Byte pos | Bit pos | SHA block | Field | | 4 | 0 | 0 | 1 | version | | 32 | 4 | 32 | 1 | previous block header hash | | 32 | 36 | 288 | 1-2 | merkle roothash | | 4 | 68 | 544 | 2 | time | | 4 | 72 | 576 | 2 | nBits | | 4 | 76 | 608 | 2 | nonce | | 80 | 640 | | (end) |
|
|
|
|
in the 22th position of your password
Electrum handles passwords with non-ASCII fairly well. (It never crashes, but it (as well as most wallets, even Bitcoin Core) has a problem in that it doesn't normalize them.) It also seems to handle non-ASCII wallet file names OK too. It doesn't handle non-ASCII usernames, but when I tested it, it just crashed on startup, so it didn't behave the way bitokman described. I couldn't figure out exactly what was wrong. If bitokman responds with their operating system, I'll try to describe how to run Electrum from a terminal in order to get a more specific error message. |
|
|
|
|
Show Posts
