Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: findftp on November 09, 2015, 11:45:01 PM



Title: Retrieve lost satoshis
Post by: findftp on November 09, 2015, 11:45:01 PM
So I was playing around a bit and lost some relative small amount of bitcoin
I used a certain method to create the public key, but I don't have it anymore.
Now I dug into python (which btw is new to me) and was able to produce a script which is able to give me a few thousand (perhaps few million) addresses of which one of them is the one I used.
In other words, I'm able to brute-force the private key because I know the pattern I used.

I exported first 200 addresses with my script and tried to import them into multibit classic.
This works, but is very slow.

Is there any other (more efficient) way to check balance of the keys?


Title: Re: Retrieve lost satoshis
Post by: achow101 on November 09, 2015, 11:46:07 PM
You could write a script which just queries an api like blocktrail's to get the balance of all of the addresses.


Title: Re: Retrieve lost satoshis
Post by: spin on November 10, 2015, 08:39:15 AM
If you are running bitcoin core you can import it there.  If you are importing many addresses you can import them with the following rpc call:
Code:
importprivkey	<bitcoinprivkey> [label] [rescan=true]	
Adds a private key (as returned by dumpprivkey) to your wallet. This may take a while, as a rescan is done, looking for existing transactions. Optional [rescan] parameter added in 0.8.0. Note: There's no need to import public key, as in ECDSA (unlike RSA) this can be computed from private key

What you could do to speed this up is set rescan=false so that it doesn't rescan for every address.

I think you can then just restart bitcoin core to rescan all?



Title: Re: Retrieve lost satoshis
Post by: amaclin on November 10, 2015, 09:34:06 AM
Is there any other (more efficient) way to check balance of the keys?
Yes, there are.
You can parse the whole blockchain and create a database of addresses with non-zero balances
The size of such database is not too big.


Title: Re: Retrieve lost satoshis
Post by: tspacepilot on November 10, 2015, 05:13:10 PM
If you are running bitcoin core you can import it there.  If you are importing many addresses you can import them with the following rpc call:
Code:
importprivkey	<bitcoinprivkey> [label] [rescan=true]	
Adds a private key (as returned by dumpprivkey) to your wallet. This may take a while, as a rescan is done, looking for existing transactions. Optional [rescan] parameter added in 0.8.0. Note: There's no need to import public key, as in ECDSA (unlike RSA) this can be computed from private key

What you could do to speed this up is set rescan=false so that it doesn't rescan for every address.

I think you can then just restart bitcoin core to rescan all?

Or just write rescan=true on the last one.  findftp, did you get this working?


Title: Re: Retrieve lost satoshis
Post by: findftp on November 10, 2015, 08:01:03 PM
If you are running bitcoin core you can import it there.  If you are importing many addresses you can import them with the following rpc call:
Code:
importprivkey	<bitcoinprivkey> [label] [rescan=true]	
Adds a private key (as returned by dumpprivkey) to your wallet. This may take a while, as a rescan is done, looking for existing transactions. Optional [rescan] parameter added in 0.8.0. Note: There's no need to import public key, as in ECDSA (unlike RSA) this can be computed from private key

What you could do to speed this up is set rescan=false so that it doesn't rescan for every address.

I think you can then just restart bitcoin core to rescan all?

Or just write rescan=true on the last one.  findftp, did you get this working?

Not yet.
I'm investigating several options.
I went blocktrail API method, but the API gives errors
I installed armory and it imported the keys much faster but it's currently still 'building databases' probably because I just installed it.
I read a bit about the RPC call method, but I'm not sure yet.

I did a bit of calculation and to cover all possible addresses of which one of them contain my satoshis I have to import about 400.000.000 addresses.
This probably a bit too much for a wallet.
I think I can better check each address against a UTXO database, but how to proceed?



Title: Re: Retrieve lost satoshis
Post by: amaclin on November 10, 2015, 09:31:42 PM
I think I can better check each address against a UTXO database, but how to proceed?
1) Create a database [address] -> [balance]
2) Check each of your 400m addresses


Title: Re: Retrieve lost satoshis
Post by: findftp on November 10, 2015, 10:38:50 PM
I think I can better check each address against a UTXO database, but how to proceed?
1) Create a database [address] -> [balance]
2) Check each of your 400m addresses

I don't have a clue how to build that database.


Title: Re: Retrieve lost satoshis
Post by: amaclin on November 11, 2015, 04:11:50 AM
I don't have a clue how to build that database.
1) Hire a programmer
or
2) Learn programming

I can help you with both items. Ready to pay me?


Title: Re: Retrieve lost satoshis
Post by: findftp on November 11, 2015, 10:00:11 AM
I don't have a clue how to build that database.
1) Hire a programmer
or
2) Learn programming

I can help you with both items. Ready to pay me?


I'm busy doing number 2.
Just started to learn python from scratch and already made the script to produce the patterns and hash them to compressed and uncompressed bitcoin public keys.
So far so good.

Now I (only) need the UTXO database and check each generated address against it.
Should I extract addresses with balance from the blockchain and put them in a sql database or something?
What would be the general approach for such a thing?



Title: Re: Retrieve lost satoshis
Post by: amaclin on November 11, 2015, 10:25:50 AM
Now I (only) need the UTXO database and check each generated address against it.
Should I extract addresses with balance from the blockchain and put them in a sql database or something?
What would be the general approach for such a thing?

Yes. You should parse blockchain and get the set of utxo.
The set is not very large.
https://statoshi.info/dashboard/db/unspent-transaction-output-set
There are ~32 mln unspent outputs today according to statoshi.info
It is quite possible to hold this set in memory in hashtable.
You can even use plain-text file for storing this data without using SQL engines.

Quote
I'm busy doing number 2.
Let us discuss number (1). I can create such engine for you. Not for free of course.


Title: Re: Retrieve lost satoshis
Post by: findftp on November 11, 2015, 10:34:03 AM
Now I (only) need the UTXO database and check each generated address against it.
Should I extract addresses with balance from the blockchain and put them in a sql database or something?
What would be the general approach for such a thing?

Yes. You should parse blockchain and get the set of utxo.
The set is not very large.
https://statoshi.info/dashboard/db/unspent-transaction-output-set
There are ~32 mln unspent outputs today according to statoshi.info
It is quite possible to hold this set in memory in hashtable.
You can even use plain-text file for storing this data without using SQL engines.

I think this is enough information for me to proceed further. Thanks for pointing me in this direction

Quote
Quote
I'm busy doing number 2.
Let us discuss number (1). I can create such engine for you. Not for free of course.

Well, I like your offer, but I only lost some satoshis. The amount of electricity I used to type this message is probably already worth more.
I like to get them back and learn from everything in the journey to reach that goal. I already learnt some python and want to see how far I can get on my own.
If in the end I'm not smart enough to build everything myself I probably won't hire someone else to do it for me. Unless those satoshis become valuable enough to do so.







Title: Re: Retrieve lost satoshis
Post by: spin on November 11, 2015, 11:06:13 AM
How many addresses do you have to check?  You mentioned 200 earlier?


Title: Re: Retrieve lost satoshis
Post by: amaclin on November 11, 2015, 11:16:06 AM
Well, I like your offer, but I only lost some satoshis.
The amount of electricity I used to type this message is probably already worth more.
I started two years ago on this forum with the absolutely same topic - finding the lost satoshis in the blockchain.
The only difference was that I did not lose them.

Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.

Quote
If in the end I'm not smart enough to build everything myself I probably won't hire someone else to do it for me.
Of course, it is economically unreasonable to spend 1 btc and find lost 0.0001 btc

Quote
Unless those satoshis become valuable enough to do so.
No chances.


Title: Re: Retrieve lost satoshis
Post by: findftp on November 11, 2015, 11:49:48 AM
How many addresses do you have to check?  You mentioned 200 earlier?

I did 200 addresses as a test to see if my script works.
In the end, my coins can be stuck among 5 to 10 million addresses but if they are not there, they are for sure among one out of 400 million addresses.


Title: Re: Retrieve lost satoshis
Post by: findftp on November 11, 2015, 12:03:07 PM
Well, I like your offer, but I only lost some satoshis.
The amount of electricity I used to type this message is probably already worth more.
I started two years ago on this forum with the absolutely same topic - finding the lost satoshis in the blockchain.
The only difference was that I did not lose them.
Precisley, but I know the syntax of the let's say brainwallet I lost.
I know the contents, but not the exact sequence of the possibilities.
Therefor I think I got a good chance.

Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

Quote
Quote
Unless those satoshis become valuable enough to do so.
No chances.

Since I'm a delusional bitcoin obsessionist I think a bitcoin could become worth a house.
In my mind they eventually become valuable enough for other people to retrieve them.
edit: It's not just a few satoshis, but it's less than a full bitcoin

But I also disagree for other reasons. For me they are already valuable enough to retrieve.
Value doesn't necessarily have to mean money.
Just like other people have hobbies which costs money.

I also do metal detecting as a hobby.
Spending hundreds of hours on a field in the hope I find a gold, silver or tarnished copper coin.
There are thousands of people doing the same thing and never earn back their metal detector in economical counter value.


But let's not have this discussion here, it doesn't belong in this subsection of the forum.




Title: Re: Retrieve lost satoshis
Post by: cr1776 on November 11, 2015, 02:44:04 PM
Quote
...
Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

It is a compiled language vs an interpreted one.  Compiled languages are almost always much more efficient than interpreted ones.

From a coding perspective, coding one may be much more efficient than the other depending on your skill set.  e.g. If you don't know Prolog or C++, but do know Python very well, it might be quicker to code and run everything in Python than to spend time learning one of them, then coding and running.


Title: Re: Retrieve lost satoshis
Post by: findftp on November 11, 2015, 02:50:57 PM
Quote
...
Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

It is a compiled language vs an interpreted one.  Compiled languages are almost always much more efficient than interpreted ones.

From a coding perspective, coding one may be much more efficient than the other depending on your skill set.  e.g. If you don't know Prolog or C++, but do know Python very well, it might be quicker to code and run everything in Python than to spend time learning one of them, then coding and running.

Ah, I already had a suspicion that it was due to compiling.
Makes sense.

I actually started programming a year ago, with arduino in C.
However, for this project I already found a lot written in python which I could use so I figured I could better go this way.
Perhaps I should try and find the same bits and pieces and redo the code in C.
I think I have more than enough homework for now to be able to continue  :)
Thanks for all the help.




Title: Re: Retrieve lost satoshis
Post by: tspacepilot on November 14, 2015, 07:14:27 PM
Quote
...
Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

It is a compiled language vs an interpreted one.  Compiled languages are almost always much more efficient than interpreted ones.

From a coding perspective, coding one may be much more efficient than the other depending on your skill set.  e.g. If you don't know Prolog or C++, but do know Python very well, it might be quicker to code and run everything in Python than to spend time learning one of them, then coding and running.

Ah, I already had a suspicion that it was due to compiling.
Makes sense.

I actually started programming a year ago, with arduino in C.
However, for this project I already found a lot written in python which I could use so I figured I could better go this way.
Perhaps I should try and find the same bits and pieces and redo the code in C.
I think I have more than enough homework for now to be able to continue  :)
Thanks for all the help.


If I'm not incorrect, the python interpreter is written in C, so yah, you're just adding a layer of abstraction when you use python.  However, for one-off scripts like this one, it's probably not worth it to worry about all the typing and memory management of C, just get yer script going so you can  have some fun.  Let us know how you're proceeding.  I, for one, appreciate your attitude.  I like doing things myself simply for the educational experience.


Title: Re: Retrieve lost satoshis
Post by: knightkon on November 21, 2015, 01:02:58 AM
I have to ask the simple question; is all of this work worth the small amount you have lost?  Would your time be better spent getting more rather than tracking a small amount down?  I just would think your process would be longer than replacing them through a faucet or something.


Title: Re: Retrieve lost satoshis
Post by: virtualx on November 21, 2015, 02:34:20 AM
Quote
...
Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

It is a compiled language vs an interpreted one.  Compiled languages are almost always much more efficient than interpreted ones.

From a coding perspective, coding one may be much more efficient than the other depending on your skill set.  e.g. If you don't know Prolog or C++, but do know Python very well, it might be quicker to code and run everything in Python than to spend time learning one of them, then coding and running.

Ah, I already had a suspicion that it was due to compiling.
Makes sense.

I actually started programming a year ago, with arduino in C.
However, for this project I already found a lot written in python which I could use so I figured I could better go this way.
Perhaps I should try and find the same bits and pieces and redo the code in C.
I think I have more than enough homework for now to be able to continue  :)
Thanks for all the help.


If I'm not incorrect, the python interpreter is written in C, so yah, you're just adding a layer of abstraction when you use python.  However, for one-off scripts like this one, it's probably not worth it to worry about all the typing and memory management of C, just get yer script going so you can  have some fun.  Let us know how you're proceeding.  I, for one, appreciate your attitude.  I like doing things myself simply for the educational experience.
good

The official Python interpreter is indeed written in C, you can find the source code on python.org. There are Python to C++ compilers but I have not tried them. I recommend using what works best for your project.


Title: Re: Retrieve lost satoshis
Post by: tspacepilot on November 21, 2015, 08:30:12 PM
Quote
...
Quote
Quote
I like to get them back and learn from everything in the journey to reach that goal.
I already learnt some python and want to see how far I can get on my own.
I prefer to use C++.
Why? Is it more efficient?

It is a compiled language vs an interpreted one.  Compiled languages are almost always much more efficient than interpreted ones.

From a coding perspective, coding one may be much more efficient than the other depending on your skill set.  e.g. If you don't know Prolog or C++, but do know Python very well, it might be quicker to code and run everything in Python than to spend time learning one of them, then coding and running.

Ah, I already had a suspicion that it was due to compiling.
Makes sense.

I actually started programming a year ago, with arduino in C.
However, for this project I already found a lot written in python which I could use so I figured I could better go this way.
Perhaps I should try and find the same bits and pieces and redo the code in C.
I think I have more than enough homework for now to be able to continue  :)
Thanks for all the help.


If I'm not incorrect, the python interpreter is written in C, so yah, you're just adding a layer of abstraction when you use python.  However, for one-off scripts like this one, it's probably not worth it to worry about all the typing and memory management of C, just get yer script going so you can  have some fun.  Let us know how you're proceeding.  I, for one, appreciate your attitude.  I like doing things myself simply for the educational experience.
good

The official Python interpreter is indeed written in C, you can find the source code on python.org. There are Python to C++ compilers but I have not tried them. I recommend using what works best for your project.

Right, but C++ is just C with classes.  You should be able to compile your C code with g++.  Anyway, I think this topic has drifted a bit from the original intention of this thread.  Let's leave off about interpreters here.  If suiyan wants to start another thread about intepreters, we can wait for that.


Title: Re: Retrieve lost satoshis
Post by: findftp on July 01, 2016, 02:50:24 PM
I got my satoshis back!  ;D
I just picked up this thing again and see where I could end up with noob python skills and new insights.
Estimated time to check all possibilities was 270 days but I found my key already within 20 hours!
I thought to let it run, and resume the script after optimizations to speed things up. I was not going to wait 270 days, but I was (extremely) lucky I guess.
My script does approximately 15 keys/second (4 instances, 60 k/s total) and checks against a cleaned up version of https://bitcointalk.org/index.php?topic=267618.msg15398605#msg15398605

The address generation goes by 100 keys/second, but to check that address against the UTxO goes even slower, 15 keys/second.
There were 750 million addresses to check...

It's very slow, but I'm still happy with the result.
Although the reward was only 1 mBTC, I learned a lot during the process.  :)


Now let's see if I can optimize this a few factors...


Title: Re: Retrieve lost satoshis
Post by: tspacepilot on July 05, 2016, 02:38:23 AM
I got my satoshis back!  ;D
I just picked up this thing again and see where I could end up with noob python skills and new insights.
Estimated time to check all possibilities was 270 days but I found my key already within 20 hours!
I thought to let it run, and resume the script after optimizations to speed things up. I was not going to wait 270 days, but I was (extremely) lucky I guess.
My script does approximately 15 keys/second (4 instances, 60 k/s total) and checks against a cleaned up version of https://bitcointalk.org/index.php?topic=267618.msg15398605#msg15398605

The address generation goes by 100 keys/second, but to check that address against the UTxO goes even slower, 15 keys/second.
There were 750 million addresses to check...

It's very slow, but I'm still happy with the result.
Although the reward was only 1 mBTC, I learned a lot during the process.  :)


Now let's see if I can optimize this a few factors...


Lol, only 20 hours of cpu time but nearly 9 months of forum time.  Anyway, congratulations!  I also learned a lot by playing around with key and address generation.  Cheers!


Title: Re: Retrieve lost satoshis
Post by: findftp on July 05, 2016, 09:38:53 PM
I got my satoshis back!  ;D
I just picked up this thing again and see where I could end up with noob python skills and new insights.
Estimated time to check all possibilities was 270 days but I found my key already within 20 hours!
I thought to let it run, and resume the script after optimizations to speed things up. I was not going to wait 270 days, but I was (extremely) lucky I guess.
My script does approximately 15 keys/second (4 instances, 60 k/s total) and checks against a cleaned up version of https://bitcointalk.org/index.php?topic=267618.msg15398605#msg15398605

The address generation goes by 100 keys/second, but to check that address against the UTxO goes even slower, 15 keys/second.
There were 750 million addresses to check...

It's very slow, but I'm still happy with the result.
Although the reward was only 1 mBTC, I learned a lot during the process.  :)


Now let's see if I can optimize this a few factors...


Lol, only 20 hours of cpu time but nearly 9 months of forum time.  Anyway, congratulations!  I also learned a lot by playing around with key and address generation.  Cheers!

Meanwhile, I optimized my script (with this (https://github.com/weex/addrgen)) so it does about 8000 keys/second  :) Still not extreme but now it only takes two days to scan all possibilities.
I also found another address with balance which had a similar pattern that I used! About 7mBTC ;D
But, the UTxO set I used is from may this year. When I checked the address at current block height I saw that the coins were already spent. The coins were on that address for few weeks.

Anyway, it's a nice way to learn python and memory management! ;)