Bitcoin Forum
December 04, 2024, 02:59:43 PM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 [13] 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 »
  Print  
Author Topic: Pywallet 2.2: manage your wallet [Update required]  (Read 208245 times)
BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 21, 2013, 11:09:06 PM
 #241

hi man, tks for the update, it will for sure speed things a little, but here's a problem:

When making a request to delete a transaction, all the parameters are being passed through GET Method, and the GET method has some limits regarding the length of the URL (http://classicasp.aspfaq.com/forms/what-is-the-limit-on-querystring/get/url-parameters.html). This length depend on the server and also the client, for example, until and including Internet Explorer 8 the maximum URL length is 2.083 characters. So, if I have about 100000 transactions to delete, and lets suppose that I will be able to delete 500 transactions per request, taking the average of 11 seconds to complete each request in my wallet of 242MB, it will take about 36 minutes to complete the deletion. This is a great advance instead of waiting 12 days to delete the transactions one by one, but as I said before, using the "all" keyword it took less than 20 seconds. I still think that reading a file with the keys to delete will still be the best way.

Another problem, When I tried to merge two wallets, in one of then the password contained the character #, and I received an error. when I checked the URL being called, I saw that it stripped all the remaining characters including the character #, so I'm assuming that the problem is that the URL isn't being encoded.


One more question:
- After deleting all the transactions from the wallet, it dropped the file size by a few KB. is it normal? why the size didn't drop more?


And once again, tks for your great tool.






jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 22, 2013, 12:00:22 PM
 #242

Yeah I forgot while doing it, I'll make this soon.

I never tried with such a passphrase. Try with replacing # with %23.
I'll look at that when I'm back home. I think it would just require to put an encodeURIComponent.

As for the small size drop, I don't know, it depends how many transactions your wallet contained. Compare the dumps before and after deletion to be sure there's no transaction left.

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 22, 2013, 10:05:24 PM
 #243

Just created the dump with my wallet with "no transactions" (actually it now has 1 transaction as I moved back my coins), and I confirmed that it has only one transaction.

Copied all the text of the JSON dump and saved it to a text file and the text file size is 32MB (noting that I have about 70k addresses generated).

And using the wallet in the bitcoin-qt it is showing only one transaction too.

before deleting the transactions it contained about 230000 transactions (didn't made the dump with the wallet file with all transactions as I think it will take too much time as it took about 11 minutes with the wallet without the transactions).

Regards.

BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 22, 2013, 10:08:46 PM
 #244

by the way, I received the following error when making the dump (but it completed successfully):

2013-05-22 18:36:52-0300 [HTTPChannel,2,127.0.0.1] Wallet data not recognized: {'__type__': 'orderposnext', '__value__': '\xc1\x8b\x03\x00\x00\x00\x00\x00', '__key__': '\x0corderposnext'}

jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 22, 2013, 10:36:00 PM
 #245

I just pushed the new version: no more '#' bugs in passphrases and you can use file for deletions

Format: 1st line=type ('tx' or 'key'), then 1line=1data to delete
Code: pywallet.py --multidelete file.txt [--wallet wallet.dat] [--datadir /bitcoin/directory]
Code: (file.txt)
tx
b0b0865e723101b5a720a79d805efd544c59fa2ceebffc724a12cd5fe683057f
07941322e6dd5b15b3bde0739648d49526e19ea78b8f4085e943fca473cbfdbf
etc...


by the way, I received the following error when making the dump (but it completed successfully):

2013-05-22 18:36:52-0300 [HTTPChannel,2,127.0.0.1] Wallet data not recognized: {'__type__': 'orderposnext', '__value__': '\xc1\x8b\x03\x00\x00\x00\x00\x00', '__key__': '\x0corderposnext'}
That just means that the 'orderposnext' entry in your wallet isn't recognized. That is because I personally don't know what it is and I doubt it's useful.
Pywallet will keep it inside your wallet when you import/delete/dump/whatever though, it just doesn't understand it

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 23, 2013, 04:07:11 AM
 #246

Hi,

I downloaded your new pywallet, modified my program to generate a file with all my transactions ids and started pywallet with the multidelete option.

After 15 minutes running I decided to interrupt the execution of the pywallet and start investigating your python code to try to determine where the problem is (I never worked with python, but as a programmer I haven't problem undertanding it.) and I think I found it.

The problem is that my file has about 200k transactions and my wallet has at least 200k+70k(addresses) keys. Well, if for each transaction in the file, it will interact trough all the keys of the wallet (I didnt' see anywhere in the code where you break the iteratioon of the second "for" when finding the key), so the pywallet will need to do a total of 54 billions of iterations.

So I was thinking a bit and I have a question and a suggestion.

Question: in the line of code "db.delete(key)", if the "key" don't exist, will it raise an error or it will pass without problem?

My suggestion: when deleting transactions, instead of applying the modifications on the "key" to compare to the "keydel", why not just transforming the "keydel" to the "key" format and deleting it without iterating through all the wallet keys?

Regards.

jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 23, 2013, 01:10:23 PM
 #247

Question: in the line of code "db.delete(key)", if the "key" don't exist, will it raise an error or it will pass without problem?
It looks like that it passes without problem. Actually the delete function searches in all the DB to find all the keys to delete, so my method is pretty dumb...
I made it at the beginning of my fork, which corresponds to the beginning of my learning of Python/serious programming. The more I look at the code I made, the more I find it ugly

My suggestion: when deleting transactions, instead of applying the modifications on the "key" to compare to the "keydel", why not just transforming the "keydel" to the "key" format and deleting it without iterating through all the wallet keys?
Yeah that's better that what's done currently, but as I wrote the delete function will take care of that even better

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 26, 2013, 04:31:45 PM
 #248

I completely forgot to work on it, I'll do this in the following hours

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 26, 2013, 07:01:41 PM
 #249

I made a small improvement
I'm not really sure it would change a lot though. I don't think it might make it faster than now, because of the way Bitcoin stores the txs

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 27, 2013, 01:29:44 AM
 #250

hi, I looked your new code and, as you said, maybe it won't change the performance to much.

I was thinking in something like this:

at line 1779
Quote
   for i in range(len(kd)):
      keydel=kd[ i ]
      if typedel == "tx" and keydel <different> "all":
         <apply modifications to keydel to transform it to the key format>
         db.delete(key)
         deleted_items+=1
      else:

         for (key, value) in db.items():
            .
            .
            .

Do you think it is possible?

Regards

cheesylard
Member
**
Offline Offline

Activity: 104
Merit: 10



View Profile WWW
May 27, 2013, 02:14:23 AM
 #251

Can this work on jailbroken iOS?
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 27, 2013, 06:38:04 AM
 #252

That's not possible, and that's what I meant by "the way Bitcoin stores txs": the key is "tx"+hash(tax). So you can apply what you proposed only if you know the hash.

Hmm, wait, I just get it, yes we have the hashes because you use the file to store them... I don't know why I thought you were using 'all'
There's some hope for speed improvement then!

Can this work on jailbroken iOS?
Yes absolutely. I made it work on iOS 5.
Look if there's a Python port for your iOS. If so that should work.

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 28, 2013, 05:44:52 PM
 #253

It's done, please try

To make it faster I don't catch exceptions so if you provide at least one transaction that isn't in your wallet it will crash without telling you which one
Maybe you prefer catching exceptions and avoiding all the deletions to be canceled just because one failed, just tell me what is better

I added the exit code 1 in case of failure




To everybody
Do you think pywallet would be more practical as a Qt app?
As I said earlier I'm wondering about recoding everything. It's far from sure but if I decide to I'd like to know how to make it more user-friendly.

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
K1773R
Legendary
*
Offline Offline

Activity: 1792
Merit: 1008


/dev/null


View Profile
May 28, 2013, 06:37:34 PM
 #254

It's done, please try

To make it faster I don't catch exceptions so if you provide at least one transaction that isn't in your wallet it will crash without telling you which one
Maybe you prefer catching exceptions and avoiding all the deletions to be canceled just because one failed, just tell me what is better

I added the exit code 1 in case of failure




To everybody
Do you think pywallet would be more practical as a Qt app?
As I said earlier I'm wondering about recoding everything. It's far from sure but if I decide to I'd like to know how to make it more user-friendly.

full CLI (ie the same capabilites like the WUI) and a WUI should be doing fine.

[GPG Public Key]
BTC/DVC/TRC/FRC: 1K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM AK1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: NK1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: LKi773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: EK1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: bK1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
BRules
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


View Profile
May 29, 2013, 12:51:43 AM
 #255

just tried and it tooked 5.5 minutes to delete all the 232386 transactions that was in my text file.

Great job, and thank you.

but after deleting it's showing the following:

"232386 elements deleted
Error: do not try to delete a non-existing transaction."

looks like it's raising an error on the command "exit(0)"


a GUI interface will be cool, as far I start earning some money with my project I will support this development.

regards


jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 29, 2013, 08:08:17 AM
 #256

I'm glad that worked and that is quick
I didn't even think that exit would raise an exception, it's ok now


You both confirm what I was afraid of, that that'd nice to have CLI+GUI+WUI
I think I won't use the twisted package anymore to deal with the WUI though, I'll manage sockets myself
I'm fed up with dependencies

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
dserrano5
Legendary
*
Offline Offline

Activity: 1974
Merit: 1030



View Profile
May 29, 2013, 09:55:54 AM
 #257

To everybody
Do you think pywallet would be more practical as a Qt app?

Just as K1773R, I'd like to have a full working command-line interface, and am comfortable enough with a web browser for point n click usage.
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 29, 2013, 10:18:17 AM
 #258

Yeah, the CLI is the priority anyway

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
phelix
Legendary
*
Offline Offline

Activity: 1708
Merit: 1020



View Profile
May 29, 2013, 10:41:44 AM
 #259

Yeah, the CLI is the priority anyway
+1 to command line

It would be nice if the project was separated in three files to be able to look over it more easily:
- core functions
- command line interface
- gui/web interface
jackjack (OP)
Legendary
*
Offline Offline

Activity: 1176
Merit: 1280


May Bitcoin be touched by his Noodly Appendage


View Profile
May 29, 2013, 02:53:24 PM
 #260

I like putting everything into a single file because I find it easier to track and easier for newbies
I can split it in three though and then push the four files to github

Own address: 19QkqAza7BHFTuoz9N8UQkryP4E9jHo4N3 - Pywallet support: 1AQDfx22pKGgXnUZFL1e4UKos3QqvRzNh5 - Bitcointalk++ script support: 1Pxeccscj1ygseTdSV1qUqQCanp2B2NMM2
Pywallet: instructions. Encrypted wallet support, export/import keys/addresses, backup wallets, export/import CSV data from/into wallet, merge wallets, delete/import addresses and transactions, recover altcoins sent to bitcoin addresses, sign/verify messages and files with Bitcoin addresses, recover deleted wallets, etc.
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 [13] 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!