BIP38 python problem

[n0nce]

I guess the real problem and let me get on my soapbox here is that Bip38 wasn't taken very seriously and thus they don't even have a standard reference implementation for current versions of python lol.

I would like to inform you that a BIP is a bitcoin improvement proposal, so more like a protocol specification, and does not need a reference implementation in any language, especially not in Python, since Bitcoin Core is mostly written in C and C++.
Like, do you expect for every BIP that is 'taken seriously' to have a reference implementation in all programming languages?

[pooya87]

I would like to inform you that a BIP is a bitcoin improvement proposal, so more like a protocol specification, and does not need a reference implementation in any language, especially not in Python, since Bitcoin Core is mostly written in C and C++.
Like, do you expect for every BIP that is 'taken seriously' to have a reference implementation in all programming languages?

You are confusing the "reference implementation of Bitcoin" with "reference implementation of BIPs" they are not the same, ergo the BIPs reference implementation doesn't have to be in C++. In fact it should be in a langauge that the author of the BIP is most familiar with so that they can write the best readable code possible without any bugs.

The other implementations in other languages are usually done by other developers doing it as volunteers to let readers of the BIP choose whatever language they like to understand the algorithm better. If a BIP doesn't have other implementations it just means there weren't any volunteers yet and you can act on it if you want to contribute.

[NotATether]

What the quote means is that you use AES with the following settings:
key size = 256 (bit) (the derived key from scrypt)
mode = ECB (each block is encrypted individually)
IV = new byte[16] (empty initialization vector)
padding = none

Yes, those settings are correct.

The AES256Encrypt function described in BIP38 doesn't chain AES blocks together, and the IV is zeroed out before using it (it's not randomized).

Since this function transforms 16 bytes of input into 16 bytes of output, no padding is needed.

[larry_vw_1955]

I would like to inform you that a BIP is a bitcoin improvement proposal, so more like a protocol specification, and does not need a reference implementation in any language, especially not in Python, since Bitcoin Core is mostly written in C and C++.
Like, do you expect for every BIP that is 'taken seriously' to have a reference implementation in all programming languages?

Well here's the thing. BIP38 is going away. Eventually there won't be any wallets or any software that supports it. It never really caught on. It's kind of a shame because I do think it's a cool piece of technology but unfortunately, we've lost the knowledge on how to implement it since all we have is a generic "protocol specification". The fact that you can't even find a python 3.7 implementation of it speaks volumes. No one cares about bip38. And no one will in the future.

Lucky for me though I was able to implement it in python 3.7 so I can use bip38 as long as I want to unless python decides to bullshit its users again like they did with the 2 to 3 upgrade.

[larry_vw_1955]

I worked on the EC non-multiply mode. I never messed with EC multiply mode but I actually think the EC Multiply mode is the far more interesting of the two. The BIP itself is genius. Maybe a bit overly complicate for the non-multiply mode in the sense that there's probably far more simpler ways to encrypt a bitcoin private key such as just using winrar but hey...

[pooya87]

BIP38 is going away.

No, it is not.

Eventually there won't be any wallets or any software that supports it. It never really caught on.

The thing is that wallets are always HD so there is little use for a single key encryption technique for them. However the method itself is still useful and will remain useful for paper wallet creation.

~ since all we have is a generic "protocol specification".

Which is more than enough for a developer to implement it.

The fact that you can't even find a python 3.7 implementation of it speaks volumes.

You are just not searching well enough:
A python bitcoin library:
https://github.com/1200wd/bitcoinlib/blob/c52cdfc9fe7feaedcf4dc3e43ceca4071865281b/bitcoinlib/encoding.py#L772-L845

A shitcoin wallet's library:
https://github.com/joshmcintyre/blockchainmower/blob/7457c361b5b2792e7cd3c4882713d6b71efddcda/src/electron_cash_lib/bitcoin.py#L1023

From a paper wallet builder:
https://github.com/ryanralph/DIY-Piper/blob/0df360f61707739dd7ba5b7758f79b7c8e136867/bitcoin/bip38.py#L17

A set of utilities from @darosior:
https://github.com/darosior/bitcoineasy/blob/master/bitcoineasy/bip38.py

Another paper wallet builder:
https://github.com/zobin8/PyperWallet/blob/master/pyperlib/cryptor.py

P.S. Note that I'm not endorsing any of the above links, just pointing out examples of python (mostly v3) implementation of BIP38. Aside from that there are lots of python 2 implementations that could simply be converted to python 3.

[n0nce]

Lucky for me though I was able to implement it in python 3.7 so I can use bip38 as long as I want to unless python decides to bullshit its users again like they did with the 2 to 3 upgrade.

Off-topic but... Python didn't bullshit its users. There was a long transition period where both were supported, Python 3 brings many needed features to the table like typing and is generally the better programming language.

[ABCbits]

Well here's the thing. BIP38 is going away. Eventually there won't be any wallets or any software that supports it.

It's literally available on many wallet, including hardware and multi-cryptocurrency wallet. Here's a hint, if they ask you to backup 12/24 words and mention it can be recovered on most wallet, most likely it use BIP 38. There are few competitor such as Electrum Seed Version System, but still can't beat BIP 38 even though it has versioning system and better security.

Lucky for me though I was able to implement it in python 3.7 so I can use bip38 as long as I want to unless python decides to bullshit its users again like they did with the 2 to 3 upgrade.

Off-topic but... Python didn't bullshit its users. There was a long transition period where both were supported, Python 3 brings many needed features to the table like typing and is generally the better programming language.

Maybe he's complaining about most linux distro had to drop lots of software which still depend on Python 2 or few software got discontinued since the author have no intention to convert it to Python 3.

[pooya87]

Here's a hint, if they ask you to backup 12/24 words and mention it can be recovered on most wallet, most likely it use BIP 38.

I think you may be confusing BIP-38 (Passphrase-protected private key) with BIP-39 (Mnemonic code for generating deterministic keys). The former is supported by only a handful of wallets and tools while the later is supported by almost everything. OP was talking about the former.

[larry_vw_1955]

Thanks for the correction, there are so many BIPs that i sometimes forget the correct number. Even so, BIP 38 is still most common option to encrypt private key, even bitaddress support it. I'm also not aware of other encryption standard solely for cryptocurrency private key.

Well I mean yeah, bitaddress does have a certain level of support for bip38 but in my limited testing, I found it failing for a certain test vector. But that's ok, the code that I have works for that test vector just fine. So I guess I'll stick to that.

[larry_vw_1955]

The thing is that wallets are always HD so there is little use for a single key encryption technique for them. However the method itself is still useful and will remain useful for paper wallet creation.

bip38 could be extended to allow for hd wallets i am sure. why they dont' do that is hard to understand. how long wold it take them to come up with that maybe 2 or 3 days tops, with lunch breaks?

[n0nce]

The thing is that wallets are always HD so there is little use for a single key encryption technique for them. However the method itself is still useful and will remain useful for paper wallet creation.

bip38 could be extended to allow for hd wallets i am sure. why they dont' do that is hard to understand. how long wold it take them to come up with that maybe 2 or 3 days tops, with lunch breaks?

The point about hardware wallets is that they don't just have 1 private key. They generate a seed phrase from which an infinite amount of private keys can be derived. So BIP38 would have to be changed to not encrypt and encode a private key, but instead to encrypt and encode a seed phrase instead.

At this point the scope changes completely and will require a new BIP in my opinion.

[larry_vw_1955]

So BIP38 would have to be changed to not encrypt and encode a private key, but instead to encrypt and encode a seed phrase instead.

At this point the scope changes completely and will require a new BIP in my opinion.

yeah that may be true but it's still a doable thing. Not sure why it hasn't been done. other than just no one caring. one thing they wuld need to make sure though is to support electrum   now that i think about it though. there's no "address" that corresponds to an hd wallet so instead of the address what would you print out? the first 10 addresses?

[HCP]

Why would you need to waste time and effort creating a method to encrypt a seed when you already have plenty of perfectly adequate (and proven) encryption systems for encrypting "plain text"?

As for supporting Electrum... Electrum already has perfectly functional wallet file encryption. ¯\_(ツ)_/¯

I'm not sure what it is that you're attempting to achieve here?

[larry_vw_1955]

Why would you need to waste time and effort creating a method to encrypt a seed when you already have plenty of perfectly adequate (and proven) encryption systems for encrypting "plain text"?

You know that's a really silly question. Let me rephrase it slightly and maybe you can help answer it:

Why would you need to waste time and effort creating a method to encrypt a bitcoin private key when you already have plenty of perfectly adequate (and proven) encryption systems for encrypting "plain text"?

[pooya87]

Why would you need to waste time and effort creating a method to encrypt a bitcoin private key when you already have plenty of perfectly adequate (and proven) encryption systems for encrypting "plain text"?

A bitcoin private key is not plain text, it is technically a number and can easily be treated as a stream of bytes with fixed length (32 bytes). A mnemonic on the other hand is technically plain text even though it can be converted to a stream of bytes but it will have variable size.

With that said I'm all for a "standard" way of encrypting a mnemonic, I've already posted about what I think such a method could look like a couple of times on this forum.

[larry_vw_1955]

A bitcoin private key is not plain text, it is technically a number and can easily be treated as a stream of bytes with fixed length (32 bytes). A mnemonic on the other hand is technically plain text even though it can be converted to a stream of bytes but it will have variable size.

any number can be encoded as a hexadecimal string. which you can think of as a text string composed of the characters 0-9,A-F. there is a 1-1 relationship between the hexadecimal text string and bitcoin private key that it represents. Or just think of 256-bit text string consisting of 0s and 1s. That's a string too. or it can be thought of as such without causing any confusion or problem.

With that said I'm all for a "standard" way of encrypting a mnemonic, I've already posted about what I think such a method could look like a couple of times on this forum.

Very good! Maybe I can locate them somehow. But you post so often that it might be kind of hard to dig them all up but it would be worth it I'm sure.

[pooya87]

any number can be encoded as a hexadecimal string. which you can think of as a text string composed of the characters 0-9,A-F. there is a 1-1 relationship between the hexadecimal text string and bitcoin private key that it represents. Or just think of 256-bit text string consisting of 0s and 1s. That's a string too. or it can be thought of as such without causing any confusion or problem.

That's true but when you enter a private key WIF (the string) it still has to be converted to bits and treated as an integer but when you enter a mnemonic (the string) is treated as a string and even decoded using UTF8 when deriving BIP32 seed from it.

Very good! Maybe I can locate them somehow. But you post so often that it might be kind of hard to dig them all up but it would be worth it I'm sure.

It's usually in topics talking about mnemonic or when someone tries creating their own encryption algorithm. Here is an example from last month:

You have a mnemonic that is n-bits (eg. 128-bits for 12 words) take those bits and encrypt them with AES256 with a key derived from a strong passphrase and a salt derived from the address (like BIP38).
Now you get 128-bits of encrypted data (encryption without IV) which you can encode to 12 words using the same BIP39 scheme. All you have to do is write down these words as if they were your mnemonic.

To import to a wallet you just decrypt these words and convert to an un-encrypted BIP39. That is decode 12 words to get the byte[], derive the AES key from the passphrase and address like above, decrypt using AES256. Now you have un-encrypted 128-bit entropy. Encode this using BIP39 scheme and you have the original words.

[larry_vw_1955]

You have a mnemonic that is n-bits (eg. 128-bits for 12 words) take those bits and encrypt them with AES256 with a key derived from a strong passphrase and a salt derived from the address (like BIP38).
Now you get 128-bits of encrypted data (encryption without IV) which you can encode to 12 words using the same BIP39 scheme. All you have to do is write down these words as if they were your mnemonic.

To import to a wallet you just decrypt these words and convert to an un-encrypted BIP39. That is decode 12 words to get the byte[], derive the AES key from the passphrase and address like above, decrypt using AES256. Now you have un-encrypted 128-bit entropy. Encode this using BIP39 scheme and you have the original words.

that's a pretty neat idea the only potential problem I see with it is, it's not very "portable". It requires a fixed wordlist. what if there is no wordlist at all? or what if the wordlist is different than the bip39 one. that means we have to store a complete copy of whatever wordlist it is we want to use and somehow the software has to be smart enough to know what wordlist we want it to use?

i'd much prefer a solution that
1)didn't require a fixed wordlist
2)allowed for variable length mnemonics
3) but still operated by changing words in to other words
4) encodes the derivation path being used so that someone can find their money
5) encodes the particular cryptocurrency that the derivation path above refers to
6) possibly allows for encoding multiple 4/5 pairs of data so that different coins can be referenced

[pooya87]

It requires a fixed wordlist.

Any encoding method requires a fixed (usually hard-coded) set of "digits" and if you don't know those you won't be able to decode it. For example if you don't know base16 uses 0-9, a-f you con't decode it. For base2048 you need the digits which in case of a BIP39 mnemonic or what I explained above is the word list that would be required to decode both the mnemonic and the encrypted mnemonic alike.

i'd much prefer a solution that
1)didn't require a fixed wordlist
2)allowed for variable length mnemonics
3) but still operated by changing words in to other words
4) encodes the derivation path being used so that someone can find their money
5) encodes the particular cryptocurrency that the derivation path above refers to
6) possibly allows for encoding multiple 4/5 pairs of data so that different coins can be referenced

1-3. You still need to know the word list but it could be changed in a way that doesn't depend on a fixed length word list like the algorithm Electrum uses
4-6. This can also be implemented but it would increase the number of words and the result will no longer resemble a normal BIP39 mnemonic.

My idea was an attempt to end up with a result that doesn't look any different from regular seed words.