Bitcoin Forum
April 16, 2021, 02:58:37 PM *
News: Latest Bitcoin Core release: 0.21.0 [Torrent]
  Home Help Search Login Register More  
  Show Posts
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 »
1  Bitcoin / Development & Technical Discussion / [Experimental-2] Better mnemonic on: April 13, 2021, 06:56:38 AM
We're trying to address some of the problems and shortcoming with the existing mnemonic algorithms namely BIP-39 and Electrum. The idea is to add the information that the wallet needs to derive child keys to final string that the user writes down and more importantly we see mnemonic as a simple encoding and avoid modifying the entropy when deriving the BIP-32 seed (see #4).

The C# implementation can be found here:
Like any experimental class under Experimental namespace this is a rough idea and the code is untested.

4 bit version | 4 bit depth | 32 bit index * depth | CompactInt entropy length | entropy | checksum (pad)

1. Scalability
To make the algorithm scalable a small 4 bit version is added to the beginning of the encoding. This way we can have different definitions using the same scheme.

2. Derivation path/script type
In order to let the wallet know which BIP-32 derivation path to use, that path has to be encoded into the mnemonic. This path could also act as the script (address) type of the child keys. For instance m/84'/0'/0'/0 is for P2WPKH addresses.
The downside is increasing the length of the final mnemonic. This could be mitigated by introducing version 2 where instead of encoding the entire path a 4 bit integer is used to indicate the predefined paths. 16 hard-coded paths can be defined this way and for anything new next version could be used.
version 1: 4-bit version | 4 bit derivation path depth | each index as 32-bit unsigned integers
version 2: 4-bit version | 4 bit hard-coded path

BIP-39 doesn't have this feature.
Electrum has a partial solution by first increasing the entropy size from 128 to 132 bits then brute forcing a version that indicates the derivation paths and script types

3. Creation time
This can be useful for the full nodes to avoid re-scanning the entire blockchain when the user imports their mnemonic. The creation time (as a LockTime object) can quickly tell the client from when in the history to begin the scan. For example a mnemonic created today doesn't have to scan the 300 GB of existing history.
Since this is LockTime as used in transactions (instead of simple DateTime) it could be used by both online and offline clients generating the mnemonic as the online synced client knows the block height and can use that and the offline (cold storage) can use the DateTime (values as Unix timestamp and above 500,000,000).

The rest is implementation detail, for example the wallet UI can show the LockTime (block height or DateTime) and still give the user the option to do a full rescan.

4. No modification of the initial entropy
This is the main and biggest difference. Both BIP-39[2] and Electrum[3] are modifying the mnemonic:
A. Modifying by normalization
BIP-39 uses a Unicode normalization with full compatibility decomposition (form KD) but Electrum takes a step further and modifies the input mnemonic more by removing accents, changing Chinese words, removing spaces for word lists other than English, etc. Then the byte array representation of the string is used to derive the keys.
This can create incompatibility between different implementations and lead to losses specially for any word list other than English.

If the words are only used to look up index of each word in the word list to get the entropy out and nothing else, this problem is eliminated.
In other words if user enters a word like "aliener" instead of "aliéner" (from French word list) it still can be searched inside the word list and if the search fails (due to bad normalization) the import process fails right away instead of going ahead. While normalizing this word can give different values:
0x616c6965cc816e6572  No normalization
0x616c6965cc816e6572  NFKD
0x616c69656e6572      Electrum
0x616c69656e6572      No accent
The problem is more palpable when using Japanese word list for example (the start of the difference is marked by ^, also note the lengths):
そつう れきだい ほんやく わかす りくつ ばいか ろせん やちん そつう れきだい ほんやく わかめ

B. Use the UTF-8 decoded bytes of the mnemonic instead of the entropy
When creating a mnemonic a random entropy with a good distribution of bits is chosen. For example in a 12 word mnemonic the entropy is an octet string with length of 16 (ie. 128 bits). Each octet can have a value between 0 and 255 inclusive which is 256 different values.
When this is converted to 12 words using English word list then UT8 decoded to be used to derive BIP-32 seed, it is a longer octet string but each octet can only have a value between 97 and 122 inclusive which is only 26 different values.
In other words a big bias is introduced in the input of PBKDF2 while the initial entropy had no bias at all (assuming it was chosen using a strong RNG).

By using the entropy itself we can eliminate this bias and also avoid the normalization bugs that could occur in different implementations while greatly simplifying the implementation.

The only disadvantage of this proposal is the longer mnemonic. For example a 12-word BIP-39/Electrum mnemonic would turn into a 26-word version 1 BetterMnemonic using BIP-84 derivation path (without locktime) or 14-word version 2+ (without locktime).

[1] Experimental-1:
2  Bitcoin / Project Development / Re: [C#][Github] SharpPusher, broadcast BTC, BCC(BCH) transactions to the network on: April 11, 2021, 09:09:30 AM
Does it really make your project runnable on every operating system?
Yes. Unlike .net framework the .net is no longer windows specific and can be used for any operating system. The UI is also designed that way to render everything based on the platform that is used.

Does it require .NET 5 or it has it already installed? I've noticed an annoying bug on .NET 5's installation. For example, I had to install .NET 5 on another machine (because it required it) and once I did, it showed me the same message again ("You need to install .NET 5 to run this"). This sours some people and they'll give up the software before they even try it.
There are two ways we can publish a .net application framework-dependent and self-contained, the former requires installing .net and the later doesn't but it is slightly bigger.
I have been releasing my other project (FinderOuter) as a self-contained executable for 64-bit Linux OS. It doesn't need any additional download or install, you just get the binaries and run them. The screenshot is from it running on Ubuntu 20.04.
3  Bitcoin / Project Development / Re: [C#][Github] SharpPusher, broadcast BTC, BCC(BCH) transactions to the network on: April 10, 2021, 12:43:36 PM
I haven't ever done this. Shouldn't this button be enabled?
You are comparing master with master on the same repository, so there is nothing to create a PR with. You have to change the branch. There is a button on top saying "compare across forks".
An easier way is to just fork the project and push changes to your own repository then inside that repository you will see a PR button that you can use.
4  Bitcoin / Project Development / Re: [C#][Github] SharpPusher, broadcast BTC, BCC(BCH) transactions to the network on: April 10, 2021, 10:09:06 AM
So it probably works. These are the lines that I changed from MainWindowViewModel.cs:
Feel free to open a new pull request so I can check and merge the changes.

What type of windows application is SharpPusher? It is definitely not WinForms. It seems an old project and since it's not built in .NET 5, I doubt that it uses the AvaloniaUI you tend to build your C# apps.
It is Windows Presentation Framework (or WPF) and yes it is a very old project that only runs on Windows.
Due to lack of interest in the project I never spent more time on it but I could migrate it to .net 5 and use AvaloniaUI (so that it would run on any OS) if there is any interest.
5  Bitcoin / Project Development / Re: [C#][Github] SharpPusher, broadcast BTC, BCC(BCH) transactions to the network on: April 06, 2021, 09:11:54 AM
I'd like to help this project by adding more options and different cryptocurrencies like litecoin, dogecoin but other networks too such as the testnet, which doesn't exist at the moment if I'm not mistaken.
You can do all using blockchair and the new API class I just added. Add their names to the enum and set their names correctly based on the blockchair API docs.

As you've written in the "To Do List", it'd be much cooler to write/choose a node's IP address instead of using a third party API. Combined with your, it could work, right?
Yes and no.
The problem is that I still haven't had the chance to write a MinimalClient so you'll have to write a lot of code to make it work using the existing Node class.
Also this project is rather ancient so it is using .net framework 4.5 which is not compatible with .net standard 2.1 you'll also have to do some updates and move SharpPusher to .net 5.


I imported your project on my Visual Studio 2019, but I don't see the user interface. I can build it and execute it successfully, but I'm not able to see it in the VS. Will I have to install any plug-ins/libraries? I'm just doing this for exercise, but it'll also be fun. (and helpful)
Double click the MainWindow.xaml file in your solution explorer, that's the UI.
The default startup project should also be SharpPusher project which means if you press f5 you should build and run and see the UI.
6  Bitcoin / Project Development / Re: The FinderOuter, a bitcoin recovery tool (v0.8.0 2021-03-20) on: April 06, 2021, 08:48:39 AM
Version 0.9.0 is released.
See changelog for details.
  • New recovery option: find encoding of an arbitrary input string

  • Add a new help view that shows up at startup and suggests which recovery option to choose

  • Add knowledge Base window that contains explanation of different parts
There are small question marks on different UI objects that can be clicked to move to the respective KB

The KB will contain some general information about what the input is, how it is used, etc.

  • Some small improvements in address validation and error message

A good news
I've made good progress on solving issue #9 and next release will hopefully solve it for good as we will replace BigInteger entirely with a UInt256 struct that doesn't have the same issues as BigInteger and also it adds a huge speed gain.
7  Bitcoin / Project Development / Re: [C#][Github] SharpPusher, broadcast BTC, BCC(BCH) transactions to the network on: April 06, 2021, 08:09:09 AM
Hello again, I'm just going through my lists on the [Guide] Broadcast Your RAW Transaction (Push TX) BTC & Alts coins thread and was just checking if your program is still active? 

The list above has a couple of broken links to site that are no longer available.
Thanks for the report.
Removed the broken APIs and added a new one (Blockchair) to be used for both bitcoin and BCH.
Here is version 0.11.0
8  Bitcoin / Project Development / Re: The FinderOuter, a bitcoin recovery tool (v0.8.0 2021-03-20) on: April 05, 2021, 05:51:11 PM
When searching for Base58 wif code, can you bring up a method of adding several addresses instead of one destination address?
It could be added but I don't see the point in it and here is why, the checksum in a WIF is big enough to have very little collisions when going through permutations. Most of the times with small number of missing characters (that would take reasonable time) only 1 valid key exists and as the number of missing characters grow the number of valid keys that are returned remain too small. For example with 7 missing chars at random positions worst case scenario is that you get 2 or 3 keys. It is easy for user to check all of them. If the number of missing chars is too big then recovery is most probably not possible, accepting more addresses won't change that. In any case the program is already printing any valid key it finds as it goes.

BTW the Missing base58 option is not using the extra input field (ie. address/pubkey) for WIF recovery. It is just used for a special case where the missing chars are all located at the end.
9  Bitcoin / Project Development / Re: The FinderOuter, a bitcoin recovery tool (v0.8.0 2021-03-20) on: April 05, 2021, 02:02:53 AM
This fraudster is selling your software :
Thanks for informing me. The corresponding google account is banned now.
10  Bitcoin / Development & Technical Discussion / Re: I need help understanding secp256k1_scalar_check_overflow (from secp256k1 lib.) on: April 04, 2021, 04:53:45 PM
Note that the last 4 words of the group order (SECP256K1 constant) are equal to 0xFFFFFFFF.
Foolishly set yes to 0, again!
Actually only the last 3 are max, N4 is 0xFFFFFFFE.
11  Bitcoin / Development & Technical Discussion / I need help understanding secp256k1_scalar_check_overflow (from secp256k1 lib.) on: April 04, 2021, 12:48:11 PM
I understand what it does and why, I just don't get how it does it.
SECP256K1_INLINE static int secp256k1_scalar_check_overflow(const secp256k1_scalar *a) {
    int yes = 0;
    int no = 0;
    no |= (a->d[7] < SECP256K1_N_7); /* No need for a > check. */
    no |= (a->d[6] < SECP256K1_N_6); /* No need for a > check. */
    no |= (a->d[5] < SECP256K1_N_5); /* No need for a > check. */
    no |= (a->d[4] < SECP256K1_N_4);
    yes |= (a->d[4] > SECP256K1_N_4) & ~no;
    no |= (a->d[3] < SECP256K1_N_3) & ~yes;
    yes |= (a->d[3] > SECP256K1_N_3) & ~no;
    no |= (a->d[2] < SECP256K1_N_2) & ~yes;
    yes |= (a->d[2] > SECP256K1_N_2) & ~no;
    no |= (a->d[1] < SECP256K1_N_1) & ~yes;
    yes |= (a->d[1] > SECP256K1_N_1) & ~no;
    yes |= (a->d[0] >= SECP256K1_N_0) & ~no;
    return yes;
12  Bitcoin / Development & Technical Discussion / Re: Hello! My name is Mark Ingle.... on: April 04, 2021, 02:47:17 AM
I am not that familiar with hashing but I think I am able figure things out with all of the information available.  I think this will an interesting rabbit hole!
Maybe this visualization can help understand what's happening in the hashing process:
Just know that headers are 80 bytes and SHA256 breaks data into 64-byte blocks when compressing them.
13  Bitcoin / Project Development / Re: Denovo (v 0.1.0) and Bitcoin.Net (v 0.10.0) on: April 03, 2021, 05:32:52 AM
Version 0.8.0 released.
  • Improvments mainly in ReplyManager and Blockchain for handling communication and header verification
  • Target stuct is improved to handle edge cases in compliance with consensus rules
  • IConsensus has a couple of new properties and methods
  • Multiple new Constants are added
  • NodeStatus properties are pure properties and new method is used to signal disconnect instead
  • Block headers are now processed directly through IBlockchain instead of IClientSettings
  • Client can now store and report its own IP address after receiving Version messages
  • Client is now capable of downloading, verifying and storing the entire block headers
  • Client's communication is based on INode's protocol version
  • Added a new word list: Portuguese (affects BIP-39, ElectrumMnemonic but not defined for BIP85)
  • Various bug fixes, tests and some improvements
Version 0.9.0 released.
  • Major changes in P2PNetwork namespace involving initial connection
  • Introduce ClientTime to get/set client time using other peers
  • Block headers now store their hash locally with an option to recalculate hash
  • Fix some issues in Node class when it got disconnected
  • Fix some issues in NodePool class with locks
  • Introduce BlockchainState and respective events to be raised when it changes
  • Peers are selected based on BlockchainState and their service flags, all handled by ClientSettings
  • Introduce a new timer for each peer to disconnect them when they are not responding to requests (this is important when syncing)
  • Improve the initial handshake to send "settings" messages based on protocol version of the peer
  • Add some new constants
  • Various optimization, tests and small improvements
(IStorage will be completely removed in next release)
Version 0.10.0 released
Implemented initial block synchronization code in Blockchain and respective classes
  • IUtxoDatabase and IFileManager have new methods
  • Add a new UTXO class
  • Mandatory ClientSettings properties are read only and can only be set in its constructor, the rest can be set using the respective property setter
  • To reduce memory usage some properties are placed in ClientSettings and are accessed from all threads (by different node instances)
  • RandomNonceGenerator is thread safe now
  • (I)Storage is entirely removed
  • Hash and HMAC functions are all sealed
  • IHashFunction and its IsDouble property are obsolete and will be removed in next release.
    Use the new ComputeHashTwice method instead1
  • Various bug fixes, tests and improvements
1 The interface and the idea for the IsDouble property is a leftover from before the library became Bitcoin.Net and was supposed to be Cryptocurrency.Net instead.
New Denovo feature
Encrypt and decrypt messages using Elliptic Curve Integrated Encryption Scheme (ECIES).
Encoding of the input or the output can be anything from the list of options that are available:

Version 0.11.0 released.
  • New BIP: Bech32m format for v1+ witness addresses (BIP-350)
  • All encoders are static now and have a TryDecode method
  • Validity check by encoders is 2 methods now: IsValid (checks characters) and IsValidWithChecksum (checks both) unless the encoding doesn't have encode without checksum like Bech32
  • IHashFunction is removed
  • IsDouble is removed
  • Almost every class in Cryptography namespace is sealed now
  • Various improvements, additional tests, bug fixes and small XML doc fixes
14  Bitcoin / Project Development / Re: [C#][Github] BitcoinTransactionTool. Make, Edit Tx; Make QR, CoinControl on: March 26, 2021, 05:09:52 AM
Only now i've found this tool, and in one hand it could be pretty cool.
But in another hand this is not suitable for Linux users.

I couldn't compile it from source code. It doesn't have configure file and also don't have (so i can't use automake for purpose).
Only for Windows users  Angry as i see
It also is incomplete and has some bugs.
When I started this project 4 years ago there wasn't any decent multi-platform UI and .net core has just been released (I didn't know about it anyways) and I didn't like Mono at all. Not to mention I was brand new and had very little understanding of both Bitcoin and C#.

I have moved on to a much better code and started implementation from scratch (instead of in the middle like this project). I've already finished the back-end and have been meaning to migrate this project over there but I keep postponing it because I have more pressing matters to attend to.
You can follow the migration here:
Suffice it to say that Denovo is using AvaloniaUI that can run on any OS and the back-end is Bitcoin.Net which is a full implementation of Bitcoin protocol so it won't have any missing parts or bugs like this project.
15  Bitcoin / Project Development / Re: The FinderOuter, a bitcoin recovery tool (v0.8.0 2021-03-20) on: March 20, 2021, 06:20:06 AM
Version 0.8.0 is released.
See changelog for details.
  • Some user interface improvements
  • New recovery option: find BIP-32 derivation path
    Note: this is a simple option with only a handful of standard paths, I'll try finding more paths used by different wallets and optimize it but for now it is very basic.
  • New recovery option: recover Armory backup phrases missing some characters
    Note: this option's speed is wildly variable because it depends a lot on the existence of the checksum. It can be from dozens of keys per second to hundred billion key per second. For example if the recovery phrase has its checksum and is missing 10 characters you can check 1 trillion variations and recover it in 1 second.
  • Main window size has a limit so it can no longer be shrinked to nearly nothing

A small RoadMap
  • For next release I'd like to work more on the user experience by adding a Help window where user can interact with at the start of the application and can figure out which option to use.
  • I would also like to add some more "tips" to the UI explaining different terminology. For example explaining what WIF, Base58, derivation path, ... are.
  • Let me know if there is any interest in changing UI theme like having light or dark theme instead of the default light one or a colored one.
Version 0.9.0 will probably be released with these. But I also have 2 more options in mind:
  • BIP-39 passphrase (the extension words) recovery (hopefully for 0.10.0)
  • BIP-38 password (for encryption) recovery

I've also started working on optimizing ECC to hopefully solve issue #9 but it's still very challenging.
16  Bitcoin / Development & Technical Discussion / Re: Verifying an encrypted message? on: March 19, 2021, 06:01:19 PM
Since ECIES uses a symmetric encryption algorithm (AES)
How can it use a symmetric encryption algorithm? The message can only be decrypted by x where gx is the public key, but by no other keys. Only by the receiver's. On symmetric encryption (AES) both private keys can decrypt it. You, also, had included EciesTests.cs on the Asymmetric/KeyPairs directory back when you were implementing ECIES on Autarkysoft.Bitcoin.
ECIES is basically combining 2 things: symmetric encryption and asymmetric key exchange.

In order to encrypt/decrypt a message using a symmetric algorithm like AES you need a key (like a password) this means when A wants to send an encrypted message to B it has to also send the "password" alongside it. Obviously that is hard and unsafe so this is where asymmetric cryptography part (Elliptic Curve) comes in. A has to use a key (password) that B can also reproduce on its own but nobody else can.

In this process we have 3 public keys:
1. Ephemeral public key chosen by A and is published with the encrypted message, lets call the key pair (de,Qe) where d is private key and Q the public key.
2. B's public key (dB,QB) and it is not published but may be known.
3. Result of (deQB) which is the encryption key and can be equally computed using (dBQe) which is obviously not published but can be calculated by both A and B but nobody else.

You see none of these have anything to do with the message which is why they can't be computed using the message or encrypted message.
And as far as the EC part goes even if you had both public keys that A and B have (#1 and #2) you can never compute the third public key (#3 ie the encryption key).
17  Bitcoin / Development & Technical Discussion / Re: Verifying an encrypted message? on: March 18, 2021, 01:29:23 PM
Is it possible to get the public key by only having the encrypted message?
Since ECIES uses a symmetric encryption algorithm (AES) I don't think there is any way to derive the user's public key. The result contains the encrypted message from AES and the public key of the ephemeral key used to derive the encryption key. Neither could be used to derive the user's pubkey.
18  Other / Beginners & Help / Re: [Study Purposes] How can we build our own ecosystem? on: March 14, 2021, 08:25:13 AM
I would like to start to develop (while I study) my own cryptocurrency, [...] I forked ethminer [...]
Forking another project and modifying it is not going to help you learn much and if you continue down this path the end result will be the copy of that project inheriting all its issues too and you won't have a good understanding of why things are the way they are.
For example you need to be able to answer why Keccak was used and why is PoW designed this way with uncles,etc. and what are the disadvantages of this approach?

Can you guys help me telling me the right way to start in this crypto world? What I have to study, examples of codes, wikis, new projects that I can keep an eye, etc.
If you want to learn and not just create a copy-coin then my suggestion is that you should start from Bitcoin and you don't need to look at the source code that much. You have to study different parts and understand why every decision was made the way it is.
I say Bitcoin because it has the best documentation and you can find the most amount of content on the internet without needing to read the code. Other altcoins usually have nothing apart from their source code.

For example I would split the whole topic into different categories like this:
1. Cryptography
This includes asymmetric cryptography (ECC in bitcoin) and hash algorithms.
The questions you need to be answer are: What cryptography functions are used in bitcoin, where and why. For example double SHA256 is used in PoW because it is a strong hash algorithm and has a good digest size (not too small to make it weak anytime soon and not too big to waste space).
What alternatives could you be using (eg. could you use Blake?).

2. Blockchain
This includes Proof of Work, structures used for blocks, transactions and finally scripts.
You have to be able to answer questions like why do blocks or transactions look the way they do, what are the issues and benefits of each.
What are scripts used for and why are they designed this way, what is a better way of doing it and what is worse?

3. P2P communication protocol
This includes the way the entire network is shaped and how each client communicates with others and how it contributes to decentralization.

After you had a good understanding of the above, you can start looking at what others have done and see what advantages and disadvantages their approach had.

Sources you could use are:
Bitcoin wiki:
Bitcoin developer's reference:
Bitcoin SE:
Mastering Bitcoin book:

Source code is only useful when you want to figure out the details not to learn:
19  Bitcoin / Armory / Re: Is there any Armory backup phrase test vectors? on: March 04, 2021, 04:18:08 PM
Thanks, I have seen that tool and used it for some initial testing but I was hoping for the reference implementation for a more reliable source. Besides the brainwalletx code is very poorly written, it doesn't even validate checksums.
20  Bitcoin / Development & Technical Discussion / Re: About GetMyExternalIP on v0.1 on: March 04, 2021, 12:13:20 PM
IRC was used initially for them to find peers, AFAIK.
Not to find peers but to find "your own" information and advertise "yourself" to others.

Your computer does not know its own IP address, the outside words does. So you have to ask someone else what your IP is. Early versions used this service and I believe some others added later to fetch your own IP address which would have been used in an addr message to advertise your own node if you wanted to listen for incoming connections.

Obviously the above method is relying on a centralized service so these days we rely on other nodes in a very decentralized way to tell us what our IP is. Each time you connect to another node it sends you back a version message that contains your IP which you can use in your future version and addr messages
Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 »
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!