I'd like to ask a hypothetical question (situation didn't really happen) to better my understanding of addresses and wallet encryption.

I had about $40 in a Bitcoin Core un-encrypted (backed up) wallet.

My hard drive crashed and the computer repair guy was able to recover 100% of the documents that I needed that were saved since my last backup.  

When picking up the computer, the repair guy commented, "I noticed you had a Bitcoin wallet so I recovered that for you too".

I restored my entire hard drive image from a week-old backup to a new hard drive including wallet.dat (no activity during the week on wallet.dat).

If I now generate a new receiving address from the wallet and receive funds on that address, would the private key for that new address (and therefore funds) be available to an opportunistic computer repair guy if he were to dump all addresses and private keys from the un-encrypted wallet.dat? i.e., if he generated all addresses in the pool in the un-encrypted wallet.dat, would my new receiving address and private key eventually show up for him?

After all of this, I encrypted my wallet.dat.  Does encrypting change replace all addresses from the un-encrypted wallet with new addresses?

Does keypoolrefill replace all old addresses with new addresses in the pool or only just replenish back to the maximum? If replaces, then that one command would insure that I'd get an address that's not in the old wallet.dat.

Do I have to create a new wallet, encrypt it and use a receiving address from the new wallet to receive future funds (and transfer the $40 to an address in the new wallet)? This is a simple thing to do, I just want to understand what addresses are in a wallet using this hypothetical situation.

Thanks.

@DaveF


I'm not surprised that you didn't find anything.   It took a bit of looking for me to find something.

I used the following and made the same changes to my NOMP that were made in the 3 commits mentioned in the first post.

https://github.com/zone117x/node-stratum-pool/pull/115

(Line numbers from these commits didn't match my line numbers so I had to search for the code in my modules.)

For my NOMP, I'm using modules from 2 NOMP forks (separately both were buggy) and a bit of coding of my own.  Although the topic in the link indicates that zone117x has SegWit, I wasn't comfortable swapping my NOMP for theirs so I made the changes to mine.

Every story needs a happy ending and although I didn’t find a Mainnet block (yet), it appears that I have the mechanisms to do so.  This is the "last chapter" but I still have a concern and a question below.

As I said in a previous post, if you make a change to mining or pool software, USE TESTNET!!!

Background:  The NOMP that I had been using for years wasn’t SegWit compliant so I couldn’t use it for Bitcoin.  I therefore needed and found a SegWit commit to a different NOMP fork, and added the code to my NOMP.  This was the solution that I chose, I could have started using different pool software or connect to a public pool.

For reference:

After adding the SegWit code, I installed a BitPay wallet to get a Testnet address.  The address began with an "m".  This address is used for the miner to connect to my NOMP pool.

My Testnet Bitcoin Core wallet - bitcoin-cli getnewaddress - gave me an address beginning with a "2" by default.  This address was used by the pool to communicate with the Bitcoin network.

When I found 6 Testnet blocks, I got errors in the PaymentProcessing module seconds after each find.  It turns out, that in checking the Bitcoin Testnet Block Explorer, the address associated with my found blocks began with an "m" but didn't belong to my wallet.  I have no idea where this address came from. How this address changed from a "2" to an "m" is my concern. It would be nice to return the faucet coins that I borrowed for testing, but I can't get access to this "m" account.  Obviously, I don't want anything of the sort to happen on Mainnet.

Next, using bitcoin-cli getnewaddress "" "legacy", I got an address beginning with an "m", confirmed that it belonged to my wallet, found 2 more blocks, and after 100 confirmations, funds were in my BitPay wallet.  So everything seems to work with like legacy addresses.

So on Mainnet, I'll use all addresses (ASIC miners and pool) starting with a "1".

I may try again with two "2" addresses on Testnet to see if that works.

Q: What would be the advantage of using "2" addresses on Testnet rather than "m" (and then all "3" addresses on Mainnet rather than "1")?

No, NOMP has no wallet.

NOMP just confirms that the address supplied belongs to the Bitcoin Core daemon.

Yet this "m" address appeared in Testnet corresponding to each of my found blocks, rather than the "2" address actually supplied by the daemon and confirmed by the pool.

The errors that I experienced


are probably related to the address not matching, so the issue in my mind is only, where did the "m" address come from?

After 100 confirmations, payment may be sent, but I wonder to where and I wonder if the same will happen on Mainnet.

@Carlton Banks
So it looks like you're right.  The address that found the blocks is not recognized by my Bitcoin Core wallet as its own.  So given that my daemon wallet created a "2" address and the pool confirmed that the "2" was from the daemon, where did this "m" address come from? I'm glad this is fake Bitcoin.

This is an interesting experience.  Hopefully someone else will find these posts and learn as I have.

I now need some help determining why Testnet is not accepting my blocks.

Brief background: I use NOMP locally to solo mine and have done so successfully for dozens of coins.  I like and have grown accustom to NOMP.  When I started the first post just 5 days ago, my NOMP wasn't SegWit compliant so I couldn't use it for Bitcoin.  I've since found a commit for a NOMP fork making it SegWit compliant and I added the code to my NOMP, tested it on Testnet and I'm having problems.

The biggest lesson learned: if you make changes to Bitcoin mining or pool software, USE TESTNET!!!!  Thank you to all who encouraged me to do so: @Carlton Banks, @jackg and @gmaxwell.  

Thanks to @DaveF for giving me some more stable alternatives to NOMP which I may still end up using, and to all others who shared their knowledge and experience with me.

These are 6 blocks that I found on Testnet.

https://tbtc.bitaps.com/mtAeMSyVpsdX3BrVdNMaKwA1G8G4GMWHpF

What I've noticed:

(1) I don't think this is anything but bitcoin-cli getnewaddress gave me an address that I used as the pools address (it began with a "2" and testnet=1 was set) but my blocks were found by the above address beginning with an "m".  I assume they equate to each other.

(2) I notice looking at my Raw Transactions that my version number is a 1 whereas all other transactions had a 2 but I didn't think transaction numbers were still being used.  If this is the issue, easily fixed and tested again.

(3) After the first rejection I started the daemon with ./bitcoind -debug=rpc -debug=http but after finding the next block, nothing additional of value was in the debug log

(4) The pool reported the issue to me in multiple lines when payment processing ran (example of block 1609642):
I don't think the block was actually orphaned because all 6 found blocks reported the same way

(5) Looking at the blocks in the link above, I notice that the blocks are getting confirmations.  I'm not sure how many are required before (fake) payment but I doubt that payment is coming.

(6) This really may be minor but when looking at the Verbose listing, I see of the 2 outputs, that my OP_RETURN is listed first whereas in all other blocks that are not mine, the OP_RETURN is 2nd.  I think there's a reason but not sure what it is.

Any help appreciated on any of the above.

That is a true statement.  That was my (hopefully incorrect) assumption.



I was hoping that this was the case but I couldn't find any examples on the use of "proposal".   I couldn't even get "proposal" to get accepted syntactically.  I figured that getblocktemplate wouldn't go through the pool software but what I think I'm understanding you to say is that I can take the output of the pool software and feed it back through getblocktemplate.

If that is correct, conceptually how do I get my block out of the pool software?  I guess I could write something that would send the output to a file but do you have an easier way in mind?

Then, and again these are assumptions, how does that file become input to getblocktemplate (or I guess if "proposal" takes a string as input, I'd cut and paste the gigantic block, but again I can't find anything beyond the documentation for "proposal", no examples (and even the examples at the bottom of the documentation (link below) for bitcoin-cli seem syntactically incorrect)):

https://bitcoincore.org/en/doc/0.18.0/rpc/mining/getblocktemplate/

So I guess the biggest help would be if you could help with the conceptual and point me toward uses of "proposal" e.g., examples.

Thanks.

I ran


(after some tries, this was the only syntax that worked)

A bunch of transactions showed but I noticed that there was no coinbase transaction (there were fields at the end of the output that may be used to create the coinbase, but no formatted coinbase transaction)

I searched and found this

https://stackoverflow.com/questions/42334832/has-the-bitcoin-getblocktemplate-response-json-changed-where-is-the-coinbasetxn

where it said


So given that the "proposal" option doesn't interact with the pool software and that I want to test everything including the pool software's ability to properly create a  codebase transaction, the proposal option didn't appear to do what I wanted.

I'm working toward setting up Testnet.

@DaveF  
Thanks.



@gmaxwell
If I'm reading the the "proposal" option to getblocktemplate correctly, it appears that I won't be testing the coinbase transaction.  Is that correct?

Thanks.  I found a NOMP fork that had implemented SegWit and added the code to my local NOMP version.  I'll run on Testnet to see how it works.


This sounds interesting.  Can you tell me more about this option?

@DaveF

I've used it for dozens of coins over the past years.
Became very comfortable with it and its logic.
It was reliable.
Was able to make changes when required.

But I'm open to other suggestions.  Do you have any?

Thank you @jackg and @gmaxwell.

@jackg it was reading the 21 comments below the initial post that suggested to me that my block would be invalid

@gmaxwell


I don't see your comments in my original thread but I see them in the above post.  Thanks again.

I don't think I provided all relevant information in my previous post with a similar title.

The conclusion in that previous topic was that I can keep doing what I'm doing and don't have to fear losing the reward if I find a block.

I want to provide more information and see if that's indeed the case, since elsewhere (e.g., link below) it is suggested that I will lose the reward if I find the block given my setup.

https://bitcoin.stackexchange.com/questions/86208/are-there-still-miners-or-mining-pool-which-refuse-to-implement-segwit

I think the assumption in my previous post was that I was using a non-SegWit compliant full node e.g., before  Bitcoin Core 0.13.  I am not.

I'm mining using a SegWit-compatible full node, Bitcoin Core 0.18 and wallet, and I'm using non-SegWit compliant pool software, NOMP.

Mining with this setup, will my block be considered invalid and I lose my reward?

@jackg

I just added an additional fact above (an edit) that I should have mentioned before and want to know if it makes a difference...


Thanks

In the last line of my original post (above) I assumed that if my non-SegWit compliant NOMP pool software found and submitted a non-compliant block, that the block would be determined to be invalid by SegWit-compliant nodes, thereby wasting all of my mining efforts.

Is that true?

I then assumed that I could either only include non-SegWit compliant transactions in the block or, since I wasn't too sure how many of those still existed, I would make Empty Blocks (blocks with only the coinbase transaction) since, as I read, the coinbase transaction doesn't use SegWit since there are no inputs...

...but if as Carlton (and I assume you) say, it doesn't matter (e.g., what I asked above is not true), then I can just keep doing what I'm doing.

Comment?

This is a true statement:


but my understanding is that SegWit compliance doesn't apply/relate to the coinbase transaction because there are no inputs:

from a bitcoin.stackexchange.com post:


so given this, I was hoping that an Empty Block (block with only the coinbase transaction) would NOT be considered invalid to a SegWit compliant node.

Is this true?  Comments?

Eight months ago or so, I wanted to solo mine some "lesser known" coins and the only way to do so was to use my own pool.

I found NOMP, which mostly didn't work and had long since been abandoned. Piecing together modules from 2 different NOMP forks, along with some coding, I was able to get it to work and successfully ASIC mine dozens of Scrypt and sha256 coins. (Every single one has lost significant value but that's another discussion.)

I now want to use my NOMP pool to solo mine Bitcoin.  (I know that I have a better chance of throwing a stone and hitting the moon than finding a block, but that's also another discussion.)

SegWit doesn't appear to be implemented in NOMP.

I read that SegWit doesn't apply the the coinbase transaction so:

QUESTION: If I change my NOMP code to create Empty Blocks (with only the coinbase transaction) will I get around the problem of other (SegWit-compliant) nodes considering my block as invalid?

To be clear, I'm using Bitcoin Core 0.18 and non-SegWit compliant pool software.

(And when I read that my block will be considered invalid, I assume that means that if I happen solve a block, that all mining effort will have been wasted, no exception.)

This is a fascinating aspect of BTC that I had no idea existed.  Thanks.

Everything posted has been interesting; indeed learning is generally interesting, but enough philosophy...

Both alani123 and nc50lc mentioned the mempool - a clue.

At the advise of some random post found by searching mempool.dat, I went to my full node wallet (which continues to just eat up disk space but it's what I started with so...) and typed:


and before me appeared data, which led to a better understanding and more questions:

The file was 9MB in size and dated 2 days ago so I wasn't sure if that meant that I didn't have the last 2 days of transactions or, as I've seen before, the OS doesn't change the date even through some file has been updated

9MB of transactions that weren't picked up?  Seems like a lot. (in JSON format, each transaction took 767 bytes so = approx 11,700 transactions)

I then noted 2 fields in particular:


If this is a transaction that hasn't seen a block yet, why does it have a height.

I didn't know how to translate the date field - help appreciated.

So the questions are:

Does the mempool only contain "unselected" transactions?

If so, why does it have a block height?

What does this date field represent?  I'm sure there's an easy way to translate it e.g., some online tool - if anyone knows of one, please let me know.

If this does contain transactions already in the blockchain, which field indicates that it was never selected (or never made its way to the blockchain)?

Thanks for the explanation thus far.

Maybe if I understood the mechanism that allows transactions to be available for selection, this would make more sense.

The idea that transaction #3 (or whatever) under certain circumstances could take hours to get to the blockchain (or never get there if hanging out to dry for 2 weeks) seems "inefficient".

Could someone explain where these transactions are before they get into a block, and how a pool gains access to them to select (or not).

Thanks.