how do miner picks transactions for candidate block

[mimosizz]

i'm a bit new on the ground. trying to explore how does it work in theory?
when miner selects transactions for candidate block, they usually pick transactions with higher fee. can they pick any valid transactions with significantly low fee? if they can, how all these things work? is it manually adding a list of transactions? or automatically? i guess if it's an automated process, miner can't pick low fee transaction at all but i read that miners can include any transactions they want for the candidate block.

[DannyHamilton]

. . . when miner selects transactions for candidate block, they usually pick transactions with higher fee. . .

The transactions are typically chosen by the software that the mining pool operates.  Individual participants in the mining pool generally don't get to choose the transactions. Since the mining pool gets to use those fees for profit and/or to pay the participants, and since there is a limit to the number of bytes of data that can fit into a block, choosing the transactions with the highest feee per used space is the most profitable thing to do.

can they pick any valid transactions with significantly low fee?

Sure. But they will earn less bitcoin if they do. Since miners need to pay for the mining equipment, internet connectivity, electricity and possibly employees that they use, and mining pools need to pay for their servers, network connectivity, electricity, and mining participants, they need to earn as much bitcoin as they can. If they fail to earn enough bitcoin to pay for all the costs, then they will run out of money and have to stop.

if they can, how all these things work?

The mining pool would just modify the computer software that they use so that it uses some other method of choosing transactions.

is it manually adding a list of transactions?

That would be possible, but it would be slow and risk errors.  Far more likely, they would just modify the software to implement whatever criteria they wished to use.

or automatically?

It depends on why they are doing it, but since mining continues 24 hours a day, every day, continuously, it would generally make more sense to have software that handles most of the process for them.

i guess if it's an automated process, miner can't pick low fee transaction at all but i read that miners can include any transactions they want for the candidate block.

If you're talking about a miner that is mining for a mining pool, then they would need to rely on the pool to handle it.  For a solo miner, they would need to modify their own software.

[mikeywith]

can they pick any valid transactions with significantly low fee?

They sure can, most people think no pool does that, but that's not the case, some pools do, a great example is ViaBTC pool which offers transaction accelerator services, they would include transactions that pay very low fees in return for a good payment directly to their wallet outside of the pool accounting.

Example:

The current fee is 1$, someone (usually by mistake ) pays 1 cent and their transaction is now stuck, they need it confirmed ASAP, they would pay $10 directly to the pool, and the pool will drop a $1 transaction and include the 1 cent transaction instead, the pool loses 4% of 0.9$ (the fees they collect from the miners), but they keep $10 to themselves, the miners, on the other hand, lose 96% of 0.9$.

i guess if it's an automated process, miner can't pick low fee transaction at all but i read that miners can include any transactions they want for the candidate block.

It is automated, the software does that, you can modify the code however you like, you can have your friend's transactions included regardless of fees, you can simply choose not to include any transaction in the block, and you can even include invalid transactions and lose a whole block, you can do just about anything you want since the code is under your control as a pool operator, but just because you "can" doesn't mean you would, it's all about profit.

[Husires]

can they pick any valid transactions with significantly low fee? if they can, how all these things work? is it manually adding a list of transactions? or automatically?

If the transaction is valid, they can do it manually, even if it does not follow the standard, but as mentioned above, the fees are the ones that govern and can be paid privately away from the transaction fees.

I remember a similar case: ~1 BTC Bounty

Poolin.com has mined the transaction: https://explorer.poolin.com/tx/fd02fd81790a05ea2ab07bbcf5cdcb2cfd21f3f5676a5c0e8f7f5ca2ac75e694

if anyone else has any issues with a transaction please reach out to me for help.

For those who want details.

that is really messed up! every single documentation that i have ever seen has always said "it must not be compressed or it will not be mined". now i went back and checked the out, they are all are ambiguous about it!
take BIP143 for example:
https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki#restrictions-on-public-key-type
Quote
As a default policy, only compressed public keys are accepted in P2WPKH and P2WSH. Each public key passed to a sigop inside version 0 witness program must be a compressed key: the first byte MUST be either 0x02 or 0x03, and the size MUST be 33 bytes. Transactions that break this rule will not be relayed or mined by default.