I just found my bug, and constructed a new Tx with actually-unspent outputs. I'm ready to plug it into your webpage, but it looks like you took it down  . Let me know when it's back up and I'll toss you a couple BTC. |
|
|
|
|
I don't need to derail your thread with my own development activities, so we could PM about this. In a nutshell, I'm doing Tx construction on the command line, and I usually end up with potential Tx's I want to broadcast, depending whether my SelectCoins did what I wanted. If so, then I copy the serialized-tx hex into the tool for broadcasting (right now, Nibor's webpage), otherwise I ignore it, tweak my code and try again. Copying raw binary from ASCII displays always made me uncomfortable... but maybe it works...
(I see that you had direct output from another process in mind for piping into your mktx-send operations, but I didn't)
|
|
|
|
Thanks for thinking of me, genjix  How do I put the serialized transaction into the stdin? raw binary characters? Would it be: echo "<binary junk>" | mktx send It's a tad easier if I can pass around hex instead. On the other hand, I will probably just pick apart your code, adapt to my client, and integrate it into my codebase Until then, I'll probably use it for experimenting until I get to that point. Looks really good, thanks for releasing this! |
|
|
|
|
Ack! This is my problem. Somehow my code picked a TxOut that had already been spent...? I guess I've got my own debugging to do... Will try again tonight.
|
|
|
|
I finally got my code together to select coins, construct the tx, and then sign with private key. I created a tx for 1.09 BTC to your address, and submitted it to your webpage. However, there was no explicit statement that it succeeded, only a restatement of the tx in a somewhat human-readable form. I didn't see it in either of the next two blocks, so I question whether it actually made it to the blockchain. Here's the full, serialized tx that I submitted, in case you want to check on it yourself: 0100000002f658dbc28e703d86ee17c9a2d3b167a8508b082fa0745f55be5144a4369873aa010000008c493046022100d4b006a55d91d3b62a040089febc12bc4ec88e570ed34ccc3286938e622e31090221007351f84554ab1c1fe4b091ed1905e261660a9863f02c0f9036aae195e3ff6b400141048d103d81ac9691cf13f3fc94e44968ef67b27f58b27372c13108552d24a6ee04785838f34624b294afee83749b64478bb8480c20b242c376e77eea2b3dc48b4bffffffff26f5e5ab244ce8d56e47b282ee2f6cba2c511a538386c86acd72279e9e557290010000008c493046022100ae301914b8070cab6584b292976ca06a05f4dd7959b14112e4ce25533d315d64022100a419f8e6f66adae096f188b69774524d78e75b2b22a06e052e0848cfbf3c122a0141048d103d81ac9691cf13f3fc94e44968ef67b27f58b27372c13108552d24a6ee04785838f34624b294afee83749b64478bb8480c20b242c376e77eea2b3dc48b4bffffffff0240357f06000000001976a9141b00a2f6899335366f04b277e19d777559c35bc888ac804a5d05000000001976a9140e0aec36fe2545fb31a41164fb6954adcd96b34288ac00000000 |
|
|
|
|
Thanks nibor! Gonna be sending you a donation soon. You will receive it if your webpage works! Btw, it's really an aesthetic suggestion, but I think the input box should be multi-line. I want to be able to see the full tx I copied into the page.
On a philosophical note, I want to reiterate the possibility of this kind of tx-forwarder enabling a simpler method of anonymity in the BTC network. Right now, Tor is the best solution for the ultra-paranoid, but it's complicated/inconvenient for the user. The developer could try to integrate Tor protocols in the client, but that's quite complicated too.
On the other hand, if we got a bunch of folks to voluntarily host tx-forwarders like nibor's, with SSL, then we've got a much easier solution for both developers and users. The client software could come with a list of tx-forwarders, which only needs to be partially complete, because it can update from any of the forwarders. When the client wants to send a tx, it makes an SSL connection to one of the forwarders at random, and uploads the tx data over the encrypted connection. That forwarder will then randomly either broadcast, or forward to another forwarder, which will do the same. Eventually, the tx will be emitted from a completely different IP address than the originator. This would be fairly straightforward to implement in client software by linking in the OpenSSL library.
|
|
|
|
Is there still one of those broadcast transaction sites, so I can check if everything's working? Can't seem to find it anymore…
It turns out the original site you are thinking of was taken offline, but Nibor is working on one based on this thread. You could PM him to find out how he's doing on it. Admittedly, such a site is useful occasionally for folks, but has remarkably low-demand on a regular basis, which is why the original site was taken offline. |
|
|
|
Take a look at about every single transaction on blockexplorer. Most of them have fees.
Well, default behavior of the Satoshi client is to include a fee, unless the user explicitly selects otherwise -- which is why I think most of them have fees. If the default fee is too low, they are notified that their tx may require a higher fee, and are given an option to include a fee more appropriate than the default. I think most users just accept the fees because they are so small. And to avoid the tx-purgatory, we have the sequence numbers.
If I understand sequence numbers correctly, you would need a non-in-the-past locktime, and replacement would have to be enabled. And in order to leverage it, you'd have to send all your transactions as not-final. For a "standard" tx that is expected to be final from the moment it hits the network, I don't think sequence numbers do anything to alleviate the issue. Again, all I'm trying to do is clarify that my interpretation of the client code is correct so I can offer fail-proof recommendations for when to include fees and how much. I don't want to end with pissed off users that their 100 BTC tx has been sitting at 0 confirmations for 2 days, and it turns out it was because they combined 50+ inputs to create the tx with no fee, and the network relayed it but no miner will include it. |
|
|
|
Actually, that transaction was 0.01, which is the minimum output size not considered to be "dust." You've have to send 0.00999999 or less to trigger the fee logic... When Bitcoin sends a transaction, it makes sure the fee is greater than or equal to GetMinFee() with blocksize=1 and fForRelay=false.
It's probably pretty rare for a transaction to get stuck, since many miners rarely require fees. I never send a fee for my personal transactions.
I recognize that what happens in practice is a lot different than what is in the Satoshi code. It does seem there will always be miners that will accept no-fee transactions, but there's also a history of txs getting stuck, too, so it's not impossible (unless some core logic changed, somewhere). I'm developing a client for other people to use -- if there's a chance of this happening, I want to give the users every opportunity to avoid it. Ideally, I implement the minimal logic that avoids tx-purgatory 100.0% of the time, and allow the "advanced" users to turn it off if they're willing to take the risk. |
|
|
|
Sending/relaying a transaction never takes blocksize into account.
The latest version won't let you send transactions with incorrect fees. It can happen with old/modified/alternate versions that don't know the current fee rules, though. Transactions with too-low fees will be stuck at 0 confirmations for a long time.
Do you know the exact conditions under which that happens? Is it just those transactions will never satisfy AllowFree() and don't include a fee? Or , a fee that is greater than MIN_RELAY but less than MIN_TX? Does "relaying" a tx mean that it is added to the node's memory pool? At the moment, I am relying on those three conditions to declare that a tx is fine to send without a fee. Otherwise, I multiply MIN_TX_FEE by the number of kB (rounded down) to determine the appropriate fee. Is this a safe strategy to use with a network full of 0.4.0 and 0.5.0 nodes? |
|
|
|
I have read a lot of posts about Tx fees, and I've been doing quite a lot of C++ diving into the Satoshi client to try to figure them out. Here's what I have concluded, and I am hoping someone can verify/correct me. - (1) I see lots of calculations concerning current blocksize. If I ignore those calculations and simply focus on the tx-based fees, then even if my fee is lower than what would be computed with the blocksize info, the tx will still be relayed and put into other clients' memory pools, for inclusion in the next block that doesn't have such high volume. In other words, if I don't mind waiting a block or two for inclusion, it's perfectly okay to ignore the blocksize components of that calculation.
- (2) MIN_RELAY_TX_FEE and MIN_TX_FEE are each set to 10,000 and 50,000 respectively (0.0001 and 0.0005 BTC). I assume that MIN_TX_FEE is for miners to decide whether to include in a block, and MIN_RELAY_TX_FEE is for regular nodes/miners to decide whether to forward the block to its peers. Is this a correct interpretation? If so, then what happens to a Tx that doesn't meet the MIN_RELAY_TX_FEE when it is broadcast during a block with high tx-volume? It seems that a tx that would otherwise be free on the next block, might have MIN_RELAY_TX_FEE enforced due to large blocksize and not forwarded to peers...?
- (3) There are three conditions for a tx to qualify to be put into the free space of one the next blocks. Those three conditions are:
-- Sum-of-priority-of-inputs > COIN*144/250 (i.e. > 1 BTC, one day old)
-- All outputs are >= 0.01 BTC
-- Tx size is smaller than 4kB (or 3500 to be safe)
If my tx satisfies all three of those conditions and I don't mind waiting a block or two, it's perfectly safe to send the Tx without a fee. - (4) Along the lines of (3), as long as my Tx satisfies those three conditions above, it doesn't actually matter whether it is 250 bytes, or 3000 bytes, it will still get into the free space of the next block if its priority is high enough. On the other hand, if it doesn't qualify as a free tx, then I should pay MIN_TX_FEE or MIN_RELAY_FEE for each kB that transaction is.
- (5) Consider a 5kB tx with one output of 0.0001. If I try send this tx with 0 tx fee, it will just be DOA at the first peer, even my own client won't add it to its own memory pool. Then, it will be as if you never even tried to send a tx in the first place. How does the client deal with this? (or does the client even let you send such a tx?).
- (6) Has the tx-purgatory problem been fixed? Is it possible to still send a tx that gets stuck in network indefinitely?
|
|
|
|
So, after spending a couple days battling my own SelectCoins algorithm, I came up with a decent idea for how to do this without thinking too hard, but still maintain the ability to expand it, later. The magic is in defining the following method: uint32_t ScoreSelectCoinsOutput(vector<unspentTxOut> selectedCoins, uint64_t targetVal, uint64_t minFee) The input to this function is a coin-selection, a target output value and a fee. By writing a method that assigns a score to a selection, you you can then randomly sort your input list 100 times, pick the top X inputs that satisfy your target+fee, and then keep the one with the best score. In my code, I am slowly adding smarter solutions and phasing out the randomly selected ones, but if you pick a high enough sample size, you're bound to get at least one good solution in pot even with nothing but random selections in the search: vector<unspentTxOut> finalSelection;
uint32_t bestScore = UINT32_MAX;
for(int i=0; i<100; i++)
{
random_shuffle(unspentTxOutList.begin(), unspentTxOutList.end())
uint64_t sum = 0;
uint32_t index=0
vector<unspentTxOut> selection;
while(sum < target+fee)
{
selection.push_back(selection[index]);
sum += selection[index].getValue();
index++
}
uint32_t score = ScoreSelectCoinsOutput(selection, target, fee);
if( score < bestScore)
{
finalSelection = selection;
bestScore = score;
}
}
// Now you have a pretty good selection!
Then, the entire SelectCoins algorithm is based on how you "score" a selectCoins solution. In my code, I evaluate a bunch of different "selection factors", such as how many input addresses it combines, if it includes zero-confirm txs, how many bytes, how obvious it is which output is change/recipient, etc. I make sure that each of these factors spits out a number between 0 and 1 (with 1 being favorable), and then multiply each score by a weight: WEIGHT_NOZEROCONF = 1000
WEIGHT_ALLOWFREE = 100
WEIGHT_NUMADDR = 100
WEIGHT_TXSIZE = 50
WEIGHT_PRIORITY = 60
WEIGHT_OUTANONYM = 30 The weights are configurable (and completely arbitrary--only significant relative to each other), and gives you the option to modify them for your use-case: you can make "WEIGHT_NUMADDR=10000" and "WEIGHT_OUTANONYM=10000" to try to max-out anonymity at any expense. |
|
|
|
+1 To this. I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed. How much work would that be--it sounds like tests/net is like 90% of the way there. I think you want this: https://gitorious.org/libbitcoin/subvertx/blobs/master/src/mktx.cppHowever it doesn't do node discovery yet, so you have to know who you're connecting to beforehand. No problem for me. For now, I plan to have the satoshi client open anyway, so I can just connect to localhost and avoid peer discovery entirely. In the future I might work on something more independent, but for now this is perfect! If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution? Of course. It's a library. Don't worry about attribution. I'm not too familiar with the various licenses and terminologies around them. It looks like your code is also AGPL v3, so there shouldn't be any compatibility problems. But I would like to do things right, and apply attribution regardless if I really need to -- I like to give credit where it's deserved. It sounds like it won't be difficult (licensing-wise) to bring in some of your code. I will battle the code details later. |
|
|
|
This looks awesome. Is it possible to connect to another host, so that we can make a transaction without having bitcoin actually running nor installed on localhost? This would be great, as it would be the cheapest (and possibly the safest) to use bitcoin. I also think this should be related to the Bitcoin Off-The-Grid project. +1 To this. I'm extremely interested in having a minimal networking interface just for getting txs not in the blockchain yet, and transmit new blocks that we just signed. How much work would that be--it sounds like tests/net is like 90% of the way there. If I'm working on an AGPL v3 project, will I be able to use your code (modified or not) in my project with proper attribution? I've been dreading how I'm going to deal with networking in my client, but this sounds like it will be the perfect solution! |
|
|
|
|
Most interesting. If I wasn't about to board a plane right now, I'd go clarify that on the Bitcoin Scripting wiki...
Do all OP_X codes push an empty string onto the stack? Or just OP_0? Is it safe to assume that it is an error if I somehow end up with HASH160(OP_3)?
|
|
|
|
I am crashing when I try to HASH160 an OP_0 byte, but I can't figure out what my scripting code did wrong.
I believe that "crashing when you try to HASH160 an OP_0 byte" is what your scripting code did wrong. ByteCoin Gah! That was stupid. I assumed that HASH160 should only ever operate on strings, and thus I must've done something wrong upstream to end up with a HASH160(OP_0). Upon closer inspection, I see that zero is actually a placeholder for the third public key that wasn't supplied. I just need to "define" HASH160(OP_0) to return...anything. That's what I get for making assumptions... Thanks. |
|
|
|
Okay, I am now battling the OP_CHECKMULTISIG example in my original post. I thought the point of OP_CHECKMULTISIG was to be simple and not require two-dozen other op-codes to get an M-of-N transaction. It seems to me that it is simple if you have everyone's public key, but not if you have their address. Is this really the simplest way to do M-of-N with 3 addresses? (see top post) Second of all, I'm a little confused about the script itself. I am crashing when I try to HASH160 an OP_0 byte, but I can't figure out what my scripting code did wrong. Perhaps someone with scripting experience can point me to my error. This is the exact state of my stack after every OP_CODE: TxIn Script:
OP_0: 0
OP_PUSH: 0 SigA
OP_PUSH: 0 SigA SigB
OP_2: 0 SigA SigB 2
OP_PUSH: 0 SigA SigB 2 PubKeyA
OP_PUSH: 0 SigA SigB 2 PubKeyA PubKeyB
OP_0: 0 SigA SigB 2 PubKeyA PubKeyB 0
TxOut Script (with stack from TxIn):
OP_3: 0 SigA SigB 2 PubKeyA PubKeyB 0 3
OP_ROLL: 0 SigA SigB PubKeyA PubKeyB 0 2
OP_DUP: 0 SigA SigB PubKeyA PubKeyB 0 2 2
OP_2: 0 SigA SigB PubKeyA PubKeyB 0 2 2 2
OP_>=: 0 SigA SigB PubKeyA PubKeyB 0 2 1
OP_VERIFY: 0 SigA SigB PubKeyA PubKeyB 0 2
OP_3: 0 SigA SigB PubKeyA PubKeyB 0 2 3
OP_ROLL: 0 SigA SigB PubKeyB 0 2 PubKeyA
OP_SIZE: 0 SigA SigB PubKeyB 0 2 PubKeyA 65
OP_NOT: 0 SigA SigB PubKeyB 0 2 PubKeyA 0
OP_OVER: 0 SigA SigB PubKeyB 0 2 PubKeyA 0 PubKeyA
OP_HASH: 0 SigA SigB PubKeyB 0 2 PubKeyA 0 PubAHash
OP_PUSH: 0 SigA SigB PubKeyB 0 2 PubKeyA 0 PubAHash PubAHash
OP_EQUAL: 0 SigA SigB PubKeyB 0 2 PubKeyA 0 1
OP_BOOLOR: 0 SigA SigB PubKeyB 0 2 PubKeyA 1
OP_VERIFY: 0 SigA SigB PubKeyB 0 2 PubKeyA
OP_3: 0 SigA SigB PubKeyB 0 2 PubKeyA 3
OP_ROLL: 0 SigA SigB 0 2 PubKeyA PubKeyB
OP_SIZE: 0 SigA SigB 0 2 PubKeyA PubKeyB 65
OP_NOT: 0 SigA SigB 0 2 PubKeyA PubKeyB 0
OP_OVER: 0 SigA SigB 0 2 PubKeyA PubKeyB 0 PubKeyB
OP_HASH: 0 SigA SigB 0 2 PubKeyA PubKeyB 0 PubBHash
OP_PUSH: 0 SigA SigB 0 2 PubKeyA PubKeyB 0 PubBHash PubBHash
OP_EQUAL: 0 SigA SigB 0 2 PubKeyA PubKeyB 0 1
OP_BOOLOR: 0 SigA SigB 0 2 PubKeyA PubKeyB 1
OP_VERIFY: 0 SigA SigB 0 2 PubKeyA PubKeyB
OP_3: 0 SigA SigB 0 2 PubKeyA PubKeyB 3
OP_ROLL: 0 SigA SigB 2 PubKeyA PubKeyB 0
OP_SIZE: 0 SigA SigB 2 PubKeyA PubKeyB 0 1
OP_NOT: 0 SigA SigB 2 PubKeyA PubKeyB 0 0
OP_OVER: 0 SigA SigB 2 PubKeyA PubKeyB 0 0 0
OP_HASH160: <ERROR>
Can anyone help identify what op-code I mis-applied? These two scripts are straight from the testnet, and the 0.4.0 client accepted them as a valid pair. |
|
|
|
|
If we could get a bunch of users to setup such forwarders on webpages with SSL, then technically users won't need Tor for the anonymity they seek.
The user accesses the tx-forwarding webpage which connects via SSL. The tx-to-be-broadcast is uploaded, sometimes it broadcast, sometimes forwarded to one of the other tx-forwarding webpages (also via SSL). The final IP address that broadcasts the tx is different than the IP of the originator, and as long as tx-requests are pooled and not sent immediately, then it would be tough to correlate the originator of the tx with any particular SSL connection in the logs (but of course, the webservers probably wouldn't be saving logs of the SSL connections anyway, so that's probably not relevant).
While this is great for anonymity, you as the webserver host should be aware that it does open up far-fetched-but-non-zero risk for you. If you allow people to upload arbitrary transactions, and those transactions are for illegal activity, the investigation might end up at your IP address, and you might be in a frustrating position.
I think the chances of this are extraordinarily slim. Additionally, theoretically, they shouldn't be able to do anything to if they can't prove you own the private key, but the legal system isn't always rational like this, especially when complicated technology/protocols are involved. It's just a warning -- I would do it anyway because I really think the risk is virtually non-existent (at least until countries start explicitly trying to outlaw BTC), but it's something worth noting.
|
|
|
|
My only missing pieces are getting the correct encoding for the keys prior to that whole Base58Check thing. I've got both the pem stuff and the Base58Check thing down.
-TT
If I understand correctly, doesn't the first half of my post cover that? Convert private key to 32-byte big-endian binary string. Prefix a 0x80 byte to it. Then double-sha256 and add the first 4 bytes of the result to the 0x80+PrivKey string. I am not familiar with these import formats, but my experience with address strings and the description you provided makes me pretty confident that's what you're looking for (if it's not right try encoding the private-key in little-endian before hashing). |
|
|
|
|
Show Posts
