1. Define its protocol then implement tests of it then implement code that satisfies the tests in a modern programming language by a software engineer and produce documentation. 2. Isolate consensus engine from wallet mining and UI at best implemented in separate processes. 3. have a trusted communication channel (bus) instead of RPC. 4. Have a voting algorithm for hard forks. 5. use an industry standard message serialization e.g. BSON. 6. use regular 2 complement integers with fixed 32 bit length in the script language. 7. have SIGHASH_WITHINPUTVALUE https://bitcointalk.org/index.php?topic=181734.0 this is just a quick list ... completing this list should be point 0. |
|
|
|
|
I am not into starting a porn site, but tried to sell my technology to someone operating a couple of them. Thereby I learned what I wrote in the OP and shared it like a good community citizen. That is all.
Yes, I think porn sites should use Bitcoin as it would fit their customer's need, but whoever tries to sell that to them has to come up with a method that can compete with the revenue stream of their current scheme.
|
|
|
|
|
At the first sight Bitcoin is ideal for porn since it protects customer privacy and cuts payment processing costs for the provider. Like many in the community I expected for long that Bitcoin will pick up in that industry and kept wondering why it takes so long...
... until today, when I learned from an industry insider why Bitcoin is not accepted at porn sites:
1. Because nearly all porn sites have introductory offers that lead to a subscription with recurring billing on the credit card until the customer explicitly recalls it. The fees are calculated with the assumption that most customers forget to cancel at least a couple of times or not even recognize that they pay for a service that they no longer use.
2. Because affiliates - those who direct traffic to the porn site - want a share of those fees in upfront. An affiliate receives a multiple of the introductory offer immediately if someone he directed to the site subscribes for a plan. Affiliates do not want the site to use bitcoin either.
3. Porn sites have difficulties working with payment processors and are called 'high risk' industries, but not because of their customers or products, but because they themselves misuse the credit card re-bill features (see 1) and cause lots of customer complaints at the credit card issuer.
4. Biggest porn sites therefore have their own credit card processor.
Summary: Bitcoin would be beneficial for customer of porn but will not happen until the industry has any other choice.
|
|
|
|
The reason we aren't spending much time on fixing the problems with the current wallet is that we don't intend to keep our current wallet for very much longer.
Implementing your suggested workaround would not take much time. I think the problem is that development is not prioritized following end-user or business needs. Decisions are rather technocratic or a the team core member simply commits time on the solution of a problem of his choice. The Bitcoin Foundation could have been coordinating this but failed miserably (also) in this respect. It is really time to come up with some governance of the project Bitcoin, including but not limited to the narrow scope of twiddling the code of bitcoin-qt. Take this as my new year's wish. |
|
|
|
I much prefer linking the keypool extension to the backup process. As in, the backupwallet RPC command extends the keypool, then does the backup, and there would be no other way to generate new keys.
this. I keep wondering why an obvious source of loss for the majority of user is being ignored for years. This is like the past opposition to wallet encryption or the current to https://bitcointalk.org/index.php?topic=181734.0 |
|
|
|
Your own software implements BIP32 as well, now— and while I cannot say you rushed... but please, if you're going to brag about some feature you have that Bitcoin-QT lacks, please don't pick one that Bitcoin-QT lacks because instead we were spending time building the feature for everyone else— including you— too. Your patches are welcome to Bitcoin-QT as well. Or does the collaboration only flow in one direction?  If I brag, then that I put BIP32 it into commercial use quite a few months ago. Or was that not its purpose? I remember asking sipa, a day before the San Jose conference in May on IRC, not to change the spec again until I finish my product presentation with it . So yes, I rushed because it is/was much of a need. I do not think I ever intentionally missed to credit this elegant design those who made it. Seven months passed since then and this masterpiece of your design did not made it to bitcoin-qt. I believe that this is not because of resource constraints, since the core team has more and better qualified resources than I do, but because bitcoin-qt is a work of a genius, a proof of concept code and a hairball written in an ancient language in a monolithic style that runs an economy of billions. Sure I can move quicker without that ballast and responsibility. This is why people create start-ups, what BOP is, because working from a clean sheet with new technologies and without inherited liability is just more productive and fun. It is not like I would not contribute to the success of the project Bitcoin. I open sourced my work more than a year ago and keep extending it: https://github.com/bitsofproof/supernode Maybe I will come up with something truly amazing comparable to BIP32, I wish for, but how could promise that. I am a quant developer and investment banker, not a cryptographer, not even a mathematician. Did I not stop using C++ more than 10 years ago (for good reasons), I would likely add more to the bitcoin-qt code. As I open sourced the first version of BOP I was asked: I wonder how the author plans to keep up with all the development done in bitcoind though... they have many competent people improving the code and adding functionality. Just to keep up would be an enormous effort. But anyway, writing such a thing was certainly an enormous effort already!
Now, BOP has products that go in usability for business beyond that of bitcoin-qt and I expect the gap to widen further. It would be great if the core team would stop bloating that monolithic code base as if it would be able to address the problems of everyone with one solution. We rather need refactoring that isolates consensus protocol, creates a trusted communication channel inside the company (instead of that badly designed RPC) and removes the wallet from bitcoind. End user use mobile wallets and companies need a border router with extensible interfaces and a frozen consensus engine and lots of custom functions that create business value. If there was a push for refactoring in above sense and if there were funds to work on it, then I am on board. In the meanwhile allow me to be proud of my hard work. I do not need to step on bitcoin-qt to stand out  |
|
|
|
It is poor design as you describe.
You can loose money if bitcoin-qt's wallet is not regularly backed up.
Deterministic key generation schemes that overcome this problem were designed long ago and exist in many alternate implementations e.g. Armory, Electrum .... and BOP.
i guess even regular backup wouldn't help in this case: even if you make an offsite backup right before running the client, if your harddrive fails while the client is running and you already made 200 transactions during that session. are you out of luck? (doesnt have to be hdd failure, any problem which would cause the wallet.dat to be destroyed or corrupted, like power outage, etc). You seem to recognize why there is a market for an enterprise implementation of Bitcoin |
|
|
|
|
It is poor design as you describe.
You can loose money if bitcoin-qt's wallet is not regularly backed up.
Deterministic key generation schemes that overcome this problem were designed long ago and exist in many alternate implementations e.g. Armory, Electrum .... and BOP.
|
|
|
|
Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind.
I hope you succeed, as there is no push in the core team to isolate concerns. Why could there not be a crowdfunding effort to pay someone to push improvements upstream from the reimplementations? Do BoP and btcd both have more comprehensive unit tests than the Satoshi client? Surely everyone would benefit from getting that same level of coverage in the reference implementation. At some point all there needs to be inter-implementation coordination of features and specs, kind of like the W3C for HTML. Unit tests of satoshi that were portable (means defined in JSON input and expected output) are all supported by BOP. It has some additional unit tests but is not as impressive as btcd sounds. I have lots of tests on much higher level, testing behavior of production systems that build on BOP, these however do not help the community as not open source. I did expect that the Bitcoin foundation would fill the role you describe, but seen nothing noteworthy. They instead focus on US lobbyism and building a profitable international franchise. |
|
|
|
Good catch! Thank you. The reason is that the context on memdb-rofile.xml was missing. it should have started with:
<?xml version="1.0" encoding="UTF-8"?>
<beans profile="memdb"
The reason it worked for me is, that without the context the load order of the files determined precedence beetween leveldb and memdb - and it seems this load order was different on CentOS. Fixing on github soon. |
|
|
|
it took less than 7 manyears labor to hack out over the past 10 months and we'll be at feature parity within the next 2 months, at most.
Congratulations. I know how that work feels. It took me about half a man year to reach feature parity in Java though - passing the block tests used by satoshi and bitcoinj - thereafter I worked mostly on higher level projects using the code in production. Regarding 100% security in distributed consensus I think in the meanwhile that it is a moving target we will not reach until we split out and freeze that part of bitcoind. I hope you succeed, as there is no push in the core team to isolate concerns. |
|
|
|
This all seems needlessly complex just to facilitate reasonable scan times. How does Electrum do it from just a seed?
You do not need to get the details why this is hard, just recognize that the only reason we have key birth date is because it reduces scan time. This lesson was learned while importing ordinary (WIF) keys or re-scanning the wallet. Now, developers who implement a HD scan will recognize that birth date is not sufficient for reasonable scan time, but at least a look ahead window is needed to master the task and that eventually birth date might be re-set if start-index is also there. Certainly these assumptions require that the Wallet software that uses HD complies with that. I do not see why this would be a problem why is it important to protect freedom of key sequence use. The point of HD is to avoid key reuse and loss through missing backups. Both can be achieved even if there are restrictions on what key to use next. |
|
|
|
|
Yes, that is also rather strange. The blocks should be persisted as received in the data directory. I am reluctant to believe that it just does not work as I have several instances running fine since months in production, but will do a fresh bootstrap in the next days to see how it should look like at the beginning.
Maybe CentOS related problem. I use ubuntu or OS X on all machines.
|
|
|
|
But instead of making us ask, why not just share your scanning algorithm here for the people reading along?
The algorithm is outlined in this thread and I open sourced the BOP Bitcoin implementation that includes this. I continue improving it free of charge for the community and even implemented this proposal there. Can I substantiate my claims better? |
|
|
|
|
Certainly, my solution uses a minuscule key space, to have that is the whole point of its assumptions.
Well, I retreat from arguing with an algorithm that is not implemented and postpone discussion of this topic until I see an actual implementation of what you describe or until you recognize the nature of the problem.
Regarding brute forcing the check sum: It does not really matter if it takes days or weeks. It is a precautionary measure to create a fake solution in addition to the real one. One only have to have it ready in case the plausible denial should be exercised, which is not a panned event in the near future. The harder the problem is the more plausible the denial. As said earlier passphrase input could be BIP39 encoded, this way any brute forced bit pattern will look equally plausible.
|
|
|
|
|
Seems like peers were suddenly no longer reachable. Hard to tell, but looks like some general networking problem. In case you run on the production network I would suggest you run it as a slave behind satoshi.
|
|
|
|
Just a note for thread readability to others trying to follow this:
payee = recipient ; payor = sender of payment
Payee appears to be used to refer to the payor in this thread?
My fault. I used customer = payee although the customer in the discussion was payor. |
|
|
|
What do you build the bloom filter on? What does a hit tell you and how does that help you to reconstruct funds accessible to keys derivable from the HD root? Forget the tree for the moment just enlighten me on how you would handle a flat key space of size 2^32 used in random order.
Granted, you wouldn't be able to handle a 2^32 space in random order without constructing a bloom filter of that size, but when you create a ridiculously large 'window' of, say, 10 million addresses, and start scanning with that, you basically no longer care much about the fact that payments may come in out of sequence even if you give the addresses out in sequence (mostly because the out-of-order-ness will be in the 100's or 1000's range). You start constructing a second bloom filter when you start to hit addresses that are getting close to the upper limit that you're currently scanning (say, 500,000 under your 10m boundary) and slowly keep adding to the second filter. When you haven't found a positive hit in the first filter, for X blocks / transactions / whatever metric you want to use, you can discard it and start working on the third filter, etc. As for what you build a bloom filter on, it's the same bloom filter that SPV clients use to request transactions from peers. They send a bit field with randomly set bits that correspond to indices generated by hash values based on the address used (either sender or receiver). The number of indices (what I called hash functions previously) and the bloom filter size can be varied. In the case of Bitcoin-qt / bitcoinj, the bloom is limited to 36,000 bytes maximum. It's a flat array, so lookups are constant regardless of size or number of elements covered. The filter guarantees that all addresses you're looking for will pass the filter, but the reverse is not true. It's possible that unrelated addresses can pass the filter (and in bitcoin-qt / bitcoinj this is actually used to preserve privacy, by dirtying up the filter a bit). It's a really fast and lightweight way to filter big amounts of data down to mostly what you're looking for plus some cruft. Then you can move on to more expensive filters, like hash tables and finally doing exact matches. Here's the bloom filter wiki if you want to read up on it some more: http://en.wikipedia.org/wiki/Bloom_filterSo you create 10 million addresses of the HD root - lets ignore that this is already a computation intensive task -, then you ask a giant bloom filter you previously populated with addresses used on the block chain and bingo you figure that 100.000 (1%) of them is possible used on the block chain. Does that tell you anything about the funds on them, nope. You still need to fetch 100.000 using some traditional database index on address. Does that tell you the funds on them ? nope. Since even those you feteched might be spent already and you would need to also fetch transaction eventually spending those outputs. Is this sufficient? nope You are marely done with 1/400th of the key space. This is you call feasible algorithm? me not. BTW, I am familiar with the bloom filter and use in my Bitcoin implementation not only for SPV support. You should rather be more curious of how I solved the problem of figuring funds for HD root instead of proclaiming without any practical experience that the parameters I propose are not needed. |
|
|
|
We have not a 1 or 10 million possible addresses but 2^32 that is 429 times 10 million. Scale your numbers accordingly. Also keep in mind that retrieving those 1% false positives is not cheap plus lookup is usually not just for transactions benefiting an address but for transactions that spend that output.
I think you will revert to the suggestions after you attempted to implement a HD scan. I did.
There's no reason to scan all possible addresses at the same time. You could make a sliding window with a bloom filter the same way you would with your 100 address lookahead. That 1% will be culled mostly by that hash table that you've got after it passes the bloom filter. And the last step with the branch reconstruction culls 100% of the false positives that managed to get through. Either way, there's no need to include a lookahead count in the encoding. As for the sequence start, what's the value of having a non zero sequence start number? I do not get it. What do you build the bloom filter on? What does a hit tell you and how does that help you to reconstruct funds accessible to keys derivable from the HD root? Forget the tree for the moment just enlighten me on how you would handle a flat key space of size 2^32 used in random order. |
|
|
|
|
Show Posts
