With apt you can add your own repos. Perhaps rather than trying to get into the official debian repo, we could just make our own repo for Bitcoin like Tor has done ( http://mirror.noreply.org/pub/tor/). Anyhow, I think that libbitcoin is a great idea, but not exactly a project for someone new to C++. For now it has a very low level api, but in the future we obviously fix that like you have pyKDE built upon KDE built upon Qt built upon X built upon the kernel graphics stack  Anyway, I'm disconnecting from the forums for a few days to focus. I can be contacted using my email on the front of bitcoin.org |
|
|
|
I'll probably add cumulative difficulty into the blocks very soon.
Cool. Still thinking about various design issues though.
I might suggest one thing from experience with Abe... CREATE TABLE inputs (
...
previous_output_id INT,
previous_output_hash hash_type NOT NULL,
previous_output_index BIGINT NOT NULL,
A hash per input will be a large fraction of storage space. Abe avoids this by putting unlinked inputs in a separate table and then deleting them when it finds the output: CREATE TABLE txin (
txin_id NUMERIC(26) PRIMARY KEY,
tx_id NUMERIC(26) NOT NULL,
txin_pos NUMERIC(10) NOT NULL,
txout_id NUMERIC(26),
...)...
CREATE TABLE unlinked_txin (
txin_id NUMERIC(26) PRIMARY KEY,
txout_tx_hash BIT(256) NOT NULL,
txout_pos NUMERIC(10) NOT NULL,
FOREIGN KEY (txin_id) REFERENCES txin (txin_id)
)
After the unlinked_txin is deleted, you can get the previous output's transaction hash via txin.txout_id to txout.tx_id to tx.tx_hash. I'm not so sure that it's good to prune out the previous_output_hash since when getting an entire block, you want the entire info (which includes the hashes)- especially for displaying on a block explorer type website, hashing the transaction or performing the OP_CHECKSIG .etc The fact is that you'll be looking up the previous hash so often that it'd be a wasted optimisation and not gain too much either (storage is cheap today). Also impact on performance is minimal since we've got fetching blocks down to being blindingly FAST. There could be merit in updating the previous_transaction_id field when you connect the blocks, but I can't see a use case that often for cycling *backwards* in the blockchain (forwards yes, but not backwards). You only really look backwards once - during the initial block-chain verification which doesn't quite warrant the effort of adding this field. |
|
|
|
Ok that makes sense. I am a bit concerned that even though the update is a single statement, it must touch a lot of rows, but it should be infrequent enough to not be a problem.
My next question is regarding the very common operation of determining whether an output has been spent. Each output could be spent in a single transaction that belongs to multiple blocks (in separate forks) or by multiple transactions (also in separate forks).
So it seems that to determine if an output has been spent in a particular chain, you will have to query for all the inputs that spend the output, find (JOIN) the transactions that contain those inputs, find (JOIN) the blocks that contain those transactions, and see if any of those blocks have a span range that contains the chain in question. I know databases are supposed to be good at this kind of thing but it sure sounds like a lot of work. On the other hand, most of the transactions will not have a double-spend, so all the joining shouldn't make a difference in the most common case where there are no inputs that spend a txout. All the joining might slow it down a bit when the blockchain forks, but a few extra indexed lookups shouldn't be that big a deal I guess.
I'm thinking I'll switch to your schema with the spans, because it bugs me how my version has dynamic fields marking whether a block is part of the 'best chain' and whether a txout has been spent in the current best chain. It is a real headache to add a block to a fork other than the current best.
I plan to add a flag indicating whether an output is_final (and the parent transaction is_final - if all child outputs all have is_final set) and an output_address and for transaction a transaction_type (standard, generated and other) fields. Also will be another table with chain_id, cumulative_difficulty... and cumulative difficulty in blocks. For now still working on the block chain verification specifics (which algorithm is best and whether to organize then verify, pre-verify then organize then verify, or verify then link and organize). There are various pros and cons for all the different ways. |
|
|
|
|
I'll probably add cumulative difficulty into the blocks very soon.
Still thinking about various design issues though.
|
|
|
|
Ooh! Lots of yumminess coming down the pike. I look forward to using your library to help display transactions before they get into a block. It'd be nice to harmonize our database schemata somewhat. While I wrap my head around libbitcoin's span_left/span_right reorg strategy, I suggest storing total_difficulty in blocks (where it is immutable) rather than chains. Abe relies on this (block_chain_work) to estimate network hash rate over a range of blocks by fetching just the start and end blocks. Thanks! Are you on IRC? You should hit me up on #bitcoinconsultancy You can quickly query the total difficulty for a chain using a single SQL command: SELECT SUM(to_difficulty(bits_head, bits_body)) FROM blocks WHERE span_left=X AND span_right=X; Will give you the total difficulty in chain X (you have to have to_difficulty() defined as: bits_body * 2^[8*(bits_head - 3)] ) |
|
|
|
Great work, I just have a question about the span. I don't understand how it works, because if you fork the block chain, won't all the existing span numbers have to be incremented?
I've been trying to implement a sqlite backend (so far just dump to sqlite which is mostly working), and it seems to be helpful to distinguish between dynamic data and data which is immutable such as the successor relationship between blocks, transaction containment within a block, etc.
Whether a block is in the best chain and whether a txout has been spent in the best chain is dynamic, so I have to implement queries that more or less correspond to ConnectBlock and DisconnectBlock. I would love to be able to eliminate it using your span mechanism or something similar but I'm not understanding how it works.
I'm still working on ProcessBlock (verifying the blocks) and everything below it (ConnectBlock/DisconnectBlock). You can see an example of how the span works in src/storage/postgresql_storage.cpp:reorganize_block_chain()... That function will be moved internally and within verify_block under include/bitcoin/verify.hpp Indeed you're correct that all the spans of the parents would have to be incremented if you fork the block chain. This is OK since it can be done in a simple single UPDATE, inserting a new block is an infrequent operation (doesn't have to be as fast as SELECTs) and it's not a big deal if adding a new block isn't immediate. Far more important is the select speed. If you run the poller and postgres benchmark: $ ./bin/examples/poller dbname dbuser dbpass localhost:8333
... wait to download all the blocks
$ make psql
$ time ./bin/tests/psql
... run benchmark to fetch 100 blocks
With my current setup it takes 10 secs to fetch 100 blocks or 0.1 secs for 1 block. That means 4 hours to fetch the entire block-chain of 140k blocks. Using a rule of thumb that verifying the block-chain will be 10 times as slow, that might mean 40 hours to verify all 140k blocks. ATM I plan to just bundle the first 120k pre-verified blocks in the code, then later see about optimising this. But I am running it over a VPS on rackspace, so that also could be a reason for the speed too. It should be possible for you to easily modify the current storage engine I'm using to enable sqllite... The library supports multiple backends. See the constructor in src/storage/postgresql_storage.cpp:postgresql_storage() ... The sql_ object is being explicitly initialised with postgresql parameters. I don't mind making this a generic sql_storage backend if you're able to get it to work  The current way of joining the blocks using reorganize_block_chain() will be removed and done as part of the block-chain verification- that's the best way I see currently to do it... But I am doing more research of the current bitcoin sourcecode to make sure I understand everything fully. The current bitcoin is very inefficient/obtuse and there's a whole bunch of ways to do things quicker or nicer. |
|
|
|
Yep, I would reduce it to a single forum so all the posts could be aggregated there. Then expand with more subforums as needed. Also would be nice to see bitcoin.org.uk as a UK portal instead of a forum: - UK news - Links to UK bitcoin groups (Britcoin and Bitcoin Consultancy for instance  - Map of users UK wide .etc Everybody speaks English so there isn't too much demand for a UK forum like there is with the Polish one. More important is to link up and find out what's happening wrt Bitcoin in my homeland |
|
|
|
Great stuff.
Learned one thing though.
If you want to start something up that has to do with money biz you need 4 more things then just the front-end and back-end.
1) A 24/7 lawyer
2) Alot of money
3) A couple of skilled programmers
4) A additional security expert.
THIS--^ You have to start somewhere. I had none of those in the beginning, slowly accumulated them and now we have that and more. |
|
|
|
Yeah, it's dodgy. But this is off topic - please relocate this post to off-topic so we off-topic people can discuss it freely without fear of the Nazi elite who want to control all we do and think.
I'd rather people see this here before more people jump in on this. Already people have been buying lots of them and wasting mining power on generating them. |
|
|
|
Hey,
IxCoin simply has increased the reward for each block from 50 to 96. That means that every 4 years the reward halves so it should eventually reach 42 million.
However the hard coded max value in the code is 21 million. There are checks (see MoneyRange and ProcessBlock) in the source code which haven't been changed to still limit it to 21 million.
Once it reaches 21 million, all new blocks minted by miners (which are still using the same reward) will be all rejected and the network will shut down overnight.
How is it a scam? When Bitcoin reaches 21 million all new blocks minted by miners will be rejected. They fact they didn't raise the max blocks hardly makes it a scam. No, check CreateNewBlock. It calculates the value using GetBlockValue which doesn't hard limit the value to 21 million. This means that the network will break overnight once it reaches 21 million. |
|
|
|
Hey, IxCoin simply has increased the reward for each block from 50 to 96. That means that every 4 years the reward halves so it should eventually reach 42 million. However the hard coded max value in the code is 21 million. There are checks (see MoneyRange and ProcessBlock) in the source code which haven't been changed to still limit it to 21 million. Once it reaches 21 million, all new blocks minted by miners (which are still using the same reward) will be all rejected and the network will shut down overnight. Reasons I now think it's a scam after playing with it: - They only made one minor modification to the source code (changing 50 -> 96) - Already mined 400k IxCoins themselves. - Copied and leveraged bitcoin community with minimal input. Product doesn't have much value on its own. - wallet.dat is 52mb already. Within a year that will be 3gb. This is not sustainable in terms of computer resources. Initially I was ambivalent about them, but after talking with the other guys here (at Bitcoin Consultancy), it's looking like a classic pump and dump. IxCoin: https://bitcointalk.org/index.php?topic=36218.0 Also all the devs on the front page of bitcoin.org (me included), got this message: So... I got this unsolicited email: -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Gavin,
I'm one of the founders of the Ixcoin fork of Bitcoin. If you're
interested in trying out the new blockchain, we set aside a 5,000 IXC
bonus for each of you.
Please email me your IXC addresses if you're interested.
Best regards,
Thomas.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
iQGcBAEBAgAGBQJORIPxAAoJENjtb+Pw2dLBvqEL/31cNrOpH6nlirriyZXsjpQD
6TYgFUZxzNjv9y9ieWBAA+2xRZBzFufo1T2hFODWZ8PZO/oobu1yHq+mgAm4QKSV
Xz6JEhs6WSXQqMvTey12ks0gjJBquftjgQKmDh5UbdKQP+ffVVd0wo6FwRt4hvha
Ikdb4TK4dVjgc/IZLBVQ5O70sFmaQMbrqqVmS8nI55cxt8ywGBoLofdvQhKwr5kD
Z5dq4H/7O06IEciJQW3CSSnLXurOSn2WxnpV80aDHshpiT/jHXfiJAsRY1ofYo3C
5aDLFWRMgDRnThreuPJL32FQBclZzfHOJi8R6+swoRD7uYB43q84ZqKN8h9dOMrb
KcAW9Bw5NjtVEtp7iXNR/pcwMnC1BJIERtYWnscCJwDYv8TXwvb+/PwNVBw/aHCU
jIiqJ5iQguB4DB61R6C6+TzkCWmsBWHhgvn9CChIfv0D00tgWsVq7eOFVx2eaSM7
Tesm69MF1X7KNwp7Li8WukWnWUHFCA8ESKjPvMI1XQ==
=lct4
-----END PGP SIGNATURE-----
... and I'm trying to figure out why my immediate reaction is "yuck." I guess I just don't see the point of IxCoin. I just spent some time looking back through all the original poster's messages, and I don't see any prior discussion of goals of IxCoin, how it should be designed, etc. If the goal is to let more people be early adopters of a new cyber-currency, then it seems to me giving early bitcoin adopters a bunch of IxCoin goes against that goal. |
|
|
|
I started the Chinese version of the bitcoin wiki which at this stage has really developed.
On the English version I've only really added the page for GLBSE, I wasn't aware that there was a spamming problem, that's something I'd like to help sort.
I think the wiki has been a great resource for the bitcoin project, it's often where users(newbs) go for more information and is in a sense an informal center of bitcoin documentation. The wiki must be preserved.
Wiki username: nefario
nefario is also a great choice. My votes: nefario and sgornick |
|
|
|
|
We discussed here, and our general idea would be to move our site's repo off that project on gitorious and open that repo on gitorious to be open to the community.
If everybody wants that.
|
|
|
|
This thread was started as the result of a conversation between me and MagicalTux on IRC. The wiki currently only has me, MagicalTux, nanotube and Kiba as admins, and we're busy that we don't get time to do the necessary cleanup always. As a long term administrator of the wiki since it's inception, I'm going to nominate sgornick on his behalf. He has been active in the community and is well trusted as the owner of bitcoinnews.com and bitcoinminer.com among his other sites (his projects are listed at https://en.bitcoin.it/wiki/User:Sgornick). He's been proactive with fighting spam and adding new info to the wiki, and enabling him with the tools he needs is not a bad decision. |
|
|
|
Done! BTW, if you have a repo and tell me certain commits that may be useful then we can pull them into the master |
|
|
|
Fellas!
I forgot one thing, the bank statement parser...
Until now, you must access your Internet Banking and export your bank statement to a CSV or TXT file, to import it into Internsango using some scripts under the intersango/cron directory.
BTW, this is the only missing part for my own Intersango installation here in Brazil, I'm working on it!
Maybe the Intersango guys can help us! We post here our bank statement as a example, and somebody adjust the parser for us, I can pay in Bitcoins for this job...
Cheers!
Thiago
Generally the gbp branch is more up to date and newer... I keep trying to migrate things over but occasionally forget but phantom won't let me delete the master branch -_- since he says others should use it. Anyway I copied over more bank stuff for you, https://gitorious.org/intersango/intersango/commits/masterYou should mainly be interested in import_csv_hsbc.py It imports CSV files by accounting for overlap and duplicate entries. phantom also wrote the parse_deposits.py which searches for anything that looks like the deposit reference on that line. Hmmm, might be a good idea for v2 that the deposit reference starts with DPsk32jkjs to make finding it easier, and then a script that generically finds the amount somehow... |
|
|
|
|
It would be cool if we're not the only stewards but we have lots of groups collaborating, pushing and pulling from each other to build a repository of scripts up for dealing with all the different bank variants and so forth.
|
|
|
|
|
Show Posts
