Any chance to update PYPI, please?
A release? It's sure overripe for one. Noted. |
|
|
|
...
block_tx 2259 2289
commit
block_tx 2260 2290
commit
block_tx 2261 2291
By the way, I don't know if anyone has ever loaded the whole thing in SQLite. I'll be surprised if it works and more surprised if it takes under a month. I recommend MySQL or better for the BTC chain. And regardless of database, an initial run with --no-serve --commit-bytes=100000 will probably speed it up. I have a pretty weak machine (512MB VPS) that takes several days to load BTC into Abe using PostgreSQL. |
|
|
|
OverflowError: long too big to convert
Looks like this problem. See if --int-type=str solves it. I was unable to reproduce it, IIRC. |
|
|
|
I'm trying to run this with sqlite and am getting unicode errors from the "magic" values in CHAIN_CONFIG (magic values set in lines 52-66 of DataStore.py) sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.
I seem to remember this error, perhaps with an older SQLite. 3.7.7 does not give me it. Anyway, Abe is supposed to test for binary data types, but the test used only bytes with bit 8 clear. I've changed that, so the test should now reject bytestrings and use binary_type=hex on your system. Please retry with this latest fix and post the result. In README-MYSQL.txt there are instructions to create a new user abe - I had to run FLUSH PRIVILEGES; after creating the user.
Really? What mysqld version, please? The 5.0 docs say: With CREATE USER, FLUSH PRIVILEGES is unnecessary.
|
|
|
|
My Abe installation stopped updating after block 188421. (2012-07-10 11:26:50) The problem appears to be that Abe only looks at blk0001.dat by default and does not look at blk0002.dat, which my Bitcoin client created after block 188421.
Well, I had the same problem, but it turned out to be a permissions issue. I had forgot that I made blk0001.dat world-readable to be shared by different users on my system, and I had to do the same for blk0002.dat. Once I did, Abe recognized it and continued processing. The real problem, at least in my case, is Abe's lack of error reporting for the case where blk0002.dat exists but is not readable. That should be easy to improve. But if adding a datadir row solves it for you, there must be something else wrong. Has anyone else seen this? |
|
|
|
I would be happy to try your patch for Abe. Is it online? Are you able to show something like http://blockchain.info/fb/3g2 already? (it has a 2of3 signature). It doesn't work on firstbits as well as on "vanilla" Abe. Sorry, I have not implemented this address type for Abe. Everything I've done is checked in.  The rules I proposed above got no reply, and I lost interest. Abe does not follow them exactly; for example, it ignores trailing OP_NOP when processing an output script. I have not taken the time to find test cases to determine how firstbits.com and blockchain.info handle nonstandard and newer script types. Here is the latest on Abe's implementation: https://bitcointalk.org/index.php?topic=22785.msg965329#msg965329 |
|
|
|
Before I went to upgrade, I found that Abe had stopped working. My block file just exceeded 1GB in size a couple hours ago which is causing this error: File "Abe\BCDataStream.py", line 27, in map_file
self.input = mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ)
WindowsError: [Error 8] Not enough storage is available to process this command It is because I'm running a 32-bit system. Apparently this issue will eventually affect Linux 32-bit systems as well because of the way mmap works; except that it will occur when the block file hits 2 or 3 GB in size. Is Abe particularly tied to mmap? If it is, it seems like it will soon be (or is now) for 64 bit systems only. Interesting. I don't think it will affect Linux, because bitcoin starts a new file (blk0002.dat) before 2GB: Just wanted to add: this mmap() issue is a problem on my 32-bit Linux system as well, with a block file size of 1.8GB. I'll try changing the bitcoind file size limit and then try again. (Or maybe I should just dump the ancient 32-bit machine... ) Thanks for the report. I saw the error too on Linux but not consistently. I guess it did not find enough contiguous process address space or hit some system limit. I think a reasonable approach would be to implement Python's sequence protocol to replace mmap() in Abe/BCDataStream.py. The implementation may be read-only, need not support negative indices, but must support slices. I would try to keep using mmap() underneath but with length and offset specifying just a chunk of the file. I would have some bookkeeping around chunk boundaries to keep the interface compatible. Using read() could work, too. Such a fix should interest Gavin for use in BitcoinTools if he's not already working on it, but he may need the array writable. I do not anticipate having time to work on this in the next couple of weeks. |
|
|
|
|
Midskes: thanks for the report. I've fixed the new installation bugs and tested on MySQL.
|
|
|
|
Update on Firstbits before I go back into hibernation. I do not trust the code to produce the same results as firstbits.com or blockchain.info. I trust it to provide nice short links such as http://abe.bit/a/14c59f, at least until I change the algorithm and a few of the links change their redirect target. If you care about agreement with other implementations, which you should if you want to give people your firstbits address to send coins to, I see two approaches. 1. Get agreement on a standard, and wait for everyone to declare adherence to it. See the firstbits thread: https://bitcointalk.org/index.php?topic=16217.msg960077#msg960077or, 2. Reverse-engineer the implementation of your choice. See how it handles all extant script types, document the algorithm, and press implementers to confirm, correct, propose modifications to, or implement it. The demo site http://abe.john-edwin-tobey.org/ is running the Firstbits code, but I will have to turn it off again if it gets a lot of traffic. |
|
|
|
I am happy with these rules. I suggest something like the following to define "appearance" in the chain: A pubkey output is a transaction output with a script of the form: An address is said to appear in a pubkey output when its version byte is 0 and its key hash equals the 160-bit hash of the output script's push operand (pubKey). A pubkey hash output is a transaction output with a script of the form: OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY OP_CHECKSIG An address is said to appear in a pubkey hash output when its version byte is 0 and its 160-bit key hash equals the output script's push operand (pubKeyHash). A script hash output is a transaction output with a script of the form: OP_HASH160 <scriptHash> OP_EQUAL An address is said to appear in a script hash output when its version byte is 5 and its 160-bit key hash equals the output script's push operand (scriptHash). An address is said to appear in a block when it appears in one of the block's transactions' pubkey, pubkey hash, or script hash outputs. In particular, output scripts not matching the above three cases can not affect whether an address appears in a block, nor can input scripts. For example, a script containing OP_DROP or OP_NOP can not match the cases, but the push operand in a pubkey output may have any length. |
|
|
|
Want to see what Piuk thinks and then lock it in 'officially'?
Request sent. |
|
|
|
The "escrow" address (as labeled by blockchain.info) is not added as a firstbits address at this point in time. All other addresses involved in a multisig transaction would be added as firstbits addresses.
Would you consider defining "appears" on http://firstbits.com/about.php to exclude appearances in OP_CHECKMULTISIG outputs? |
|
|
|
Hi, I recently implemented firstbits for Abe. What do you propose to do if/when a previously unseen address appears in a multisig output or (horrors) a script-hash input? Neither Abe nor BlockExplorer recognizes multisig scripts, but bitcointools decodes them into addresses, which blockchain.info displays: example. That site also displays firstbits of script-hash addresses: example. The addresses I've looked at all appeared earlier in the chain in "normal" pubkey or single-address outputs, but nothing's to prevent someone from sending to a multiple-signature destination that includes a new address. Would that allocate a new firstbits? I'm tempted to say "no" out of laziness and because Firstbits are for published addresses, whereas multiple-signature transactions are more for self-transfers. |
|
|
|
I've committed Firstbits calculation to master. There is not yet an API or UI, but if you upgrade and run with "use-firstbits=true" in config, Abe will create and maintain the abe_firstbits table as described here: https://github.com/jtobey/bitcoin-abe/blob/master/README-FIRSTBITS.txtIf you run without use-firstbits, Abe will default it to false and will never create the table. I'd like to have a script that turns firstbits on and off, but for now the best you can do is to stop Abe and run these (UNTESTED) SQL commands, once you have configured use-firstbits=true: DELETE FROM configvar WHERE configvar_name = 'use_firstbits' AND configvar_value <> 'true';
UPDATE configvar SET configvar_value = 'Abe29.3' WHERE configvar_name = 'schema_version' AND configvar_value = 'Abe30'; I tried a few dozen addresses, they match firstbits.com. Please report issues here or by email, PM, or the github issue system, since I will not spend much time testing. Edit:It seems the last available 4-character firstbits appeared in block 160686, '1u7c'. I'd appreciate if another implementer can confirm this. abe=> select max(cc.block_height) from chain_candidate cc join abe_firstbits fb using( block_id) where cc.chain_id = 1 and fb.address_version = '00' and cc.in_longest = 1 and length(fb.firstbits) < 5;
max
--------
160686
(1 row)
abe=> select fb.firstbits from chain_candidate cc join abe_firstbits fb using( block_id) where cc.chain_id = 1 and fb.address_version = '00' and cc.in_longest = 1 and length(fb.firstbits) < 5 and cc.block_height = 160686;
firstbits
-----------
1u7c
(1 row)
Twelve 4-character combinations can never be firstbits (according to http://firstbits.com/about.php) because they first appeared as a prefix of two different addresses in the same block. Regardless, however, firstbits.com shows them, perhaps because the code still follows the original spec, "Ties in a block are by tx ID." Any preference here? I don't see a big problem with keeping the extra characters. It's possible some addresses may never have a firstbits, because an earlier address differs only by upper versus lower case. I think this would be "cryptographically hard" for two valid public keys, but someone could shadow a valid (or invalid) address with a black-hole address. In that case, abe_firstbits.firstbits should hold the empty string. I have not found a real example as of Block 169770, but I'm still finding firstbits on my weak VPS... |
|
|
|
I'm very interested in seeing Firstbits support.
what exactly do you have in mind? an api call "/q/firstbits/15ArtC" that returns "15ArtCgi3wmpQAAfYx4riaFmo4prJA4VsK"? Yes. I'd prefer some SQL code, but I know that Abe doesn't store addresses in base58 form for now, so that'd be difficult to implement. What you describe is adequate. I have committed some preliminary work on Git branch " firstbits". It is not yet usable. It purposely throws an error to avoid incrementing the schema version after it populates the abe_firstbits table, since it has no logic yet to maintain the data when new blocks arrive. So don't use it. I don't know if or when I will have time to finish this work, so I put it out there for intrepid developers to pick up. My design would have firstbits support enabled by the --use-firstbits option, which is consulted only at install time or upgrade time. The data looks like this: mysql> select * from abe_firstbits order by pubkey_id limit 10;
+-----------+----------+-----------------+-----------+
| pubkey_id | block_id | address_version | firstbits |
+-----------+----------+-----------------+-----------+
| 1 | 1 | 00 | 1 |
| 3 | 3 | 00 | 12 |
| 4 | 4 | 00 | 1h |
| 5 | 5 | 00 | 1f |
| 6 | 6 | 00 | 15 |
| 7 | 7 | 00 | 1j |
| 8 | 8 | 00 | 1g |
| 9 | 9 | 00 | 16 |
| 10 | 10 | 00 | 1j6 |
| 11 | 11 | 00 | 12c |
+-----------+----------+-----------------+-----------+
10 rows in set (0.00 sec)
I expect this to support two-way lookup, /chain/Bitcoin/q/firstbits_to_address and address_to_firstbits. I plan to use it in address short links where enabled. I expect the firstbits table to grow quite large, otherwise I'd enable it by default. Omitting address_version (thus supporting only Bitcoin) would be a nice option, since it would reduce the space requirement. Since Abe is fundamentally multi-coin, I don't plan to implement this optimization. Perhaps it could be achieved using views or triggers. |
|
|
|
Apparently 19.53408036 was the total received as of Block 180750 and until Block 180779 (8 days ago). You would see that number if your database were not up to date. You can check it through the browser interface or with SQL: SELECT MAX(block_height) FROM chain_candidate
WHERE chain_id = 1 AND in_longest = 1; The output should be the current block number, 181904 as of right now. Check bitcoind and Abe's log if you get a lower number. |
|
|
|
Is this donation address still valid?: 1PWC7PNHL1SgvZaN7xEtygenKjWobWsCuf
Yes, it is. Thanks. |
|
|
|
Oh yeah I forgot to ask, what needs to be changed to this query to get the "total over all received" balance?
Take out the first sub-select and simplify, leaving this: SELECT SUM(txout.txout_value) / 100000000
FROM pubkey
JOIN txout ON txout.pubkey_id=pubkey.pubkey_id
JOIN block_tx ON block_tx.tx_id=txout.tx_id
JOIN block b ON b.block_id=block_tx.block_id
JOIN chain_candidate cc ON cc.block_id=b.block_id
WHERE
pubkey.pubkey_hash = LOWER('11B366EDFC0A8B66FEEBAE5C2E25A7B6A5D1CF31') AND
cc.chain_id = 1 AND
cc.in_longest = 1
|
|
|
|
Dude your awsome!, I would have never figured that query out on my own -- works great too
Thanks for confirming. |
|
|
|
|
Show Posts
