Bitcoin Forum
December 06, 2016, 08:07:10 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 [21] 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 175689 times)
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
March 23, 2013, 12:28:51 AM
 #401

Any news on multisig support? Are you working on it or waiting for donations? Smiley
i would contribute to a multisig bounty.

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
1481054830
Hero Member
*
Offline Offline

Posts: 1481054830

View Profile Personal Message (Offline)

Ignore
1481054830
Reply with quote  #2

1481054830
Report to moderator
ShadesOfMarble
Donator
Hero Member
*
Offline Offline

Activity: 543



View Profile
March 23, 2013, 12:34:13 PM
 #402

+1

Review of the Spondoolies-Tech SP10 „Dawson“ Bitcoin miner (1.4 TH/s)

[22:35] <Vinnie_win> Did anyone get paid yet? | [22:36] <Isokivi> pirate did!
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 25, 2013, 03:44:07 PM
 #403

Any news on multisig support? Are you working on it or waiting for donations? Smiley
i would contribute to a multisig bounty.
Like a zero-fee transaction, eventually it'll get in.  Probably.  Smiley  In the absence of a business model, my priorities are driven by my personal needs, which lately are about space efficiency, bitcoind 0.8.x compatibility, and integrating contributions from other developers.  Donations do encourage me to work, at roughly $40/hour for features that I want to add anyway, up to $100 for site-specific things that don't improve Abe for most users.  (I'd quote a price in BTC, but BTC/USD is too volatile, as I have ranted in other threads, for reasons that led me to create Abe in the first place.)

I'd like to get clear what "multisig support" means.  Now that I have a full database, I can search for new transaction types.  There have been a few hundred transaction outputs that use OP_CHECKMULTISIG as specified in BIP 11 M-of-N Standard Transactions.  Blockchain.info calls these "escrow" outputs.  Examples: 1 2.  Given an address that redeemed an M-of-N output, Blockchain.info lists the transaction, as in this example.  But if the address did not sign it, its page does not list the transaction: example.  Mapping from addresses back to these transactions will take some extra work.

There have also been a few hundred outputs as in BIP 16 Pay to Script Hash (P2SH).  These have receiving addresses that start with "3" as per BIP 13.  Example: 1.  These output scripts are opaque, but the redeeming input reveals one or more addresses.  As far as I know, Blockchain.info does not correlate an address so revealed with the original P2SH transaction.  Perhaps it would be useful to do so, and this would require some extra work.

There have also been a few thousand regular pubkey outputs as in this example that Abe can not parse because the public key is shorter than usual.  This has nothing to do with multisig and is really a bug in Abe, but it would be nice to fix.  There are also a thousand or so outputs of miscellaneous types unrecognized by Abe.  They may all be nonstandard, but it would be nice to make sense of any that are frequent or demonstrably redeemable.

ThomasV sent me this:

Hi,

it would be nice if Abe could support multisig addresses. For this I think you only need to update deserialize.py.

I did it in Electrum, so you can lookup Electrum's deserialize.py file:
https://github.com/spesmilo/electrum/blob/master/lib/deserialize.py
(you don't want to use that file as is, it probably has other differences)

see at the end, the function called get_address_from_input_script.

It is not directly usable: Abe would have to pass the script to the function where currently it passes pubkey_hash to another function.  But this would be very easy and would add at least some value regarding P2SH.

A full implementation of M-of-N, P2SH, and mapping from address back to M-of-N transaction would take me about 20 hours, so figure 11 BTC at current rates.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
ShadesOfMarble
Donator
Hero Member
*
Offline Offline

Activity: 543



View Profile
March 25, 2013, 03:54:59 PM
 #404

I would pledge up to 2 BTC (for full support).

Anyone else?

Review of the Spondoolies-Tech SP10 „Dawson“ Bitcoin miner (1.4 TH/s)

[22:35] <Vinnie_win> Did anyone get paid yet? | [22:36] <Isokivi> pirate did!
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
March 25, 2013, 07:12:08 PM
 #405

I would pledge up to 2 BTC (for full support).

Anyone else?

4 BTC (at today prices)

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
March 30, 2013, 05:39:06 AM
 #406

The experimental branch (no-statistics) now imports UNCONFIRMED (MEMPOOL) transactions (as well as blocks) over RPC from a running bitcoind, provided that:

  • Bitcoind is new enough to support getrawmempool, getrawtransaction, getblock, and getblockhash (tested 0.8.x; 0.7.x may work)
  • If using bitcoind 0.8.x, -txindex is in effect (see here)
  • The datadir.chain_id column is populated: for BTC, this means execute the following SQL: UPDATE datadir SET chain_id = 1 WHERE chain_id IS NULL

My demonstration server satisfies these requirements and includes unconfirmed transactions in search results.

This feature ought to be easily portable to the master branch, in case anyone wants it.  However, I'd feel more comfortable with a configuration option to tell Abe whether or not to use RPC.  By default, it tries RPC and falls back to blockfile scans if that fails.  Also, the new RPC code could use testing and improvements.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
April 01, 2013, 12:22:28 AM
 #407

Absolutely awesome!

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
akaihola
Newbie
*
Offline Offline

Activity: 13


View Profile
April 18, 2013, 12:55:16 PM
 #408

John,

Does the current master branch work with bitcoind 0.8.1 if the -txindex=1 setting is used?
What about the no-statistics branch?
Can I leave bitcoind 0.8.1 running while using Abe?

Also, do you know any way to check that -txindex=1 is indeed active and was used for the last reindex? My ~/.bitcoin/blocks is currently 8.6GiB of which 837MiB is index. ~/.bitcoin/chainstate is 208MiB.

Thanks for your work, I just donated ฿0.2.
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 18, 2013, 03:16:20 PM
 #409

John,

Does the current master branch work with bitcoind 0.8.1 if the -txindex=1 setting is used?
What about the no-statistics branch?
Can I leave bitcoind 0.8.1 running while using Abe?

In theory, both branches work with and without txindex.  You only need txindex if you want to load data over RPC from bitcoind.  Benefits of RPC loading are (1) you see unconfirmed transactions, and (2) large new blocks show up faster because most of the transactions are already in (provided that you frequently force a catch-up by loading a page or running in --no-serve mode).  See the comments about "default-loader" in abe.conf or Issue #17 about setting up RPC.

You might even manage to run in RPC mode without txindex, given an already loaded database, but this situation is vulnerable to a condition where bitcoind processes an output and its spending input before Abe loads the output.  So I specify txindex.

Note that I mistakenly documented a need to run "bitcoind -rescan" the first time with txindex in effect; that should be "bitcoind -reindex" instead.

In practice, I sometimes see the following error [EDIT: fixed] with bitcoind 0.8.1 when a blockfile other than the current one contains trailing NULs (zero bytes).
Code:
MerkleRootMismatch: Block header Merkle root does not match its transactions. block hash=14508459b221041eab257d2baaa7459775ba748246c8403609eb708f0e57e74b
I work around it by trimming the NULs from the file (using /usr/bin/truncate in Linux and the blkfile_offset value from the error trace) and restarting.  I would like to fix this in the code.

In theory, reading the blockfiles while bitcoind runs is unsafe due to race conditions and the use of an undocumented feature.  In practice, I do not recall this ever causing a problem until Bitcoin 0.8, and the problems related to running simultaneously with 0.8.x are fixed.  Abe checks each block's Merkle root so as not to load a corrupt or incomplete block, so at worst, an error should involve resetting the datadir row to the start of the current file (or pass --rescan to Abe for a slower but easier fix).

Also, do you know any way to check that -txindex=1 is indeed active and was used for the last reindex? My ~/.bitcoin/blocks is currently 8.6GiB of which 837MiB is index. ~/.bitcoin/chainstate is 208MiB.

I am sure there is some file that will be a certain size, but I don't know it.  My blocks, index, and chainstate directories are similar to yours in size, and I have a confirmed working txindex.  The way I know is by finding a transaction all of whose outputs are spent, and fetching it with "bitcoind getrawtransaction".  If it says "transaction not in index" bad, if it gives you a hex string, good.

Thanks for your work, I just donated ฿0.2.

Thank you for your support.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
salfter
Hero Member
*****
Offline Offline

Activity: 550


My PGP Key: 92C7689C


View Profile WWW
April 18, 2013, 07:45:44 PM
 #410

I have a P2Pool monitor webapp:

http://alfter.us/p2pool.html

I'd like to have it display the balance of the default payout address for the P2Pool instance.  In my case, I'm mining Litecoin, so I had considered hitting up the Litecoin block explorer for this information...subtract the result of the getsentbyaddress query from the result of the getreceivedbyaddress query and you're done.  Actually getting that information into your browser with just JavaScript is troublesome, though.  Obtaining data from another domain is usually blocked by the browser.  The existing queries against P2Pool are handled by YQL, but the Litecoin block explorer has a robots.txt which blocks YQL. JQuery would facilitate direct queries...but only if the data source supports JSONP.  Most of the Abe API calls return plaintext, not JSON or JSONP.  I'm getting my own copy of Abe up and running against the Litecoin blockchain so that I can at least aim YQL queries at it (it'll take nearly any kind of data and format it as either XML or JSON), but are there any plans to add/improve JSON support in the Abe API?  Is there some other approach to what I'm trying to do that would work?  The more functionality I can keep in the client, the better.  (I could've knocked together some PHP to do what I want, but that seems a bit like cheating.)

My Bitgem Pool - PPLNS, Stratum | Gridseed Miners - $25 off your first order | BTG Explorer
Tipjars: BTC 1TipsGocnz2N5qgAm9f7JLrsMqkb3oXe2 LTC LTipsVC7XaFy9M6Zaf1aGGe8w8xVUeWFvR BTG gTipsVB9qmyYHuqMMKTuCYMHpfkUFBXKrZ | My Bitcoin Note Generator | Pyramining: 1 2 3 4 5
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 18, 2013, 08:26:06 PM
 #411

@salfter

Thanks for the education.  It would be pretty easy to add JSON and JSONP support for API functions.  I'll add it to the to-do list and create an issue on GitHub.  Patches are welcome.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 19, 2013, 06:24:55 PM
 #412

A bug has caused http://abe.john-edwin-tobey.org to lose track of the bitcoins outstanding as of Block 230560.  Details are in TODO.txt.

This affects the experimental no-statistics branch and possibly the master branch.  It appears to depend on multiple concurrent loading processes.  For best results, use the master branch and avoid concurrent loading until this is fixed.  To avoid concurrent loading, either use a single process (not FastCGI) or have all but one process use an empty datadir list ("datadir []" in abe.conf).  When I set up FastCGI, I use datadir=[] and create a separate loader job that runs Abe continuously in a loop with the live datadir(s) and --no-serve.

Sorry for any inconvenience!

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
April 23, 2013, 06:23:37 PM
 #413

Is most of the slowness with importing the blockchain in the SQL inserts?

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 24, 2013, 03:49:01 PM
 #414

Is most of the slowness with importing the blockchain in the SQL inserts?
I think so, but I don't have profile data.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
April 24, 2013, 05:32:50 PM
 #415

I'll have to have a closer look. I'm thinking I'd like something of this but more for event-based notification rather than aggregate/historical data.

There may be some way to improve the speed of the inserts too. I'll take a look at that.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 24, 2013, 07:26:56 PM
 #416

I'll have to have a closer look. I'm thinking I'd like something of this but more for event-based notification rather than aggregate/historical data.

There may be some way to improve the speed of the inserts too. I'll take a look at that.
I'll appreciate it.  You may want to peruse the GitHub issue.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
April 24, 2013, 07:59:04 PM
 #417

I'll have to have a closer look. I'm thinking I'd like something of this but more for event-based notification rather than aggregate/historical data.

There may be some way to improve the speed of the inserts too. I'll take a look at that.
I'll appreciate it.  You may want to peruse the GitHub issue.


Absolute first thing in my book would be to give up on sqlite. It's really not meant for this kind of abuse. Other than that though...


Edit: That is to say, you could continue to offer sqlite as an option but I wouldn't waste any time trying to optimize it.

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 24, 2013, 08:46:46 PM
 #418

Absolute first thing in my book would be to give up on sqlite. It's really not meant for this kind of abuse. Other than that though...

Edit: That is to say, you could continue to offer sqlite as an option but I wouldn't waste any time trying to optimize it.

Well, if you mean the BTC chain, or any kind of server use, I agree.  It would be nice to add some advice about back-end selection in the docs.

On the other hand, given Abe's SQL abstraction layer, any optimizations tend to benefit all databases, unless we start parallel loading, which SQLite won't support.

I'd like to support a lot more configurable features.  Currently, there is keep-scriptsig and use-firstbits, both of which control the existence of certain columns in the database.  The no-statistics branch also omits some columns and a table.  We could have a bare-bones option that omits transaction inputs and outputs from the database, fetching them as needed from bitcoind.  At the other extreme, we could support denormalization (copying columns into dependent tables) for faster queries at the cost of insertion time and space.  This motivated me to clean up the SQL abstraction layer (not yet merged from no-statistics to master).  I want to abstract over more SQL dialect differences to facilitate configuration changes that modify table structure.

Keeping the front end usable under all configurations would be a challenge, but not to worry: we can declare some configurations unsupported by the HTML front end.  One might even imagine a zero-init mode that starts loading and using new blockchain data before it fills in the old.

Of course, optimization could be easier if we commit to MySQL or PostgreSQL, but I think there is enough low-hanging fruit to postpone that idea for a while.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Richy_T
Legendary
*
Offline Offline

Activity: 1246


1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k


View Profile
April 24, 2013, 08:50:52 PM
 #419

I don't know much about the blockchain format but I wonder if there would be much mileage in merely indexing into the blockchain data and only storing the indexes in the database...

1RichyTrEwPYjZSeAYxeiFBNnKC9UjC5k
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
April 25, 2013, 02:44:00 AM
 #420

I don't know much about the blockchain format but I wonder if there would be much mileage in merely indexing into the blockchain data and only storing the indexes in the database...
Yes, Abe could store blockfile offsets and read from the files as bitcoind does, or store hashes and fetch raw data by RPC to bitcoind.  A table with just address and transaction identifier (hash or blockfile coordinates), indexed by address, would be useful all alone.  So would a table mapping transaction to block number.  Ideally, the whole database would be built of minimal pieces such as these, and the loader would consult metadata to figure out what it has to do.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 [21] 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!