Bitcoin Forum
December 10, 2016, 03:19:18 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 175854 times)
K1773R
Legendary
*
Offline Offline

Activity: 1526


/dev/null


View Profile
November 28, 2012, 06:48:54 PM
 #361

Code:
Traceback (most recent call last):
  File "/usr/lib/python2.6/wsgiref/handlers.py", line 93, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python2.6/dist-packages/Abe/abe.py", line 205, in __call__
    handler(page)
  File "/usr/local/lib/python2.6/dist-packages/Abe/abe.py", line 459, in handle_chain
    ss = int(ss)
TypeError: int() argument must be a string or a number, not 'NoneType'

Thanks for the report.  I've replied on GitHub.  Unfortunately, I don't think it will be easy to debug, since it appears to depend on the order the blocks were loaded, and that in turn depends on network timings.  I will post when I have time to address this.  Meanwhile, I suggest using an older version of Abe, either v0.6 or the Sept. 15 commit (fc10484).

lets continue the chat @ github Wink

[GPG Public Key]  [Devcoin Builds]  [BBQCoin Builds]  [Multichain Blockexplorer]  [Multichain Blockexplorer - PoS Coins]  [Ufasoft Miner Linux Builds]
BTC/DVC/TRC/FRC: 1K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM AK1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: NK1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: LKi773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: EK1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: bK1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
1481339958
Hero Member
*
Offline Offline

Posts: 1481339958

View Profile Personal Message (Offline)

Ignore
1481339958
Reply with quote  #2

1481339958
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
molecular
Donator
Legendary
*
Offline Offline

Activity: 2142



View Profile
November 28, 2012, 11:50:20 PM
 #362

I just produced a dataset called "difficulty and hashrate by day" using bitcoin-abe (thanks so much, John Tobey, I love bitcoin-abe): https://bitcointalk.org/index.php?topic=123238.msg1365248#msg1365248

When designing the query I wanted to see the resulting sql string. How can I set the loglevel of store.sqllogger without modifying the python code in Datastore.py?



PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
November 29, 2012, 06:15:05 PM
 #363

When designing the query I wanted to see the resulting sql string. How can I set the loglevel of store.sqllogger without modifying the python code in Datastore.py?

--log-sql shows all queries' SQL and parameters.  This is currently just a boolean switch.  To set the level to something other than DEBUG, you would use log-sql and a custom logging config with (for example) "loggers":{"Abe.DataStore.sql": {"level": "INFO"}}.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
December 04, 2012, 09:01:31 PM
 #364

Code:
Traceback (most recent call last):
  File "/usr/lib/python2.6/wsgiref/handlers.py", line 93, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python2.6/dist-packages/Abe/abe.py", line 205, in __call__
    handler(page)
  File "/usr/local/lib/python2.6/dist-packages/Abe/abe.py", line 459, in handle_chain
    ss = int(ss)
TypeError: int() argument must be a string or a number, not 'NoneType'

Thanks for the report.  I've replied on GitHub.  Unfortunately, I don't think it will be easy to debug, since it appears to depend on the order the blocks were loaded, and that in turn depends on network timings.  I will post when I have time to address this.  Meanwhile, I suggest using an older version of Abe, either v0.6 or the Sept. 15 commit (fc10484).


I think this is fixed.  It turned out to depend not on block order, but on duplicate coinbase transactions.  If this works out, I'll call it 0.7.2.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
December 07, 2012, 02:39:25 AM
 #365

A maintenance release to address "TypeError: int() argument must be a string or a number, not 'NoneType'": 0.7.2

* Fixed bug affecting chains containing duplicate coinbase transactions.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
crazy_rabbit
Legendary
*
Offline Offline

Activity: 1162


RUM AND CARROTS: A PIRATE LIFE FOR ME


View Profile
December 20, 2012, 02:25:48 PM
 #366

This is a decidedly newbie question- but how can I safely turn off ABE? I have installed it to install Electrum, but once it's running, how do I turn it off? So as to not have any conflicts?

thanks!

more or less retired.
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
December 20, 2012, 05:15:23 PM
 #367

This is a decidedly newbie question- but how can I safely turn off ABE? I have installed it to install Electrum, but once it's running, how do I turn it off? So as to not have any conflicts?

thanks!
Ctrl-C it from the command line or use "kill" in Linux.  Don't worry about the stack trace that it prints.  One of Abe's strongest (IMHO) features is its ability to recover automatically from interruption.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
BitAurum
Sr. Member
****
Offline Offline

Activity: 319


View Profile
January 10, 2013, 09:59:08 PM
 #368

Hi folks, hello developers (i.e. John Tobey),
I am playing with the thought of implementing this on my server, since I am already running a full 24/7 BTC-node, LAMP and still have some HDD-space left. Also, since the main fallibility of P2P-Networks is the fact that the truth is determined in a democratic process, you can't have enough block explorers with different blockchain-sources out there.

Nevertheless, I have a problem with understanding the license this was published under: Let's say I want to change the look of the site by rewriting a bit of the CSS-sheet and adding a different logo, I basically have to publish the new sources, even for small derivations, right? But I am allowed to do so, even commercially (aka. linking the logo to my site, which is adversing and therefore commercial...)

Second question: Is anyone running this on MySQL? How big is the database when finished, and is there a somewhat linear relationship to the size of the blockchain (one might assume so...)? Had it loading for a few hours now and got to block 143000, it's around 2.9 GB so far.

Thanks for the code and all the hard work!

edit:
Just some arbitrary data points regarding database size (MySQL) and block number...
Block 143000     2.9 GB
Block 152000     3.7 GB
Block 170000     5.3 GB
Block 184200     8.2 GB (this is where the heavy lifting starts...)
Block 200000   14.9 GB
Block 216276   22.9 GB (full chain January 12th 2013)

Got bored and built LAMP on my gentoo home box. Had the database done within about 18 hours (some delay due to full hdd, note to future self: default Gentoo my.cnf has binary logging enabled) Speed benefits from giving innodb some 6GB ram. Should have done some multithreading, but most of the load still comes from read/write-operations, even with excessive buffering. So one 3.3 GHz Phenom II core might have gotten slightly warmer. Good news is: I will maybe be able to host an instance on my VPS for a few weeks before the size of the blockchain and the database eat my 40 GB HDD. Yay, I guess Cheesy
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
January 11, 2013, 03:14:00 PM
 #369

Hi folks, hello developers (i.e. John Tobey),

Hi!

I am playing with the thought of implementing this on my server, since I am already running a full 24/7 BTC-node, LAMP and still have some HDD-space left. Also, since the main fallibility of P2P-Networks is the fact that the truth is determined in a democratic process, you can't have enough block explorers with different blockchain-sources out there.

Nevertheless, I have a problem with understanding the license this was published under: Let's say I want to change the look of the site by rewriting a bit of the CSS-sheet and adding a different logo, I basically have to publish the new sources, even for small derivations, right? But I am allowed to do so, even commercially (aka. linking the logo to my site, which is adversing and therefore commercial...)

Right.  One easy way to respect the license is to get an account on GitHub (or another public Git service like Gitorious) and maintain your version there.  They give you plenty of handy features.  If it is just a logo and CSS changes (branding as opposed to functionality) I don't care.  If this becomes a sticking point, let me know and I will try to fix it by adding exceptions to the license.

Second question: Is anyone running this on MySQL? How big is the database when finished, and is there a somewhat linear relationship to the size of the blockchain (one might assume so...)? Had it loading for a few hours now and got to block 143000, it's around 2.9 GB so far.

I think it is roughly 4-5 times the sum of the block file sizes.  You can check which block file it is on and how far along it is by reading the datadir table as it loads.  blkfile_number=1 means it has not finished reading blk0001.dat.  blkfile_offset is how far into the current file it has processed.  If blkfile_number is greater than 1, add the sizes of the lower blkNNNN.dat files on disk.

Code:
mysql> select * from datadir;
+------------+------------------------+----------------+----------------+----------+
| datadir_id | dirname                | blkfile_number | blkfile_offset | chain_id |
+------------+------------------------+----------------+----------------+----------+
|          1 | /home/bitcoin/.bitcoin |              1 |      699231568 |     NULL |
+------------+------------------------+----------------+----------------+----------+

Thanks for the code and all the hard work!

You're welcome and good luck!

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
molecular
Donator
Legendary
*
Offline Offline

Activity: 2142



View Profile
January 13, 2013, 10:02:38 AM
 #370

some charts I produced with bitcoin-abe.

https://bitcointalk.org/index.php?topic=135982.msg1450346#msg1450346

Thanks so much again, John Tobey, for such an awesome product.

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
January 24, 2013, 09:05:48 AM
 #371

Is there an easy way to get all unspent transactions out of a list of bitcoinaddresses?

And is there maybe a server on wich I can test stuff on?

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
January 26, 2013, 07:28:17 PM
 #372

Nevermind, I am running my own server atm.

Code:
Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "abe.py", line 205, in __call__
    handler(page)
  File "abe.py", line 909, in handle_rawtx
    abe.do_raw(page, abe.do_rawtx(page))
  File "abe.py", line 921, in do_rawtx
    return json.dumps(tx, sort_keys=True, indent=2)
  File "/usr/lib/python2.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 203, in encode
    chunks = list(chunks)
  File "/usr/lib/python2.7/json/encoder.py", line 428, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 326, in _iterencode_list
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib/python2.7/json/encoder.py", line 436, in _iterencode
    o = _default(o)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('4294967295') is not JSON serializable
<ip> - - [26/Jan/2013 20:15:37] "GET /rawtx/38b46ec5330c445d0e6451d305a08e0fd91a660510ba5aa3ed3c9b1f43356905 HTTP/1.1" 500 59


And I am willing to put a bounty of 3btc for this:


Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
January 28, 2013, 11:39:47 PM
 #373

Code:
TypeError: Decimal('4294967295') is not JSON serializable
<ip> - - [26/Jan/2013 20:15:37] "GET /rawtx/38b46ec5330c445d0e6451d305a08e0fd91a660510ba5aa3ed3c9b1f43356905 HTTP/1.1" 500 59

Fixed, thanks.

And I am willing to put a bounty of 3btc for this:


This is done, but with caveats: I do not have a full database to test with, it may require optimization, the output fields are slightly different, and it won't work with your example address because Abe does not yet parse script-to-hash transactions.

/unspent/ADDRESS1|ADDRESS2|...
http://john-edwin-tobey.org:2752/unspent/1CBXL5tBSqJZyyCTWVkrLiQsKm4KULBfPA|1CfoGajDKynwB1JpaWq2tML4DDsP8VnuPy

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
January 29, 2013, 10:24:11 AM
 #374

Code:
TypeError: Decimal('4294967295') is not JSON serializable
<ip> - - [26/Jan/2013 20:15:37] "GET /rawtx/38b46ec5330c445d0e6451d305a08e0fd91a660510ba5aa3ed3c9b1f43356905 HTTP/1.1" 500 59

Fixed, thanks.

And I am willing to put a bounty of 3btc for this:


This is done, but with caveats: I do not have a full database to test with, it may require optimization, the output fields are slightly different, and it won't work with your example address because Abe does not yet parse script-to-hash transactions.

/unspent/ADDRESS1|ADDRESS2|...
http://john-edwin-tobey.org:2752/unspent/1CBXL5tBSqJZyyCTWVkrLiQsKm4KULBfPA|1CfoGajDKynwB1JpaWq2tML4DDsP8VnuPy

Thanks! Smiley

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
February 07, 2013, 12:08:58 PM
 #375

Because I was impatient, I already had my own unspent transactions working by interpreting the pages already provided by bitcoin abe.

I would get the all the adresses (/bitcoin-abe/address/<address>). If it had any balance I would make a list of the transactions, sorting input and output transactions. I would then fetch all the input transactions to see if the transaction was already used in an output transaction and would then subtract that transaction from the output transaction list. The remaining transactions are the unspent transactions.

This way is a lot faster then your unspent api call, maybe you could one day look in to it.

(and I hope you still use your donation address, because that is where I sent the coins to)

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
February 07, 2013, 08:09:55 PM
 #376

@Jouke,

Thanks for the generous tip and the report of slow performance.  It should be simple to change /unspent to match the speed of repeated /address calls.  Please feel free to remind me if I haven't got to it in a week. Smiley

Yes, 1PWC7PNHL1SgvZaN7xEtygenKjWobWsCuf is still under my control.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
February 13, 2013, 04:49:20 AM
 #377

I've reimplemented /unspent so it should outperform /address.  A couple of things to note.  The denial-of-service limit on history rows applies to the total across all addresses given to /unspent, so in some cases, /unspent will return "too many records" when iterated /address calls would work.  This limit applies to both spent and unspent coins, unlike in the previous implementation, so you may see "too many records" where my first /unspent worked.  In abe.conf, address-history-rows-max controls this limit.  For now, I've left the first implementation available by adding "?impl=1" to the URL.  It should yield the same results apart from "too many records" errors.

Let me know if this helps.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
February 13, 2013, 02:27:29 PM
 #378

I've reimplemented /unspent so it should outperform /address.  A couple of things to note.  The denial-of-service limit on history rows applies to the total across all addresses given to /unspent, so in some cases, /unspent will return "too many records" when iterated /address calls would work.  This limit applies to both spent and unspent coins, unlike in the previous implementation, so you may see "too many records" where my first /unspent worked.  In abe.conf, address-history-rows-max controls this limit.  For now, I've left the first implementation available by adding "?impl=1" to the URL.  It should yield the same results apart from "too many records" errors.

Let me know if this helps.


Excellent! I'll look into it this week.

Question regarding this: https://bitcointalk.org/index.php?topic=22785.msg741984#msg741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?


And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
John Tobey
Hero Member
*****
Offline Offline

Activity: 481



View Profile WWW
February 13, 2013, 04:18:47 PM
 #379

Question regarding this: https://bitcointalk.org/index.php?topic=22785.msg741984#msg741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?

A loop that runs it with --no-serve would keep it up to date.  It brings it up to date and then exits.

And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley

It gives a statistical estimate of amount in circulation.  I am curious which block number you got the Abe value from?  You can add the number to the URL like /q/totalbc/220962 to be sure.

The number is normally 50*210000 + 25*(height-209999) but is lower due to the following rare events:
  • coinbase transactions outputting less than they are entitled to,
  • coins sent to known dead addresses (Abe recognizes the hash consisting of all 0 bits),
  • duplicate coinbase transactions (I don't think Abe accounts for them, but the others may)

and there may be bugs.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
February 13, 2013, 06:14:17 PM
 #380

Question regarding this: https://bitcointalk.org/index.php?topic=22785.msg741984#msg741984
An Abe instance that is started with --no-serve automatically keeps the blockchain up to date?

A loop that runs it with --no-serve would keep it up to date.  It brings it up to date and then exits.

And what does /q/totalbc show? Bitcoins created, or bitcoins included in coinbase transaction? Because
Local Abe: 10774286.89978955
Blockexplorer: 10736300.00000000
Blockchain.info: 10761275.00000000

Not that it really matters, I don't use it, but just wondering Smiley

It gives a statistical estimate of amount in circulation.  I am curious which block number you got the Abe value from?  You can add the number to the URL like /q/totalbc/220962 to be sure.

The number is normally 50*210000 + 25*(height-209999) but is lower due to the following rare events:
  • coinbase transactions outputting less than they are entitled to,
  • coins sent to known dead addresses (Abe recognizes the hash consisting of all 0 bits),
  • duplicate coinbase transactions (I don't think Abe accounts for them, but the others may)

and there may be bugs.


I was expecting 50*210000 + 25*(height-209999) as well, that was why I was wondering if it added fees or something.

chain/Bitcoin/q/totalbc/100000 : 5000049.98
chain/Bitcoin/q/totalbc/200000 : 10000037.06486171
chain/Bitcoin/q/totalbc/220998 : 10774961.89978955

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 [19] 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!