Bitcoin Forum
March 28, 2024, 01:35:53 PM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 »
  Print  
Author Topic: [ANNOUNCE] Abe 0.7: Open Source Block Explorer Knockoff  (Read 220721 times)
John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 01, 2013, 05:48:44 PM
 #481

I'll probably just pick one of those chains. I didn't suspect trying to get 30 chains loaded in Abe would require you to fix a bunch of stuff Smiley

Your reports are helping me improve the code.  Thank you!!

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
1711632953
Hero Member
*
Offline Offline

Posts: 1711632953

View Profile Personal Message (Offline)

Ignore
1711632953
Reply with quote  #2

1711632953
Report to moderator
1711632953
Hero Member
*
Offline Offline

Posts: 1711632953

View Profile Personal Message (Offline)

Ignore
1711632953
Reply with quote  #2

1711632953
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711632953
Hero Member
*
Offline Offline

Posts: 1711632953

View Profile Personal Message (Offline)

Ignore
1711632953
Reply with quote  #2

1711632953
Report to moderator
Jouke
Sr. Member
****
Offline Offline

Activity: 426
Merit: 250



View Profile WWW
July 02, 2013, 07:50:46 AM
Last edit: July 02, 2013, 01:23:07 PM by Jouke
 #482

Run this SQL and see if it is fixed:
Code:
UPDATE datadir SET chain_id = 1, loader = 'rpc' WHERE datadir_id = 1;

In the datadir table I have the following columns:
datadir_id
dirname
blkfile_number
blkfile_number
blkfile_offset
chain_id
datadir_loader

Whoops, I guess I meant:
UPDATE datadir SET chain_id = 1, datadir_loader = 'rpc' WHERE datadir_id = 1;

How does Abe keep track of blockfiles and blockfile offsets when it is updating from RPC?

Putting chain_id and datadir_loader in the table was bad design.  I think they get their values from the "chain" and "loader" (or "default-loader") config elements when the datadir is first used, and thereafter, the config values are ignored.  I plan to make it use the config values only, but meanwhile, I think you can copy the correct values into the table row, and it will work.


Thanks. I see it is adding mempool transactions now. But I did understand correctly that it also updates blocks right? It does.

Ah, I can look up mempool transactions, but it does not add the addresses in that transaction to the database? I was hoping I could do a /unspent/ on mempool transactions as well.

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 02, 2013, 03:14:22 PM
 #483

Ah, I can look up mempool transactions, but it does not add the addresses in that transaction to the database? I was hoping I could do a /unspent/ on mempool transactions as well.
This is a little tricky to do.  Abe does not delete transactions that can never get into the main chain because they contain inputs that are spent elsewhere.  /unspent/ excludes these from its output by filtering on block_tx and chain_candidate.in_longest = 1.  This also excludes mempool transactions.  Can you see them on the address page?

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
crendore
Sr. Member
****
Offline Offline

Activity: 363
Merit: 250


View Profile
July 02, 2013, 07:29:12 PM
 #484

Hey John,

Thanks for your awesome contribution to the open source community.  I really appreciate the work you've done!

I have been wondering if there is any way to accomplish this blockexplorer.com feature using abe:
In blockexplorer.com i can do this:
http://blockexplorer.com/q/getreceivedbyaddress/ADDRESS/N
Where N specifies the number of confirmations before a deposit is considered valid at that address. 

With Abe, i don't seem to be able to accomplish this functionality, eg:
http://bbq.cryptocoinexplorer.com/chain/BBQCoin/q/getreceivedbyaddress/ADDRESS/N
does the same thing as
http://bbq.cryptocoinexplorer.com/chain/BBQCoin/q/getreceivedbyaddress/ADDRESS

I checked in abe.py and it doesn't look like that extra parameter is handled at all.

I know a bit of python and wouldn't be too afraid to dive in and attempt to solve this myself, but before i do that i would like to hear any pointers / advice you would suggest on what the best approach would be to accomplish this (without making too much of a muck of your code).

Cheers,
Crendore

Jouke
Sr. Member
****
Offline Offline

Activity: 426
Merit: 250



View Profile WWW
July 02, 2013, 09:20:18 PM
 #485

Ah, I can look up mempool transactions, but it does not add the addresses in that transaction to the database? I was hoping I could do a /unspent/ on mempool transactions as well.
This is a little tricky to do.  Abe does not delete transactions that can never get into the main chain because they contain inputs that are spent elsewhere.  /unspent/ excludes these from its output by filtering on block_tx and chain_candidate.in_longest = 1.  This also excludes mempool transactions.  Can you see them on the address page?

I'll find some time to play with the code.

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
robanswe
Newbie
*
Offline Offline

Activity: 24
Merit: 0


View Profile
July 02, 2013, 09:27:56 PM
 #486

Hello, I really appreciate the work you are putting into this. I currently only got one "problem" with it. It takes forever to do the "initial data load". I'm trying to load the bitcoin blockchain into a mysql db and I'm currently at block 176100 and it takes around 5seconds/block is this normal or I'm I doing something wrong?
crendore
Sr. Member
****
Offline Offline

Activity: 363
Merit: 250


View Profile
July 02, 2013, 10:38:23 PM
 #487

Hello, I really appreciate the work you are putting into this. I currently only got one "problem" with it. It takes forever to do the "initial data load". I'm trying to load the bitcoin blockchain into a mysql db and I'm currently at block 176100 and it takes around 5seconds/block is this normal or I'm I doing something wrong?

It takes a while, depending on your system, but that sounds about right. i think it took me almost 2 days to get it running the first time.  What might make sense though, is having the database it is creating hosted seperately, like on mega.co.nz or something, so that people could just download the database directly instead of creating it everytime.  Obviously any database dump would be a little bit outdated, but then you could run abe and update from that point.

K1773R
Legendary
*
Offline Offline

Activity: 1792
Merit: 1008


/dev/null


View Profile
July 02, 2013, 11:04:26 PM
 #488

Hello, I really appreciate the work you are putting into this. I currently only got one "problem" with it. It takes forever to do the "initial data load". I'm trying to load the bitcoin blockchain into a mysql db and I'm currently at block 176100 and it takes around 5seconds/block is this normal or I'm I doing something wrong?
db @ tmpfs and its extreme fast Wink

[GPG Public Key]
BTC/DVC/TRC/FRC: 1K1773RbXRZVRQSSXe9N6N2MUFERvrdu6y ANC/XPM AK1773RTmRKtvbKBCrUu95UQg5iegrqyeA NMC: NK1773Rzv8b4ugmCgX789PbjewA9fL9Dy1 LTC: LKi773RBuPepQH8E6Zb1ponoCvgbU7hHmd EMC: EK1773RxUes1HX1YAGMZ1xVYBBRUCqfDoF BQC: bK1773R1APJz4yTgRkmdKQhjhiMyQpJgfN
John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 03, 2013, 01:57:32 AM
 #489

Thanks for your awesome contribution to the open source community.  I really appreciate the work you've done!

Happy to hear it.

I have been wondering if there is any way to accomplish this blockexplorer.com feature using abe:
In blockexplorer.com i can do this:
http://blockexplorer.com/q/getreceivedbyaddress/ADDRESS/N
Where N specifies the number of confirmations before a deposit is considered valid at that address. 

With Abe, i don't seem to be able to accomplish this functionality, eg:
http://bbq.cryptocoinexplorer.com/chain/BBQCoin/q/getreceivedbyaddress/ADDRESS/N
does the same thing as
http://bbq.cryptocoinexplorer.com/chain/BBQCoin/q/getreceivedbyaddress/ADDRESS

I checked in abe.py and it doesn't look like that extra parameter is handled at all.

I know a bit of python and wouldn't be too afraid to dive in and attempt to solve this myself, but before i do that i would like to hear any pointers / advice you would suggest on what the best approach would be to accomplish this (without making too much of a muck of your code).

Actually, this ought to be very simple.  I'd do it right now, but I am not at my secure workstation.  Where q_getreceivedbyaddress calls abe.store.get_received, it could pass the optional block_height parameter to ignore blocks after a certain number.  That number would be something like:
Code:
abe.store.get_block_number(chain['id']) + 1 - N

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
bbxx
Hero Member
*****
Offline Offline

Activity: 728
Merit: 500


cryptoshark


View Profile WWW
July 05, 2013, 08:53:54 AM
 #490

i have some problems with bitcoin blockchain

database size is huge!
i am at block 228k and already it took 31.6 GB!
142,140,449 records inserted so far.

my cpu i5 3570, 4gb ram, 50gb assigned for vdi.
ubuntu server, mysql, 1gb left on ssd Sad

i need only one information, balance on given address.
can i clear some of tables to reduce mysql database size ?

should i use postgresql or maybe there is a fork with reduced db size needed ?
what i can do to reduce resource usage ?
Jouke
Sr. Member
****
Offline Offline

Activity: 426
Merit: 250



View Profile WWW
July 05, 2013, 09:05:04 AM
 #491

i have some problems with bitcoin blockchain

database size is huge!
i am at block 228k and already it took 31.6 GB!
142,140,449 records inserted so far.

my cpu i5 3570, 4gb ram, 50gb assigned for vdi.
ubuntu server, mysql, 1gb left on ssd Sad

i need only one information, balance on given address.
can i clear some of tables to reduce mysql database size ?

should i use postgresql or maybe there is a fork with reduced db size needed ?
what i can do to reduce resource usage ?

https://bitcointalk.org/index.php?topic=22785.msg1554489;topicseen#msg1554489

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
mapleshadow
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
July 08, 2013, 05:17:00 PM
 #492

my conf is wdc

wdc.conf:
dbtype sqlite3
connect-args abe.sqlite
datadir = [{
        "dirname": "/home/mapleshadow/.worldcoin",
        "chain":   "Worldcoin",
        "code3":   "WDC",
        "address_version": "\u0049" }]
int-type str
upgrade
port 2750
host 12.16.2.24

DataStore.py:
 {"chain":"Worldcoin",
     "code3":"WDC", "address_version":"\x49", "magic":""},

ERROR-1:
failed to load /home/maple/.worldcoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/maple/.worldcoin/bitcoin.conf'
catch_up_rpc: abort
Opened /home/mapleshadow/.worldcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://12.16.2.24:2750

WHY?
John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 08, 2013, 06:02:30 PM
 #493

my conf is wdc

wdc.conf:
dbtype sqlite3
connect-args abe.sqlite
datadir = [{
        "dirname": "/home/mapleshadow/.worldcoin",
        "chain":   "Worldcoin",
        "code3":   "WDC",
        "address_version": "\u0049" }]
int-type str
upgrade
port 2750
host 12.16.2.24

DataStore.py:
 {"chain":"Worldcoin",
     "code3":"WDC", "address_version":"\x49", "magic":""},

ERROR-1:
failed to load /home/maple/.worldcoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/maple/.worldcoin/bitcoin.conf'
catch_up_rpc: abort
Opened /home/mapleshadow/.worldcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://12.16.2.24:2750

WHY?


Abe tries to load data over RPC, and when that fails, it reads block files.  (See default-loader in abe.conf to choose one or the other.)

Abe does not know about Worldcoin, so it looks for RPC credentials in bitcoin.conf.  There is no bitcoin.conf, so it logs an error and tries to load blocks from files, starting with blocks/blk00000.dat.  It did not find any blocks in that file, so perhaps the file is empty or the block format is not the same as Bitcoin's.

I am surprised that the configured dirname is in /home/mapleshadow but Abe looked for bitcoin.conf in /home/maple.

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
mapleshadow
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
July 09, 2013, 02:04:50 AM
 #494

Sorry, I was wrong
Paths are maple
mapleshadow
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
July 09, 2013, 02:09:42 AM
 #495


That I can build wdc of abe services?

my conf is wdc

wdc.conf:
dbtype sqlite3
connect-args abe.sqlite
datadir = [{
        "dirname": "/home/maple/.worldcoin",
        "chain":   "Worldcoin",
        "code3":   "WDC",
        "address_version": "\u0049" }]
int-type str
upgrade
port 2750
host 12.16.2.24

DataStore.py:
 {"chain":"Worldcoin",
     "code3":"WDC", "address_version":"\x49", "magic":""},

ERROR-1:
failed to load /home/maple/.worldcoin/bitcoin.conf: [Errno 2] No such file or directory: u'/home/maple/.worldcoin/bitcoin.conf'
catch_up_rpc: abort
Opened /home/maple/.worldcoin/blocks/blk00000.dat
Abe initialized.
Listening on http://12.16.2.24:2750

WHY?


Abe tries to load data over RPC, and when that fails, it reads block files.  (See default-loader in abe.conf to choose one or the other.)

Abe does not know about Worldcoin, so it looks for RPC credentials in bitcoin.conf.  There is no bitcoin.conf, so it logs an error and tries to load blocks from files, starting with blocks/blk00000.dat.  It did not find any blocks in that file, so perhaps the file is empty or the block format is not the same as Bitcoin's.

I am surprised that the configured dirname is in /home/mapleshadow but Abe looked for bitcoin.conf in /home/maple.

John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 09, 2013, 04:10:23 PM
 #496


That I can build wdc of abe services?

I don't know.  Can you get Abe working with BTC or NMC or LTC?  Those are known to work.

I need a little more information to understand your problem.  What is your python command line?  How big is /home/mapleshadow/.worldcoin/blocks/blk00000.dat ?  What exact version of Abe do you have?  (Post the output of "git rev-parse HEAD" and "git --no-pager diff")

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
mapleshadow
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
July 09, 2013, 05:26:06 PM
 #497

Yes
Operating in BTC state, there is no problem
I do not know LTC, Not tested
home/mapleshadow/.worldcoin/blocks/blk00000.dat have 500M Size
ABE version is GITHUB the MASTER
--------
My configuration as described above, for the WDC, then it is an error
mapleshadow
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
July 09, 2013, 05:29:19 PM
 #498

Also
blk00000.dat file, for WDC, it is not present, the default is blk0001.dat and blkindex.dat files.
Currently reading blk00000.dat file that I copied blk0001.dat file, rename came.
madmax_ger
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250



View Profile
July 13, 2013, 04:21:56 PM
 #499

hi guys,

I have some trouble with installing ABE on my macbook.

everything went fine so far, I..
* installed mysql 32bit, it's running, path configured
* installed python 2.7, it's running
* installed python-mysql in 32bit arch (i386) mode, seems to be running
* installed ABE, no problem so far

then when I want to run ABE...

Code:
$ ARCHFLAGS="-arch i386" python -m Abe.abe --config /usr/local/mysql/abe-my.cnf --commit-bytes 100000 --no-serve

this comes up:
Code:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 2026, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 2020, in main
    store = make_store(args)
  File "/Library/Python/2.7/site-packages/Abe/abe.py", line 115, in make_store
    store = DataStore.new(args)
  File "/Library/Python/2.7/site-packages/Abe/DataStore.py", line 2843, in new
    return DataStore(args)
  File "/Library/Python/2.7/site-packages/Abe/DataStore.py", line 139, in __init__
    store.module = __import__(args.dbtype)
  File "build/bdist.macosx-10.7-intel/egg/MySQLdb/__init__.py", line 19, in <module>
  File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 7, in <module>
  File "build/bdist.macosx-10.7-intel/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found.  Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture

I ran "file <filename>" for mysql (bin file) and the given .so file above, it's both i386.
Can anyone help me out?

http://btcinvest.net/bitcoin-mining-profit-calculator.php - check it out!next difficulty + time leftcustomizable monthly (diff + USD/BTC) increasements device lead timeupdate: auto-compares device costs to BTC-buy profit ♥ 1btciBCKb59TbzNj5QzC2EXWDARxtJL1f
John Tobey (OP)
Hero Member
*****
Offline Offline

Activity: 481
Merit: 529



View Profile WWW
July 14, 2013, 12:08:18 PM
 #500

Code:
ImportError: dlopen(/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so, 2): no suitable image found.  Did find:
/Users/thisuser/.python-eggs/MySQL_python-1.2.4-py2.7-macosx-10.7-intel.egg-tmp/_mysql.so: mach-o, but wrong architecture

Sorry, this is an issue with Python modules on Mac, and I have no relevant experience.  Put this in a file and run it with "python FILE.py".  It will have to complete without error before you can use Abe:

Code:
import MySQLdb

Can a change to the best-chain criteria protect against 51% to 90+% attacks without a hard fork?
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!