Bitcoin Forum
December 04, 2016, 06:12:37 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 [3] 4 5 6 7 8 9 10 »  All
  Print  
Author Topic: .  (Read 34926 times)
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 18, 2012, 07:06:53 PM
 #41

I am having trouble compiling.
http://pastebin.com/TrfSecqA

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
1480831957
Hero Member
*
Offline Offline

Posts: 1480831957

View Profile Personal Message (Offline)

Ignore
1480831957
Reply with quote  #2

1480831957
Report to moderator
1480831957
Hero Member
*
Offline Offline

Posts: 1480831957

View Profile Personal Message (Offline)

Ignore
1480831957
Reply with quote  #2

1480831957
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480831957
Hero Member
*
Offline Offline

Posts: 1480831957

View Profile Personal Message (Offline)

Ignore
1480831957
Reply with quote  #2

1480831957
Report to moderator
1480831957
Hero Member
*
Offline Offline

Posts: 1480831957

View Profile Personal Message (Offline)

Ignore
1480831957
Reply with quote  #2

1480831957
Report to moderator
1480831957
Hero Member
*
Offline Offline

Posts: 1480831957

View Profile Personal Message (Offline)

Ignore
1480831957
Reply with quote  #2

1480831957
Report to moderator
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 18, 2012, 07:18:16 PM
 #42

I am having trouble compiling.
http://pastebin.com/TrfSecqA

From the README:

Code:
sudo apt-get install libssl-dev build-essential g++-4.4 libboost-all-dev libsparsehash-dev git-core perl

Did you do that ?


yes, updated better pastebin btw

http://pastebin.com/tQUq28Zk

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 18, 2012, 07:50:38 PM
 #43

almost complete fresh install of ubuntu 64 bit.

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 18, 2012, 09:35:15 PM
 #44

installing 12.04 x64 in VMWare player.

I really want to play with this. I need to brush up on my C++ skills... haven't used it in about 10 years..

runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 18, 2012, 10:13:58 PM
 #45

yay! compiled on my first try!

EDIT:

snap... I need to DL the blockchain?! ungh... (What was I thinking? lol)

copy and paste from host machine...

runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 18, 2012, 10:21:49 PM
 #46

yup, i found that out...

Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 19, 2012, 07:49:59 AM
 #47

xubuntu 12.10 or something like that

Code:
In file included from cb/allBalances.cpp:4:0:
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = Addr*; Hasher = Hash160Hasher; Equal = Hash160Equal]’:
cb/allBalances.cpp:67:37:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, Addr*, Hash160Hasher, Hash160Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, Addr*> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
In file included from cb/taint.cpp:25:0:
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = long double; Hasher = Hash256Hasher; Equal = Hash256Equal]’:
cb/taint.cpp:70:35:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, long double, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, long double> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
In file included from cb/transactions.cpp:5:0:
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = int; Hasher = Hash160Hasher; Equal = Hash160Equal]’:
cb/transactions.cpp:64:37:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, int, Hash160Hasher, Hash160Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, int> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
make: *** [.objs/taint.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [.objs/allBalances.o] Error 1
make: *** [.objs/transactions.o] Error 1
In file included from parser.cpp:2:0:
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = const unsigned char*; Hasher = Hash256Hasher; Equal = Hash256Equal]’:
parser.cpp:449:29:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, const unsigned char*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, const unsigned char*> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = Block*; Hasher = Hash256Hasher; Equal = Hash256Equal]’:
parser.cpp:450:32:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, Block*, Hash256Hasher, Hash256Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, Block*> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
make: *** [.objs/parser.o] Error 1
In file included from cb/closure.cpp:4:0:
./util.h: In instantiation of ‘void GoogMap<Key, Value, Hasher, Equal>::Map::setEmptyKey(const Key&) [with Key = const unsigned char*; Value = long unsigned int; Hasher = Hash160Hasher; Equal = Hash160Equal]’:
cb/closure.cpp:43:38:   required from here
./util.h:122:21: error: ‘set_empty_key’ was not declared in this scope, and no declarations were found by argument-dependent lookup at the point of instantiation [-fpermissive]
./util.h:122:21: note: declarations in dependent base ‘google::dense_hash_map<const unsigned char*, long unsigned int, Hash160Hasher, Hash160Equal, google::libc_allocator_with_realloc<std::pair<const unsigned char* const, long unsigned int> > >’ are not found by unqualified lookup
./util.h:122:21: note: use ‘this->set_empty_key’ instead
make: *** [.objs/closure.o] Error 1

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 19, 2012, 08:10:35 AM
 #48

Ok, it does not compile on g++ 4.7, it does on g++ 4.5

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 19, 2012, 09:02:03 AM
 #49

Ok, it does not compile on g++ 4.7, it does on g++ 4.5

OK.
Let me install 4.7 and see what's what.


Fixed and fix pushed to github



Compiles nicely

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 19, 2012, 09:08:31 AM
 #50

Is it possible to point it to an other dir then ~/.bitcoin ?


Tried it some other way.

I have a specific problem. I had a blk0001.dat already when I first started the bitcoin-client. Instead of using that file it made a blk0002.dat and recently it made a blk0003.dat. The parser won't open the blockchain file: "Invalid argument: failed to open block chain file /home/jouke/.bitcoin/blk0001.dat"

I guessed that 0002 was actually 0001 and tried to symlink 0002 to 0001 and 0003 to 0002 in a separate directory, but your parser still won't open the file.

Any sugestions?


Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
dooglus
Legendary
*
Offline Offline

Activity: 1988



View Profile
July 19, 2012, 10:01:53 AM
 #51

Is it possible to point it to an other dir then ~/.bitcoin ?


Tried it some other way.

I have a specific problem. I had a blk0001.dat already when I first started the bitcoin-client. Instead of using that file it made a blk0002.dat and recently it made a blk0003.dat. The parser won't open the blockchain file: "Invalid argument: failed to open block chain file /home/jouke/.bitcoin/blk0001.dat"

I guessed that 0002 was actually 0001 and tried to symlink 0002 to 0001 and 0003 to 0002 in a separate directory, but your parser still won't open the file.

Any sugestions?



Temporarily set HOME to wherever you want it to read the blockchain from:

Quote
$ HOME=/path/to/folder ./parser closure
[...]
fatal: No such file or directory: failed to open block chain file /path/to/folder/.bitcoin/blk0001.dat

I doubt the parser made any blockchain files.  It opens the files readonly, which won't create any files.

About a week ago the blockchain file grew bigger than 2GB and so a 2nd file was created by bitcoin-qt:

Quote
$ ls -l blk*
-rw------- 1 chris chris 2097307549 2012-07-12 20:07 blk0001.dat
-rw------- 1 chris chris  104659251 2012-07-19 02:52 blk0002.dat
-rw------- 1 chris chris  751022080 2012-07-19 02:52 blkindex.dat

I don't know where the 0003 file came from though!

Jouke
Sr. Member
****
Offline Offline

Activity: 427



View Profile WWW
July 19, 2012, 10:09:06 AM
 #52

Temporarily set HOME to wherever you want it to read the blockchain from:
Already did that. That's where I symlinked the file to.
Quote
I doubt the parser made any blockchain files.  It opens the files readonly, which won't create any files.
I didn't say it did?
Quote


About a week ago the blockchain file grew bigger than 2GB and so a 2nd file was created by bitcoin-qt:

Quote
$ ls -l blk*
-rw------- 1 chris chris 2097307549 2012-07-12 20:07 blk0001.dat
-rw------- 1 chris chris  104659251 2012-07-19 02:52 blk0002.dat
-rw------- 1 chris chris  751022080 2012-07-19 02:52 blkindex.dat

I don't know where the 0003 file came from though!
I Know a 2nd file was created.

Because I already had a blk0001.dat, my client started with blk0002.dat.

Koop en verkoop snel en veilig bitcoins via iDeal op Bitonic.nl
runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 23, 2012, 11:23:19 PM
 #53

How hard would it be to add address balance to the closure method? (and a total balance at the bottom with address count)

I can't figure it out. My C++ is old and unused... Sad - at least I can read the code!

runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 24, 2012, 07:45:03 PM
 #54

Thanks for that! i'll modify the output so its just the address. from there a simple script should do what i need Smiley

runlinux
Hero Member
*****
Offline Offline

Activity: 567



View Profile WWW
July 25, 2012, 02:29:54 AM
 #55

OK, got a question now:

Why does this address: 12BdKh5HHUyGFduGm6y8nY4KS1LxzsduXo

Link to this address: 12cbQLTFMXRnSzktFkuoG3eHoMeFtpTu3S

Closure reports it has no outputs then returns that one.

On a side note, script written and code modified to take a wallet address, and then just output the balance of the final wallet Smiley

Code:
runlinux@ubuntu:~/blockparser$ ./wallet.sh 1MrPonziaM4QT2S7SdPEKQH88BGa4LRHJU

    transactions  = 211
    received      =      207.30710185
    spent         =      207.30680287
    balance       =        0.00029898


dooglus
Legendary
*
Offline Offline

Activity: 1988



View Profile
July 25, 2012, 04:02:51 AM
 #56

Why does this address: 12BdKh5HHUyGFduGm6y8nY4KS1LxzsduXo

Link to this address: 12cbQLTFMXRnSzktFkuoG3eHoMeFtpTu3S

Closure reports it has no outputs then returns that one.

I see the same.

It tells me:
Code:
error: specified key was never used in a TX output

but this transaction has 12BdKh5HHUyGFduGm6y8nY4KS1LxzsduXo as an output, so the message is a little confusing.

And then the address isn't in its own reported cluster - that's got to be a bug.

Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
July 31, 2012, 12:56:05 PM
 #57

What's your definition of "taint"?

If there is 6% taint from pirate in that huge address, does that mean 6% of these BTC were at one point in time owned by a pirate address or that pirate deposited 6% of these coins to that address?

If it is the first definition, isn't there the possibility to have >100% total taint? E.g. pirate sends BTC to MtGox and then MtGox sends these to the huge address - now they are "tainted" by pirate AND "tainted" by MtGox...

What I would still like to do is to enter an address, get the closure of that and then get the closures of every address where money was sent to or received from that cluster (ideally I could enter how many levels deep, with "0" being the current closure setting).

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
August 02, 2012, 12:23:44 PM
 #58

I'm sure you know I've spent a lot of time doing blockchain scanning, and attempting to optimize it in various ways in Armory.  In the end, the fastest I could achieve was about 20-25s on a full blk0001.dat (2GB) on Linux-64bit using memory-mapping.  I couldn't achieve the same thing in Windows (mmap'ing seems to be garbage there).   I just wanted to share my experience from playing with leveldb, how awesomely optimized it is.

First, I iterated through blk0001.dat and stored all transactions in a leveldb database using Key:TxHash-->Value:RawTx.   I plugged 4 addresses into a set<BinaryData> object.  I then iterated (out of order) through the entire database, pulling each transaction into memory from the DB, and then searching for TxOuts matching any of those 4 addresses.  This took 8.0 seconds! (but was only the TxOuts).  This included putting the TxOuts into a map<OutPoint,uint64_t> (OutPoint, nBTC), so that I could then scan the entire DB again and search for TxIns. 

At the end of both scans, I had the balance and unspent TxOut list from those four addresses.  It took approximately 12-13 seconds!  And this is with a non-SSD drive:  it seems like it shouldn't even be possible to go that quickly... I had to quadruple-check everything to verify that I wasn't missing something or accidentally supplying the answer.  Perhaps there was some OS-level caching of the DB files, though I would've expected such caching to improve my custom blockchain utilities performance. 

I don't intend to discourage or insult your efforts (in fact, I think you said you were getting similar speeds).  Plus, I think that writing such a parser/scanner is both useful and educational.  I just bring it up because I would guess that this is about as good as anyone can do with a non-SSD drive.  And it's especially impressive considering I did no optimizations.  I'm especially curious how leveldb works under the hood that it could be so efficient.  Maybe there's something to learn from it.

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1470


View Profile
August 02, 2012, 02:06:36 PM
 #59

I don't intend to discourage or insult your efforts (in fact, I think you said you were getting similar speeds).  Plus, I think that writing such a parser/scanner is both useful and educational.  I just bring it up because I would guess that this is about as good as anyone can do with a non-SSD drive.  And it's especially impressive considering I did no optimizations.  I'm especially curious how leveldb works under the hood that it could be so efficient.  Maybe there's something to learn from it.

LevelDB provides the backend to Google's services, so it is not surprising that it is so heavily optimized.  One of the key optimizations is that writes go to a log, and then another thread comes along in the background and "optimizes" the log data back into the main database, using hueristics to hopefully place the data in a useful, locality-friendly location on disk.

BitcoinJ author Mike Hearn has proposed replacing BDB with LevelDB in the Satoshi client, and we are looking seriously at that.  Here is his pull request: https://github.com/bitcoin/bitcoin/pull/1619


Jeff Garzik, bitcoin core dev team and BitPay engineer; opinions are my own, not my employer.
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
etotheipi
Legendary
*
expert
Offline Offline

Activity: 1428


Core Armory Developer


View Profile WWW
August 02, 2012, 02:33:52 PM
 #60


Finally, it's a good idea to do this: in the end the blockchain will anyway grow
so big that even the hashmaps won't fit in RAM, and my approach will start
swapping like crazy. Pre-indexing in a disk-level DB hash table is the right thing
to do.


Well that's why I started looking into this.  My original blockchain utilities design in Armory were more of an educational adventure to see how fast I could get the scanning from scratch (no external libraries), with lots of pointers, etc.  It was educational, and I was happy with the results.  But I had the same realization that even the index will eventually not fit in RAM, and thus there is no way to make the library scalable without using some kind of disk-based engine.  To that end, I had no interest re-inventing disk-based DB operations, etc.  When I heard the devs talking about LevelDB, I looked into and quickly realized why they chose it.  It is very simple, portable... and apparently fast!

However, gmaxwell just told me how to clear my disk cache and then retest.  It was dramatically slower for sure.  Probably by a factor of 5-10x.  However, I never ran the same test using my other blockchain utilities, so I don't know the actual comparison.  Given that modern OS'es are good at doing that caching, the LevelDB version is clearly a very good solution.

I'm currently investigating how I can transition Armory to maintain it's own blockchain data now with LevelDB Smiley

Founder and CEO of Armory Technologies, Inc.
Armory Bitcoin Wallet: Bringing cold storage to the average user!
Only use Armory software signed by the Armory Offline Signing Key (0x98832223)

Please donate to the Armory project by clicking here!    (or donate directly via 1QBDLYTDFHHZAABYSKGKPWKLSXZWCCJQBX -- yes, it's a real address!)
Pages: « 1 2 [3] 4 5 6 7 8 9 10 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!