 Show Posts
|
|
Pages: [1]
|
|
Is it possible to initialize a blockchain and store to run in its own thread, and add wallets to it as they are created in fullNode Mode? then disconnect a wallet as a user leaves and reconnect, catchup to the chain and display balances when they come back?
i cant just have all wallets constantly added to the blockchain and watched as long as the blockchain is running, there would be millions of event listeners on the blockchain and that could never be done properly. every time a block comes in the server would lag to oblivion alerting 1000000 different event listeners, i need the wallets to catchup when they are added back. i need them to recognize the blocks when they are added back to the peergroup and chain, i can never re-download the blockstore when doing this, its not spv.
So how can i replay the chain to the wallet when it is added in WITHOUT redownloading an entire blockchain, or even any of the blockchain? i only need the blocks after GetEarliestKeyCreationTime() but it seems if i dont have the wallet added when the blocksDownloadedEventListener is called, the wallet does not see any blocks, ever.
if its not possible, when i update bitcoinj to make it possible (if thats even possible) i will post the code.
i could give each wallet its own blockchain but its not working, but i want a centralized store and peergroup. i cant have 1000 user connections each accessing its own peergroup and blockstore, id need a supercomputer server. in theory you would think a blockchain loads the neccessary parts of a store in to memory.
i can grab address balance and UTXOs with a store, is it possible to manually grab all this and load a wallet so i can still sue the wallets send and receive functionality?
I want to use the hundreds of thousands of blocks and transactions i have saved in the database and replay that to the wallet without having to fetch thousands of blocks from the network for every wallet every time it is loaded. i know they all work on event listeners, but what is the point of a store if i can never go back and replay it?
edit: i am also willing to hire somebody for help with some of these issues if it is more advanced.
|
|
|
|
|
found my solution, added a check if tally is null in the NEtworkParameters.java file and re-compiled the jar. seems liek a workaround but really i think its fine as in mainnet it will work as expected and it allows regtest to work and checkLockTime will still be set down the line on the chain.
|
|
|
|
|
Well in the main thread i had somebody point out that it was the java, you are 100% correct.
however why is it that in testnet a blocks version is = 1 (why my node works with testnet)
but in regtest i have some very high block version because regtest is generating blocks with all sorts of verification requirements.
is there a way to fix this to mirror mainnet properly so bitcoinj can work?
also does mainnet start with version=1 and slowly increase as new BIP versions were released? why cant regtest do this too?
|
|
|
|
|
hello,
debug.log file is empty i believe since i use printtoconsole.
and im not running debug=net, im running debug=1 which should be full debug or the same as -debug in the command line right? im also mixing cmd flags and bitcoin.conf file
i will look in to how to get a proper debug, i thought i had full debug. i responded with the bitcoinj code to other thread.
UPDATE: got rid of printtoconsole and debug=1 in config, set -debug flag in command line when running bitcoind, same error log, no additional info. i will try some other flags and see if i pull more info somehow.
|
|
|
|
|
update:
So it seems to me bitcoinj could be bugged? well kind of. it may be intended functionality. my block version is high because of the transaction verification rules, but a chain is expected to have a block version 1 at the genesis block, so this strange version is really throwing off bitcoinj i think. mainnet works, testnet works, regtest does not primarily because bitcoinj was programmed to handle the forks when verifying the transactions, and it knows mainnet will always have a version of 1 at its genesis block and for awhile thereafter. and when the code was made it expects something in the block-chain. and the people who made bitcoinj never thought to add an if tally != Null. but maybe checklocktimeverify should be ran on the first blocks anyway? i dont know.
i think the only fix is to update bitcoinj and re-compile it which i will do however i don't want to compromise security by doing this, and im not sure how it will effect me when i move back to mainnet.
|
|
|
|
So i recently posted a large question here that involves both bitcoind and j. https://bitcointalk.org/index.php?topic=2245534.msg22688171#msg22688171this talks about what bitcoinj is doing but half the question related to bitcoind exclusively. so i have a few questions i will list here without the bitcoinj stuff and split the one question in to 2 sets of questions basically. if this is not allowed please just delete this topic and i wont re-post. bitcoind is disconnecting my bitcoin j node (i think) after the block is served. bitcoinj says the block is invalid and it is due to locktime i think based on the errors i see. but why would bitcoind set its locktime to a time that is invalid or null? block verification is failing, and it only is in regtest mode, testnet is fine meaning i am missing something in my regtest node i cant figure out. either it is stuck in intialblockdownload, or the locktime is off somehow, or some otehr time issue causing a failure of verification. bitcoind regtest nodes sync but still have the timing mishap. 1. what is a timing mishap? i cant find any info on it ANYWHERE, maybe i am not searching right. why would this be happening? 2. what are transaction verification flags? and how can i see them in bitcoin-cli? 3. how can i see if bitcoind is serving the block or disconnecting before it serves the block, is there any way to get more detailed logging? or a way to snoop on the communication? can i tell who initiated the connection termination? 4. can a protocol version 70000 communicate effectively with 70015? 5. is there any reason a regtest enviornment would act differently from mainnet in regards to block/transaction flags or the way the locktimes are set? i really dont know where to even look here. all i know is bitcoind nodes sync but have timing mishaps, bitcoinj cant pull FULL . for the record, bitcoinjs full node code mirrors satoshi's exactly, it has to, so i have read. so im not sure how it communicates but i assume the same way bitcoind clients communicate. and if you do have experience iwth bitcoinj, please check my other thread, i will be eternally greatful to anybody who can help. feel free to message in private as well. bitcoind running with this: and the log from bitcoind:
2017-10-08 03:29:13 Bitcoin version v0.15.0.1
2017-10-08 03:29:13 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2017-10-08 03:29:13 Validating signatures for all blocks.
2017-10-08 03:29:13 Using the 'standard' SHA256 implementation
2017-10-08 03:29:13 Using RdRand as an additional entropy source
2017-10-08 03:29:14 RandAddSeedPerfmon: 605584 bytes
2017-10-08 03:29:14 Default data directory C:\Bitcoin
2017-10-08 03:29:14 Using data directory C:\regtest
2017-10-08 03:29:14 Using config file C:/bitcoin.conf
2017-10-08 03:29:14 Using at most 125 automatic connections (2048 file descriptors available)
2017-10-08 03:29:14 Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2017-10-08 03:29:14 Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2017-10-08 03:29:14 Using 4 threads for script verification
2017-10-08 03:29:14 scheduler thread start
2017-10-08 03:29:14 Allowing HTTP connections from: 127.0.0.0/8 ::1/128
2017-10-08 03:29:14 Binding RPC on address ::1 port 8444
2017-10-08 03:29:14 libevent: getaddrinfo: nodename nor servname provided, or not known
2017-10-08 03:29:14 Binding RPC on address ::1 port 8444 failed.
2017-10-08 03:29:14 Binding RPC on address 127.0.0.1 port 8444
2017-10-08 03:29:14 Initialized HTTP server
2017-10-08 03:29:14 HTTP: creating work queue of depth 16
2017-10-08 03:29:14 Starting RPC
2017-10-08 03:29:14 Starting HTTP RPC server
2017-10-08 03:29:14 Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcuser for rpcauth auth generation.
2017-10-08 03:29:14 Registering HTTP handler for / (exactmatch 1)
2017-10-08 03:29:14 Registering HTTP handler for /wallet/ (exactmatch 0)
2017-10-08 03:29:14 Starting HTTP server
2017-10-08 03:29:14 HTTP: starting 4 worker threads
2017-10-08 03:29:14 init message: Verifying wallet(s)...
2017-10-08 03:29:14 Entering http event loop
2017-10-08 03:29:14 Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2017-10-08 03:29:14 Using wallet wallet.dat
2017-10-08 03:29:14 CDBEnv::Open: LogDir=C:\regtest\database ErrorFile=C:\regtest\db.log
2017-10-08 03:29:14 Cache configuration:
2017-10-08 03:29:14 * Using 2.0MiB for block index database
2017-10-08 03:29:14 * Using 8.0MiB for chain state database
2017-10-08 03:29:14 * Using 440.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2017-10-08 03:29:14 init message: Loading block index...
2017-10-08 03:29:14 Opening LevelDB in C:\regtest\blocks\index
2017-10-08 03:29:14 Recovering log #12
2017-10-08 03:29:14 Level-0 table #14: started
2017-10-08 03:29:14 Level-0 table #14: 179 bytes OK
2017-10-08 03:29:14 Delete type=0 #12
2017-10-08 03:29:14 Delete type=3 #10
2017-10-08 03:29:14 Opened LevelDB successfully
2017-10-08 03:29:14 Compacting 4@0 + 0@1 files
2017-10-08 03:29:14 Using obfuscation key for C:\regtest\blocks\index: 0000000000000000
2017-10-08 03:29:14 LoadBlockIndexDB: last block file = 0
2017-10-08 03:29:14 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=102, size=24012, heights=0...101, time=2011-02-02...2017-10-06)
2017-10-08 03:29:14 Checking all blk files are present...
2017-10-08 03:29:14 LoadBlockIndexDB: transaction index disabled
2017-10-08 03:29:14 Opening LevelDB in C:\regtest\chainstate
2017-10-08 03:29:14 Recovering log #13
2017-10-08 03:29:14 Level-0 table #15: started
2017-10-08 03:29:14 Generated table #16@0: 105 keys, 14217 bytes
2017-10-08 03:29:14 Compacted 4@0 + 0@1 files => 14217 bytes
2017-10-08 03:29:14 Level-0 table #15: 308 bytes OK
2017-10-08 03:29:14 compacted to: files[ 0 1 0 0 0 0 0 ]
2017-10-08 03:29:14 Delete type=2 #5
2017-10-08 03:29:14 Delete type=2 #8
2017-10-08 03:29:14 Delete type=2 #11
2017-10-08 03:29:14 Delete type=2 #14
2017-10-08 03:29:14 Delete type=0 #13
2017-10-08 03:29:14 Delete type=3 #11
2017-10-08 03:29:14 Opened LevelDB successfully
2017-10-08 03:29:14 Using obfuscation key for C:\regtest\chainstate: dd6ff88733cb0b53
2017-10-08 03:29:14 Loaded best chain: hashBestChain=35ebf25b797550e724f1877bc1f64531efce23eae68d804fc9301b8754b67cd2 height=101 date=2017-10-06 05:28:27 progress=1.000000
2017-10-08 03:29:14 init message: Rewinding blocks...
2017-10-08 03:29:14 Writing final batch of 0.00 MiB
2017-10-08 03:29:14 Committed 0 changed transaction outputs (out of 0) to coin database...
2017-10-08 03:29:14 init message: Verifying blocks...
2017-10-08 03:29:14 Verifying last 6 blocks at level 3
2017-10-08 03:29:14 [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
2017-10-08 03:29:14 No coin database inconsistencies in last 7 blocks (7 transactions)
2017-10-08 03:29:14 block index 94ms
2017-10-08 03:29:14 Reading estimates: 190 buckets counting confirms up to 48 blocks
2017-10-08 03:29:14 Reading estimates: 190 buckets counting confirms up to 12 blocks
2017-10-08 03:29:14 Reading estimates: 190 buckets counting confirms up to 1008 blocks
2017-10-08 03:29:14 init message: Loading wallet...
2017-10-08 03:29:14 nFileVersion = 150001
2017-10-08 03:29:14 Keys: 2003 plaintext, 0 encrypted, 2003 w/ metadata, 2003 total
2017-10-08 03:29:14 wallet 78ms
2017-10-08 03:29:14 setKeyPool.size() = 2000
2017-10-08 03:29:14 mapWallet.size() = 101
2017-10-08 03:29:14 mapAddressBook.size() = 1
2017-10-08 03:29:14 mapBlockIndex.size() = 102
2017-10-08 03:29:14 nBestHeight = 101
2017-10-08 03:29:14 Imported mempool transactions from disk: 0 successes, 0 failed, 0 expired
2017-10-08 03:29:14 Bound to 127.0.0.1:18444
2017-10-08 03:29:14 Bound to 127.0.0.1:18444
2017-10-08 03:29:14 init message: Loading P2P addresses...
2017-10-08 03:29:14 Loaded 0 addresses from peers.dat 0ms
2017-10-08 03:29:14 init message: Loading banlist...
2017-10-08 03:29:14 Loaded 0 banned node ips/subnets from banlist.dat 0ms
2017-10-08 03:29:14 init message: Starting network threads...
2017-10-08 03:29:14 net thread start
2017-10-08 03:29:14 init message: Done loading
2017-10-08 03:29:14 opencon thread start
2017-10-08 03:29:14 dnsseed thread start
2017-10-08 03:29:14 addcon thread start
2017-10-08 03:29:14 msghand thread start
2017-10-08 03:29:14 Loading addresses from DNS seeds (could take a while)
2017-10-08 03:29:14 0 addresses found from DNS seeds
2017-10-08 03:29:14 dnsseed thread exit
2017-10-08 03:30:07 Added connection peer=0
2017-10-08 03:30:07 connection from 127.0.0.1:19377 accepted
2017-10-08 03:30:07 received: version (103 bytes) peer=0
2017-10-08 03:30:07 sending version (104 bytes) peer=0
2017-10-08 03:30:07 send version message: version 70015, blocks=101, us=[::]:0, peer=0
2017-10-08 03:30:07 sending verack (0 bytes) peer=0
2017-10-08 03:30:07 receive version message: /bitcoinj:0.14.5/: version 70001, blocks=0, us=127.0.0.1:18444, peer=0
2017-10-08 03:30:07 added time data, samples 2, offset +0 (+0 minutes)
2017-10-08 03:30:07 sending alert (168 bytes) peer=0
2017-10-08 03:30:07 received: verack (0 bytes) peer=0
2017-10-08 03:30:07 sending ping (8 bytes) peer=0
2017-10-08 03:30:07 received: pong (8 bytes) peer=0
2017-10-08 03:30:07 pong peer=0: Timing mishap, 5d8e9xxxxxx54126 expected, 5d8e9xxxxxx54126 received, 8 bytes
2017-10-08 03:30:09 received: ping (8 bytes) peer=0
2017-10-08 03:30:09 sending pong (8 bytes) peer=0
2017-10-08 03:30:11 received: ping (8 bytes) peer=0
2017-10-08 03:30:11 sending pong (8 bytes) peer=0
2017-10-08 03:30:12 received: getblocks (69 bytes) peer=0
2017-10-08 03:30:12 getblocks 1 to end limit 500 from peer=0
2017-10-08 03:30:12 sending inv (3637 bytes) peer=0
2017-10-08 03:30:12 received: getdata (3637 bytes) peer=0
2017-10-08 03:30:12 received getdata (101 invsz) peer=0
2017-10-08 03:30:12 received getdata for: block 237446d4xxxxxxxxxxxab151a393df9a42731c894fe087379bcc24484820c843 peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 socket closed
2017-10-08 03:30:12 sending block (226 bytes) peer=0
2017-10-08 03:30:12 disconnecting peer=0
2017-10-08 03:30:12 Cleared nodestate for peer=0
2017-10-08 03:30:14 Added connection peer=1
2017-10-08 03:30:14 connection from 127.0.0.1:19378 accepted
2017-10-08 03:30:14 received: version (103 bytes) peer=1
2017-10-08 03:30:14 sending version (104 bytes) peer=1
2017-10-08 03:30:14 send version message: version 70015, blocks=101, us=[::]:0, peer=1
2017-10-08 03:30:14 sending verack (0 bytes) peer=1
2017-10-08 03:30:14 receive version message: /bitcoinj:0.14.5/: version 70001, blocks=0, us=127.0.0.1:18444, peer=1
2017-10-08 03:30:14 sending alert (168 bytes) peer=1
2017-10-08 03:30:14 received: verack (0 bytes) peer=1
2017-10-08 03:30:14 sending ping (8 bytes) peer=1
2017-10-08 03:30:14 received: getblocks (69 bytes) peer=1
2017-10-08 03:30:14 getblocks 1 to end limit 500 from peer=1
2017-10-08 03:30:14 sending inv (3637 bytes) peer=1
2017-10-08 03:30:14 received: pong (8 bytes) peer=1
2017-10-08 03:30:14 pong peer=1: Timing mishap, 90exxxxxxf137c88 expected, 90exxxxxxf137c88 received, 8 bytes
2017-10-08 03:30:14 received: getdata (3637 bytes) peer=1
2017-10-08 03:30:14 received getdata (101 invsz) peer=1
2017-10-08 03:30:14 received getdata for: block 237446d4xxxxxxxxxxxab151a393df9a42731c894fe087379bcc24484820c843 peer=1
2017-10-08 03:30:14 sending block (226 bytes) peer=1
2017-10-08 03:30:14 sending block (226 bytes) peer=1
2017-10-08 03:30:14 socket closed
running bitcoind with (C is not the actual path): bitcoind -regtest -listen -datadir=C:/ -conf=C:/bitcoin.conf -whitebind=127.0.0.1:18444 -whitelist=127.0.0.1
Peer info on bitcoind node:
{
"id": 45, (this is because node has been running through failed sync disconnecting 44 prior peers)
"addr": "127.0.0.1:19094",
"addrlocal": "127.0.0.1:18444",
"addrbind": "127.0.0.1:18444",
"services": "0000000000000000",
"relaytxes": true,
"lastsend": 1507401729,
"lastrecv": 1507401729,
"bytessent": 722,
"bytesrecv": 340,
"conntime": 1507401723,
"timeoffset": 0,
"version": 70001,
"subver": "/bitcoinj:0.14.5/",
"inbound": true,
"addnode": false,
"startingheight": 0,
"banscore": 0,
"synced_headers": -1,
"synced_blocks": -1,
"inflight": [
],
"whitelisted": true,
"bytessent_per_msg": {
"alert": 192,
"block": 250,
"ping": 32,
"pong": 96,
"verack": 24,
"version": 128
},
"bytesrecv_per_msg": {
"getdata": 61,
"ping": 96,
"pong": 32,
"verack": 24,
"version": 127
}
}
|
|
|
|
TL/DR: bitcoinj disconnects from bitcoind in regtest mode when bitcoinj is in fullpruned mode right after bitcoind serves a block. logs below. timing mishap in log seems to be an issue. Hello again. im curious if anybody here that understands bitcoin better than myself can take a look at these logs and maby identify what is going on. why, when i try to download the blockchain with a fullprunedblockchain (everything else works, any store, spvchain,etc..) does bitcoinj force disconnect right after bitcoind serves the full block? I hope somebody here can help me, im like 3 weeks in on the same issue with no progress even with a few sets of eyes on it, and documentation on regtest with bitcoinj in full node mode is lacking/non existant (or i just cant find it). i can not get the bitcoinj library working with a fullprunedblockchain in a regtest environment (testnet + mainnet work fine). i along with a few others built a full wallet from scratch using only the lower level classes in spvblockstore mode, but it failed during the change to full pruned so i set that aside for now and started testing with a custom version of the walletappkit class and also re-writing a fresh class that is as simple as can be. i literally just began using that class, changed the chain + store, and started debugging the kit to take the variable out of incorrect code being the issue (however the FPBC may require something additional i dont know about). the few paragraphs of info i found on fullprunedblockstore state it should be handled about the same as anything else in terms of how to connect and DL the chain. im curious if anybody can spot why im having this issue. i believe the code is correct but i may be mistaken, if it was code it would be solved by now. i even tried with no verification scripts, whitelisting, whitebinding,multiple local nodes in round robin in an attempt to help with verification to no avail. but i think it is my bitcoin node in -regtest mode. running no special flags other than -server and -listen and -regtest and -bind to 127.0.0.1 (well recently ive added quite a bit more, no changes) here is the recent bitcoinj log. the final null pointer exception and illegal state exception repeat on each peer reconnect, alsop triggering peerDisconnectedEventListener and occasionally a peerConnect EV every 30 or so disconnects. bitcoind is clearing node state on each disconnect and adding a new peer every time the disconnect message shows. it works with testnet3params just fine and dns peer discovery but i cant use testnet now. the wallet/chain height mismatch is strange to me. the timing mishap is biggest red flag, not sure what it means or how to resolve, even local nodes have timing mishaps when running multiple bitcoind nodes. bitcoind version 12,14,and 15 were tried. the bitcoind getpeerinfo on bitcoinj peer when bitcoinj is just grabbing a single block header, pulling during block sync is too hard since its disconnecting too often.
{
"id": 45, (this is because node has been running through failed sync disconnecting 44 prior peers)
"addr": "127.0.0.1:19094",
"addrlocal": "127.0.0.1:18444",
"addrbind": "127.0.0.1:18444",
"services": "0000000000000000",
"relaytxes": true,
"lastsend": 1507401729,
"lastrecv": 1507401729,
"bytessent": 722,
"bytesrecv": 340,
"conntime": 1507401723,
"timeoffset": 0,
"version": 70001,
"subver": "/bitcoinj:0.14.5/",
"inbound": true,
"addnode": false,
"startingheight": 0,
"banscore": 0,
"synced_headers": -1,
"synced_blocks": -1,
"inflight": [
],
"whitelisted": true,
"bytessent_per_msg": {
"alert": 192,
"block": 250,
"ping": 32,
"pong": 96,
"verack": 24,
"version": 128
},
"bytesrecv_per_msg": {
"getdata": 61,
"ping": 96,
"pong": 32,
"verack": 24,
"version": 127
}
}
Bitcoind Log: (had to edit some things with "x"s, sorry for this) 2017-10-01 07:47:33 receive version message: /bitcoinj:0.14.5/: version 70001, blocks=0, us=127.0.0.1:18444, peer=75
2017-10-01 07:47:33 sending alert (168 bytes) peer=75
2017-10-01 07:47:33 received: verack (0 bytes) peer=75
2017-10-01 07:47:33 sending ping (8 bytes) peer=75
2017-10-01 07:47:33 received: getblocks (69 bytes) peer=75
2017-10-01 07:47:33 getblocks 1 to end limit 500 from peer=75
2017-10-01 07:47:33 sending inv (3637 bytes) peer=75
2017-10-01 07:47:33 received: pong (8 bytes) peer=75
2017-10-01 07:47:33 pong peer=75: Timing mishap, f53e1xxxxxx71798 expected, f53e1xxxxxx71798 received, 8 bytes
2017-10-01 07:47:33 received: getdata (3637 bytes) peer=75
2017-10-01 07:47:33 received getdata (101 invsz) peer=75
2017-10-01 07:47:33 received getdata for: block 0f09eb8064aa08a45fa5f49097b150875ac752944xxxxxxxxxxxxxxxxxxxx peer=75
2017-10-01 07:47:33 sending block (226 bytes) peer=75
2017-10-01 07:47:33 sending block (226 bytes) peer=75
2017-10-01 07:47:33 socket closed
2017-10-01 07:47:33 sending block (226 bytes) peer=75
2017-10-01 07:47:33 disconnecting peer=75
2017-10-01 07:47:35 Added connection peer=76
The bitcoinj log: [main] INFO org.bitcoinj.core.Context - Creating bitcoinj 0.14.5 context.
[CUSTOMKIT STARTING] INFO CUSTOMKIT - Starting up with directory = test10021711.wallet (not saving or loading wallets, this is not applicable)
[CUSTOMKIT STARTING] INFO org.bitcoinj.wallet.DeterministicKeyChain - 133 keys needed for M/0H/0 = 1 issued + 100 lookahead size + 33 lookahead threshold - 1 num children
[CUSTOMKIT STARTING] INFO org.bitcoinj.wallet.DeterministicKeyChain - Took 54.86 ms
[CUSTOMKIT STARTING] INFO org.bitcoinj.wallet.DeterministicKeyChain - 133 keys needed for M/0H/1 = 0 issued + 100 lookahead size + 33 lookahead threshold - 0 num children
[CUSTOMKIT STARTING] INFO org.bitcoinj.wallet.DeterministicKeyChain - Took 24.25 ms
[CUSTOMKIT STARTING] INFO org.bitcoinj.core.AbstractBlockChain - chain head is at height 0:
block:
hash: 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
version: 1
previous block: 0000000000000000000000000000000000000000000000000000000000000000
merkle root: 4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b
time: 1296688602 (2011-02-02T23:16:42Z)
difficulty target (nBits): 545259519
nonce: 2
[CUSTOMKIT STARTING] WARN org.bitcoinj.core.AbstractBlockChain - Wallet/chain height mismatch: -1 vs 0
[CUSTOMKIT STARTING] WARN org.bitcoinj.core.AbstractBlockChain - Hashes: null vs 0f9188f13cb7b2cxxxxxxxxx3a4fc328bf5beb436012afca590b1a11466e2206
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Starting ...
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Localhost peer detected, trying to use it instead of P2P discovery
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Attempting connection to [127.0.0.1]:18444 (0 connected, 1 pending, 0 max)
[NioClientManager] INFO org.bitcoinj.net.NioClientManager - Connected to /127.0.0.1:18444
[NioClientManager] INFO org.bitcoinj.core.Peer - Announcing to /127.0.0.1:18444 as: /bitcoinj:0.14.5/
[NioClientManager] INFO org.bitcoinj.core.Peer - [127.0.0.1]:18444: Got version=70015, subVer='/Satoshi:0.14.2/', services=0x13, time=2017-10-01 03:47:27, blocks=101
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - [127.0.0.1]:18444: New peer (1 connected, 0 pending, 1 max)
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - Setting download peer: [127.0.0.1]:18444
[NioClientManager] INFO org.bitcoinj.core.listeners.DownloadProgressTracker - Downloading block chain of size 101.
[NioClientManager] INFO org.bitcoin.Secp256k1Context - java.lang.UnsatisfiedLinkError: no secp256k1 in java.library.path
[NioClientManager] WARN org.bitcoinj.core.PeerSocketHandler - [127.0.0.1]:18444 -
java.lang.NullPointerException
at org.bitcoinj.core.NetworkParameters.getTransactionVerificationFlags(NetworkParameters.java:517)
at org.bitcoinj.core.FullPrunedBlockChain.connectTransactions(FullPrunedBlockChain.java:234)
at org.bitcoinj.core.AbstractBlockChain.connectBlock(AbstractBlockChain.java:561)
at org.bitcoinj.core.AbstractBlockChain.add(AbstractBlockChain.java:489)
at org.bitcoinj.core.AbstractBlockChain.add(AbstractBlockChain.java:352)
at org.bitcoinj.core.Peer.processBlock(Peer.java:996)
at org.bitcoinj.core.Peer.processMessage(Peer.java:491)
at org.bitcoinj.core.PeerSocketHandler.receiveBytes(PeerSocketHandler.java:182)
at org.bitcoinj.net.ConnectionHandler.handleKey(ConnectionHandler.java:223)
at org.bitcoinj.net.NioClientManager.handleKey(NioClientManager.java:86)
at org.bitcoinj.net.NioClientManager.run(NioClientManager.java:122)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$2.run(AbstractExecutionThreadService.java:60)
at com.google.common.util.concurrent.Callables$3.run(Callables.java:95)
at org.bitcoinj.utils.ContextPropagatingThreadFactory$1.run(ContextPropagatingThreadFactory.java:49)
at java.lang.Thread.run(Unknown Source)
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - [127.0.0.1]:18444: Peer died (0 connected, 0 pending, 1 max)
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - Download peer died. Picking a new one.
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - Unsetting download peer: [127.0.0.1]:18444
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Peer discovery took 19.33 ?s and returned 0 items
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Waiting 1500 msec before next connect attempt to [127.0.0.1]:18444
[NioClientManager] WARN org.bitcoinj.net.ConnectionHandler - Error handling SelectionKey: java.lang.IllegalStateException
java.lang.IllegalStateException
at com.google.common.base.Preconditions.checkState(Preconditions.java:158)
at org.bitcoinj.net.ConnectionHandler.handleKey(ConnectionHandler.java:224)
at org.bitcoinj.net.NioClientManager.handleKey(NioClientManager.java:86)
at org.bitcoinj.net.NioClientManager.run(NioClientManager.java:122)
at com.google.common.util.concurrent.AbstractExecutionThreadService$1$2.run(AbstractExecutionThreadService.java:60)
at com.google.common.util.concurrent.Callables$3.run(Callables.java:95)
at org.bitcoinj.utils.ContextPropagatingThreadFactory$1.run(ContextPropagatingThreadFactory.java:49)
at java.lang.Thread.run(Unknown Source)
[PeerGroup Thread] INFO org.bitcoinj.core.PeerGroup - Attempting connection to [127.0.0.1]:18444 (0 connected, 1 pending, 1 max)
[NioClientManager] INFO org.bitcoinj.net.NioClientManager - Connected to /127.0.0.1:18444
[NioClientManager] INFO org.bitcoinj.core.Peer - Announcing to /127.0.0.1:18444 as: /bitcoinj:0.14.5/
[NioClientManager] INFO org.bitcoinj.core.Peer - [127.0.0.1]:18444: Got version=70015, subVer='/Satoshi:0.14.2/', services=0x13, time=2017-10-01 03:47:29, blocks=101
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - [127.0.0.1]:18444: New peer (1 connected, 0 pending, 1 max)
[NioClientManager] INFO org.bitcoinj.core.PeerGroup - Setting download peer: [127.0.0.1]:18444
[NioClientManager] INFO org.bitcoinj.core.listeners.DownloadProgressTracker - Chain download switched to [127.0.0.1]:18444
(this is the first null pointer error) line 517 of org.bitcoinj.core.NetworkParameters: if (block.getVersion() >= Block.BLOCK_VERSION_BIP65 &&
tally.getCountAtOrAbove(Block.BLOCK_VERSION_BIP65) > this.getMajorityEnforceBlockUpgrade()) {
verifyFlags.add(Script.VerifyFlag.CHECKLOCKTIMEVERIFY);
}
|
|
|
|
|
Wow you really know what you are doing, im checking through your github now. ill spend a few hours reading through code to see if its something i could tackle in a reasonable amount of time or not.
My biggest issue with bitcoinj s that i cant manage to persist a balance without deleting and re-downloading the block-chain. in main net this would be horribly slow for a server side app. maybe i am wrong and im doing it incorrectly, but i have about a week experience with bitcoinj.
My plan b was, use bitcoin-j, rewrite the source a bit to allow for a more "direct" approach. but your classes look a lot like whats in bitcoinj too, did you use bitcoinj as a base? how did you go about writing that?
can you outline a bit about what was involved in writing yoru bitcoin core framework/library or whatever you may call it.
also i notice a lot of bloom filtering and merkel root classes, is this doing the same as bitcoinj by only using spv wallets? or are those classes just carried over?
|
|
|
|
|
I recently posted a thread looking for help finding a proper alternative to Bitcoind for a web app.
I began using bitcoinj after it was recommended in the thread, steering away from fully writing everything. i finally have a base understanding of it, but a very small base.
Its been working great, but i see some issues. Wallets must be saved all the time, if a save is missed due to some issue, the wallet will show messed up balances and give restrictions on available inputs. i prefer a more live approach where i can verify the inputs/outputs against the chain that is already downloaded, but bitcoinj seems to only allow a one time update so to speak. if you dont save the wallet to disk when the tx comes in or is replayed, its non existent to your wallet. no "quick check" of the block store to be sure. maybe i am wrong, but its what it seems like. i spent days trying to get a balance to persist without saving the wallet to disk when the TX comes in, and only saving the SPV store and the wallet keys, i dont think its possible with the current setup of bitcoinj for the balance to persist using a live sync without re-download. it seems i must completely delete the chain file, re-download block chain and finally re-populate wallet. could be very time consuming on mainnet correct?
I like bitcoin j, i like how its set up and give a lot of control, but i dont like how i am unable to have a full node. i want a node that i can scan TXs against the downloaded chain, update immediate, but also replay without issues. the point of bitcoinj is that its a light client, so i think i have the wrong tool for the job.
this being said, does any of what i said make sense? is it a valid concern or do i just have no understanding of what i actually need?
if this is a valid concern, is bitcoinj still a viable library for a server side app? or is there a tool better suited for this?
i read some material on writing a client from "scratch", its just too involved. So if i shouldn't continue to use bitcoinj server-side, is there an equivilent that is also pretty low level and allows for "infinite" wallet creation?
|
|
|
|
|
I did not expect all these great answers!
samson, you are the man, thank you!
That is what i was looking for, i couldnt find much. im somewhat educated with asymmetrical encryption but im sure i could get the rest taken care of pretty easily. i mean its not like i have to write the encryption algorithims, i just have to find a library with teh functionality built in.
however the random number generator thing seems scary to me. It makes me doubt my ability to keep a system like that secure. writing a non clock-based random numebr generator from scratch can be difficult, and the built in random functions in C are useless when trying to build entropy for a key since they can be broken with ease in most cases.
Regardless im just rambling, that is exactly what i needed! even just your few pointers gave me enough to get started with where to look. I think i will start with attempting to use bitcoinj then if it fails i will "do it the hard way" so to speak.
To the guy above:
I can not use a third party wallet provider, for a business of this scale. i will look in to it as maybe i dont 100% understand what exactly it is they offer, but as yo usay its limited functionality i doubt it would be of any use. i am not looking for a way to host a wallet to take payments and process payments, im looking for a way to have full control over everything from creating multisig individualized wallets, raw transactions, selecting which inputs to output, when to sign, when to broadcast, and so on. i doubt much else other than a library would solve my issue, or a REALLY good RPC (like bitcoind but with a few added features).
but if you think these third party providers could offer me this functionality please let me know! im excited to try anything and everything at this point.
|
|
|
|
|
genesis.vision:
i heard of bitcoinj in passing but dismissed it due to teh fact it is java based, i have minimal experience in java although your recommendation makes me think it deserves a second look. i bet i could pick the language up rather easily due to how similar it is to C++. i just worry about it in terms of security, anybody have any thoughts on this? is java the best route when security is top priority?
you say it has an RPC though, can anybody elaborate on this? if its a library how can it have an RPC? i assume by just giving classes that allow for an RPC to easily be created once compiled and ran in java? im asking here just for redundancy, i will dig in to it more, but so far i have found little about anything to do with an RPC.
I will dig in to this a lot more, thanks a bunch for recommendation!
|
|
|
|
CR1776- thank you! it may not be but i will read and let you know! also i agree about the header, but i assure you i am not ignorant to what i was writing, it just did not come out the way i planned after editing my post. it does not come from a place of ignorancet hough, the "mess" was in reference to its current state as it is in a transation period. it is currently in the process of accounting being deprecated, with partially integrated multi-sig and soon-to-be partially implemented multi-wallet support. its only part way done in regards to the RPC. That is my opinion although i agree with your statement 100%, it does not make a good impression even though it was a focal point ofm y message before i changed it around. HCP- i may wait, i just wonder how long, ill have to look in to it a bit more. thanks for the info! no - sorry, no. I think this is too easy. Forces me into another comment (rant? Roll Eyes): im all for a good rant, im just hoping the misunderstanding is now clear. i responded to your points just out of good fun, but it does help me as well since if i should change i will. im playing devils advocate a bit here as i do agree with you, but i still think c# is best for my project and there is no viable way to run c# on a linux os using apache since it can not handle the .net framework. so then what, python+php? c++ even has issues. i just dont know about that. yo ucan not ignore the extreme lack of EXTENSIVE software developed for linux, the repositories are growing but it is a fraction of what we can get with windows. Windows 10 comes per default with "calling home" functionlaity, and you can de-activate it. Yes, but that's not my point. Also not the ongoing automatic "security" updates (which proofs, that software wasn't written correctly - and this is from professinal enterprise software developpers). Without wanting to go into real details and the overhowling long history of vulnarabilities in the operating system, you cannot trust closed source systems and drivers. I think you confuse anonymity and personal security with software security. 100% agreed, however i will play devils advocate again for a moment. you also can not trust open source just becausei t is open source, its all on a per case basis depending on procedures the developers have in place. its a fallacy if you believe open source is secure because many eyes are better than few eyes. we know from experience that users of open source all depend on the next guy to check out and verify it, but most people with the ability to audit a software dont have the time or motivation, so it goes un-audited. truecrypt was open source, it took YEARS to audit and many flaws were found. the only people iwth true motivation to go through the source are hackers, paid auditors (some use them, but not all, but do you really trust them either?), the developers, Gov, and some oddballs that are very interested and bored, or enterprise that really care about security. this leaves a lot of open source projects left un-checked with their source open for all to see... and break. and we are not discussing windows 10 here, we are talking about windows server. different kernal + os all together (slightly, but still different). and software is never written correctly from day 1, new exploits arise, it usually is not fault of the developers (well it is fault of the developers but it is impossible for projects of that complexity to have all current and potential holes covered. the more complex a system, the more ways it can be broken). it may be fault of a library used or a plugin, it may be the fault of a partner software. look at ubuntu, it is a bloatware filled OS, i personally feel it is more secure than windows 10, but it has the same issues. updates help this, and while an ANONYMITY concern, its not a security concern due to modern day signature verification of the updates. unless you read and understand the source yourself, and i mean actually read it through, you can trust it about as much as closed source. You almost never have the source code to evaluate the possibly of hidden malfunctions. Compare this to Unix or xBSD, deep down into the kernel. Ahh, the kernel... the windoofs kernel integrates sooooo many functions and drivers, that the layer of rights can be easily violated with a single piece of software, and you don't know it. In Linux and xBSD and OSX the underlying OS has an onion type approach, for segregation of duties (best is OpenBSD IMHO). I love debian, its my favorite os in conjunction with kde. but that is my personal preference, linux has flaws. its more secure because less hackers understand it, and you have more control over its operations. linux with an uneducated user is as bad as windows. but regardless, how often have you looked through openBSDs source? id bet good money you have not read and understood more than a few pages unless you are involved in the project. you expect others have though, but who? open bsd is the most popular so has the most eyes, but do you trust those eyes? you say openBSD may have hidden malfunctions, well if that is the case it was not developed correctly from the start (ofcourse) and who will catch it first, an attacker or the good guys? Whereas in older times it was annoying, that you got a blue screen, and had to restart your machine, nowadays it became a threat. And this threat became very much clear with the audio driver on HP systems. The audio driver logged each (cool, eh?) keystroke. Well, one might ask, why does an audio driver need to log each keystroke, but that's not the point where I want to go to. Fact is, this happens all the time, and imagine you are typing in your privkey or seed, and it is logged into a data structure, where you don't know what happens with it. Bitcoin is well enough known today, to store enourmos amounts of money... How can someone evaluate a closed source driver update (for sure only for security reasons)?
again we are discussing personal anonymity, not corporate security. and again, this can easily be prevented using proper firewall rules, the logging is bad buit without access to teh network it is useless. the good thing is, if i dont know what happens to it, neither does an attacker, would i consider hp an attacker? no, a threat yes but to freedom as a whole not the security of my software unless the software was in some way a target of hp or the gov, which at this point is not the case. if hp ever caused financial losses to me or the users, its like a built in insurance policy when they are sued in a class action. I really am an anonymity nut, but in business i feel windows is a valid option. its very enterprise oriented. C# is an amazing language, i mean its not VB were talking about here. C# can be everybodies best friend. i know real dirt low level languages, im pretty experienced with ASM to a small extent even, i even know very high level like java. c# can be both, sure its .net but it still has c++ functionality. and nobody would call c++ an insecure language. windows should be avoided at all costs, windows 10 disgusts me with how they treat users as advertising IDs and nothing more, they are a curse on teh technology community and will put an end to interest in IT as a whole. but ill be damned if they didnt produce a great way to develop in depth web applications and software. if only iis and microsoft sql were up to speed. Windows, apache, mysql, c# is my preferred setup now second to lamp. i dont touch java, never plan to unless persuaded properly. What I try to say: Windows might be ok, as an operating system for companies and enterprises, cause they can afford to calculate the risk against the necessary security extensions. Same for professional or enterprise devs: how much additional effort do I need to take, to make it secure? This does not prevent developpers from providing bad code (as we have seen with many failing exchanges), due to financial constraints.
so we do agree here it seems... ok, enough ranting around, let's go to work! It is better to start coding, and then improve, as to buid an entry barrier that would make people scary and avoid diving into the crypto space. So remains only my personal view, and is by no means inetnded to be offensive to anyone... well before you said read before working i think, but as i have said, im already working. im just at a stand still until i decide how to move forward. write my own wallet over a 6 month span, or find an API/RPC/library. |
|
|
|
|
thanks for your answer about wallet functionality. however be careful, many will instantly dismiss you as a "pro" the second you bash c#/windows as a viable web application setup. wamp is a hugely popular setup, and C# is also very popular and a great php alternative for a lower level programming language if needed. but these are opinions and in no way reflect my, or your ability to make a viable piece of software.
I said i was going to look through the SOURCE of bitcoin qt, the actual source to see how they are doing everything and how it communicates with the network, not the source to learn how to use bitcoind and how to communicate using the json rpc, i can solve that with google if i needed to, i think you misunderstood. i am very familiar with how to use the bitcoind json rpc and i am well versed enough in c to read the source and understand it as it is open source, its just a lot to go through to find how they sign transactions and broadcast them and what protocols they are using.
so before i jump the gun here, can you explain what you mean by code = doc? i dont understand what that is in reference to. it sounds to me like you interpreted that i had an issue understanding bitcoind, and i claimed i would look at the source to figure out how to use it... as i explained already, that is not the case.
and laziness i must disagree with. im asking for pointers on what i should start digging in to, ive been reading for a decade straight, 90% of my job is reading and i am asking for more stuff to read. however, i have more experience with business than bitcoin based programming at this point, i have developed projects that most likely you or somebody you are close to has taken advantage of. but i agree, a team does not mean i have experience. But i am very experienced in many aspects of programming, and i dont beleive lacking knowledge in one field is deserving of a public shaming due to a misunderstanding of the term documentation and source (which i never did confuse to be clear).
after months of researching and learning how to accomplish what i needed, i find all i have learned is no longer viable because the project's scope changed. I also made it very clear i have minimal experience on this level aside from high level APIs/RPCs/libraries (like bitcoind). my reason to give my background was to show i am not looking to be spoon fed info, that i can compensate whoever is willing to help and as an introduction because it is my first post. if i was a 13 year old kid learning to program as a hobby, i would say that in my first post, instead i own a few tech related companies so that is what i put in my first post, maybe i will meet somebody of value here, maybe not.
Also i agree with your comment on linux, its a more secure OS out of the box, it allows more control to the user, it is less liekly to be hacked however a poorly managed linux server is far worse than a poorly managed windows server. and a properly managed windows server is just as capable as you have come to expect from linux/unix distros. however PHP is in no way a better language than c# for MY project (or in general, they are just different, PHP is a scripting language) if its better for the bitcoin side of things, i will use it basedo n the opinions of experienced forum members and my own research, but i doubt that as well. you did not say php though, i am just assuming based on your past posts, maybe you mean python/java. im expecting python/cpp/c# to be a good fit however. i chose c# as this project is very large, its not 100% bitcoin related either, i had other obstacles to overcome.
plus, any real developer can secure a windows server, it takes work but same applies to securing a linux distro.
I have began coding, and ive made a lot of progress, i feel the wording is fine. bitcoind is not capable of managing enterprise level softwarre, i understand a lot about bitcoind, i have read the docs and im not looking for help on that.
|
|
|
|
|
Hello to everybody,
TL;DR: i believe bitcoind is not capable of creating multiple/individualized wallets, how can i do that without third parties using only C#/C++/PHP/maybe python? are there any libraries that may be helpful? or is it easy enough to communicate directly with the nodes and write my own client? (in a perfect world, i want an api that is not owned by a third party like blockchain.info or coinbase, and id like a library that will not require jumping languages and using HTTP communication or any form of request/response type interactions)
------ My issue/goals/questions: -------
I have created past relatively well known financial services in the past, however i used higher level APIs to achieve my goals. i have a large team working for me but i have run in to a major problem on a recent project we are getting in to. The problem is, we dont know what we are doing, and if i dont understand something i cant teach the team.
So, i need to create a web application that has bitcoin functionality, of course, my requirements are
- low level transaction broadcasting/signing. bitcoind-RPC works for this with raw transaction broadcasting.
- Multi-sig support, 2 of 2 or 2 of 3. bitcoind has only partial implementation
- unlimited wallet creation, not accounts like bitcoind, but actual wallets with individual encryption, primarily for individualized multi-sig transactions/wallets. bitcoind is crippled on this front
- wallet encryption
-potentially the ability to generate predictable addresses using a seed (not required, may be too advanced to start with, but if i figure out the bitcoin system, this should be a piece of cake if api limitations are not an issue)
so as bitcoind is useless to me (I THINK. correct me if i can expand functionality somehow), i cant think of or find a lower level API besides some experimental c++ one. how did all these other sites do it? like blockchain.info that stores countless individualized wallets? how about coinbase? i doubt they used bitcoind but im also doubting that they actually do it 100% raw. I in no way intend to develop anything as intensive as an exchange, but i do need similar functionality where user security is the only priority, and bitcoind/shared wallets are out of the question.
I am currently developing in C# with asp.net, i would love to keep it all in c#, but i can very easily switch to php, i chose c# for security purposes. i will not do java unless it is the only option. i also am willing to write a server for executing the commands in c++, but i feel c++ may be too low level for such a networking intensive application and will triple the work load. and im sure if c++ can do it, so can c# since i dont expect a ton of low level requirements like memory allocations and byte manipulation.
------ if a 'ground-up' approach IS necessary: --------
So, if i do need to start doing this all without an api, where do i start? im sure i can dig through qt's source but wow... there is a LOT to look through. and ift his is the case, how are wallets created? are they even a thing or is it all a matter of just storing public and private keys, decrypting, deserializing responses from the nodes, and handling however i like?
what IS a wallet? does the blockchain/nodes/miners/etc... even recognize wallets or is it a construct we use for organizing the addresses like an account?
is there an API for communicating with the nodes? how does something actually get sent to the blockchain? simple post request/response over http? im lost here.
----- In conclusion: ------
So i need some help, and i hope one of you geniuses can assist me. if somebody can prove they are knowledgeable, i am more than happy to pay handsomely for your time and expertise. my goal here is more so to learn than it is to get the project done. i know i could hire somebody with the experience, but in my opinion, if i cant write and understand the code myself, im unfit to even oversee the project let alone be called its owner.
any insight will be greatly appreciated.
|
|
|
|
|
