|
ErebusBat
|
 |
August 30, 2012, 07:33:40 PM |
|
That's true. The thing is though, that I imagine IP connectivity opens up a lot more potential attack surfaces than a USB connection. I mean, any program on the key box listening for incoming connections will suddenly be an attack surface. And it's really hard to control which programs do this; a lot will. With the device acting as a USB client this won't be an issue. You can't - not as far as I know at least - "listen" for connections via USB, and so, there will only be one central point - one attack surface - that we need to secure, which is the USB driver and the program that communicates over this connection. If this is secure, the key box should be secure.
I see you point, but a slim install and a firewall would be all you need. A lockdown of 'only allow connection to port XYZ from host ABC' would be sufficnt. |
|
|
|
|
|
|
|
|
|
|
The grue lurks in the darkest places of the earth. Its favorite diet is adventurers, but its insatiable appetite is tempered by its fear of light. No grue has ever been seen by the light of day, and few have survived its fearsome jaws to tell the tale.
|
|
|
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
|
|
|
|
|
|
mav (OP)
|
|
September 17, 2012, 04:52:21 AM |
|
The simple Armory Daemon is now on github. It aims to be as close to a direct replacement for the Satoshi interface as possible. https://github.com/thedawnrider/BitcoinArmory-DaemonAvailable json-rpc calls are based on the Satoshi rpc interface and are as follows: getbalance getnewaddress getreceivedbyaddress sendtoaddress getbalance: Returns a decimal value in BTC for the total remaining balance in the wallet. getnewaddress: Returns the next address in the wallet as a string. getreceivedbyaddress: Returns a decimal value in BTC for the amount received by the address. sendtoaddress: Returns an unsigned transaction as a string. Implementation of signing and broadcasting is left to the client. Features that may be included in the future: User authentication SSL More API methods as required (the current set fills my needs). listtransactions is one method I am looking to include in the future. Please suggest anything else you personally may find useful, as I have designed this only to fill my specific needs. Clarifications required: - Are incoming transactions which eventually do not become part of the blockchain dealt with correctly? I am unsure how / unable to test this. - What happens if multiple transactions are made which would result in the balance being less than zero? Presumably when it comes to broadcasting the last of the signed transactions there will be an error from the Armory client which is performing the signing/broadcasting. Should tracking of the likely future balance in the daemon be enforced even though the transactions have not yet been broadcast and maybe never will be? How should this be managed, if at all? |
|
|
|
|
etotheipi
Legendary
 Offline
Activity: 1428
Merit: 1093
Core Armory Developer
|
|
September 17, 2012, 05:09:43 AM |
|
Clarifications required:
- Are incoming transactions which eventually do not become part of the blockchain dealt with correctly? I am unsure how / unable to test this.
- What happens if multiple transactions are made which would result in the balance being less than zero? Presumably when it comes to broadcasting the last of the signed transactions there will be an error from the Armory client which is performing the signing/broadcasting. Should tracking of the likely future balance in the daemon be enforced even though the transactions have not yet been broadcast and maybe never will be? How should this be managed, if at all?
- There is full re-org/double-spend handling, which has been tested at the library level. But it hasn't been tested at the GUI/interface level, because I never set up a way to test block/tx injection over the networking interface. I see re-orgs happen all the time -- you'll see output to the console that says "Invalidating old chain tx -- Marking new chain tx valid". For web-servers, this should be all you need. - Armory is pretty dumb when it comes to... lots of network-stuff. It was written with the assumption that Bitcoin-Qt is going to feed it trustworthy, reasonable data. I have noticed, under strange testing conditions, if a conflicting tx happens to make it in, it will show up in the ledger but it actually won't affect your balance at all. Again, this is not all that relevant for web-servers, but it's worth noting: Armory receives the conflicting tx, and immediately detects it's relevant, and adds it to the ledger on the GUI. Then Armory tries to add it to it's pool of tx for each address, etc, but finds those outputs have already been spent. So it gets dropped, and the underlying UTXO list is untouched. It rarely ever matters, because Bitcoin-Qt/bitcoind can't pass it conflicting tx... So I need to fix the auto-display-in-ledger bug. But balances and spendability should maintain correctness in this situation. Please let me know if they appear not to. As for storing data between loads: some people complain about the load times, etc, but it comes with a tremendous benefit of robustness, because there's no way for Armory to ever be "out of sync" with the blockchain. One issue with Bitcoin-Qt is that it can mark outputs as spent, or believe that some transaction succeeded that wasn't accepted and get stuck in a state that isn't consistent with the rest of the network. Armory avoids all this by having no memory. At the expense of load times. This functionality will eventually have to be added to do this (especially with the blockchain size getting so big), but right now it's not there, and it will probably require a lot of testing. If the server is going to process lots and lots of transactions separately, you might consider taking my examples above for creating transactions, and add that memory/tracking around it (maintain the tracking yourself). Maybe that's too much work... I don't know.
Btw, I haven't looked at your stuff yet, but this sounds fantastic! Thanks so much for doing this, and the JSON-RPC interface is something I've wanted to do for a while. Perhaps we'll find a way to merge functionality together... |
|
|
|
|
Herbert
|
|
September 17, 2012, 12:09:52 PM |
|
The simple Armory Daemon is now on github. It aims to be as close to a direct replacement for the Satoshi interface as possible. https://github.com/thedawnrider/BitcoinArmory-DaemonAvailable json-rpc calls are based on the Satoshi rpc interface and are as follows: getbalance getnewaddress getreceivedbyaddress sendtoaddress This looks very interesting. Combining this with bitcoinmonitor.net url callbacks (using the API) will provide you a complete solution to accept payments without the need to put your hot wallet online anywhere and without the hassle to set up/maintain a long list of pre-generated addresses with your bitcoinmonitor.net agent. Awesome!  As soon as I find some time I will set this scenario up for one of my sites. |
www.bitcoinmonitor.net - Free payment notification via email, newsfeed, xpmm/jabber, url callback and full API access! Send SMS with www.txt4coins.net! No registration, pay-per-use, full API access, bulk messages - All inclusive!
|
|
|
|
mav (OP)
|
|
September 18, 2012, 06:06:06 AM |
|
The simple Armory Daemon is now on github. It aims to be as close to a direct replacement for the Satoshi interface as possible. https://github.com/thedawnrider/BitcoinArmory-DaemonAvailable json-rpc calls are based on the Satoshi rpc interface and are as follows: getbalance getnewaddress getreceivedbyaddress sendtoaddress This looks very interesting. Combining this with bitcoinmonitor.net url callbacks (using the API) will provide you a complete solution to accept payments without the need to put your hot wallet online anywhere and without the hassle to set up/maintain a long list of pre-generated addresses with your bitcoinmonitor.net agent. Awesome! As soon as I find some time I will set this scenario up for one of my sites. The api for everything except getnewaddress depends on loading the blockchain. If you stripped it right down to just that one call, then yes it would be great when combined with bitcoinmonitor.net, and as you say, it's not even required to have bitcoin running. But for any functionality beyond getting new addresses from the wallet, you will still need to have bitcoin running so there's access to the blockchain. |
|
|
|
|
Morranr
Newbie
Offline
Activity: 17
Merit: 0
|
|
September 22, 2012, 05:33:29 AM |
|
Just thought to check this thread again, and that daemon looks like it could be extremely useful. Thanks for working on this and putting it out for the public to use! I was pointed to this thread by etothepi when I sent an email asking about generating addresses for my server to use, and ended up deciding that the best implementation for my needs was to generate chunks of addresses at a time with the method described in the first post, and putting them into a redis list (acting as a queue) which would get refilled when it dropped below a certain amount of addresses. I'll probably stick with my current implementation as it is more than sufficient for what I need right now, but I will certainly keep an eye on this daemon and might decide to switch to it later on.  |
|
|
|
|
|
unclescrooge
|
|
October 09, 2012, 03:02:56 PM |
|
Hi mav, You daemon is very very useful. However I haven't been able to make any function beside getNextUnusedAddress work. getbalance() return 0 even though there's one adress with 1 btc on it, listransactions doesn't return anything (but seems like the function isn't developped yet in your code) and getreceivedbyaddress raises an unknown error. Did you manage to make it work? Any way I'm not a python programmer but I'll try to debug that and post what I found, if any unclescrooge |
|
|
|
|
|
mav (OP)
|
|
October 16, 2012, 10:04:21 AM |
|
Hi mav, You daemon is very very useful. However I haven't been able to make any function beside getNextUnusedAddress work. getbalance() return 0 even though there's one adress with 1 btc on it, listransactions doesn't return anything (but seems like the function isn't developped yet in your code) and getreceivedbyaddress raises an unknown error. Did you manage to make it work? Any way I'm not a python programmer but I'll try to debug that and post what I found, if any unclescrooge Sorry for the slow response, I am subscribed to this thread but very rarely come on this forum any more so have not been updated on the new posts. I recently started at a new job which has sadly diverted my attention away from bitcoin dev. I will be looking at this code on the weekend. I made some changes a while ago but never pushed them to git. By the end of the weekend check back on this thread, there should be updates. It's been a while since I've looked at this daemon, and certainly have not been using it in production on any of my systems. But from memory I also could only get getNextUnusedAddress to work properly, although the other calls did something slightly related to the required task, just not something completely useful yet. The end of the weekend - I am sure some updates will be ready. |
|
|
|
|
|
unclescrooge
|
|
October 16, 2012, 10:16:33 AM |
|
Thanks for the response mav. Although I never touched python in my life nor am I a programmer, I added some codes from ArmoryQt and manage to get getbalance and listtransactions working (for this last only a few items are returned and some are wrong so I need to work more on that). Here is yur modified code so far, ugly as hell lol: ################################################################################
#
# Copyright (C) 2012, Ian Coleman
# Distributed under the GNU Affero General Public License (AGPL v3)
# See http://www.gnu.org/licenses/agpl.html
#
################################################################################
from twisted.web import server
from twisted.internet import reactor
from txjsonrpc.web import jsonrpc
import decimal
import os
RPC_PORT = 7070
STANDARD_FEE = 0.0005 # BTC
class Wallet_Json_Rpc_Server(jsonrpc.JSONRPC):
def __init__(self, wallet):
self.wallet = wallet
def jsonrpc_getnewaddress(self):
addr = self.wallet.getNextUnusedAddress()
return addr.getAddrStr()
def jsonrpc_getbalance(self):
int_balance = self.wallet.getBalance()
decimal_balance = decimal.Decimal(int_balance) / decimal.Decimal(ONE_BTC)
return float(decimal_balance)
def jsonrpc_getreceivedbyaddress(self, address):
if CLI_OPTIONS.offline:
raise ValueError('Cannot get received amount when offline')
# Only gets correct amount for addresses in the wallet, otherwise 0
addr160 = addrStr_to_hash160(address)
txs = self.wallet.getAddrTxLedger(addr160)
balance = sum([x.getValue() for x in txs if x.getValue() > 0])
decimal_balance = decimal.Decimal(balance) / decimal.Decimal(ONE_BTC)
float_balance = float(decimal_balance)
return float_balance
def jsonrpc_sendtoaddress(self, bitcoinaddress, amount):
if CLI_OPTIONS.offline:
raise ValueError('Cannot create transactions when offline')
return self.create_unsigned_transaction(bitcoinaddress, amount)
def jsonrpc_listtransactions(self, p_count=10, p_from=0):
#TODO this needs more work
result2 = []
txs = self.wallet.getTxLedger('blk')
#txs = json.dumps(txs)
#id_le_pairs = [[le] for le in txs]
for x in txs:
#result.append('{')
account = ''
txHashBin = x.getTxHash()#hex_to_binary(x.getTxHash())
cppTx = TheBDM.getTxByHash(txHashBin)
pytx = PyTx().unserialize(cppTx.serialize())
for txout in pytx.outputs:
scrType = getTxOutScriptType(txout.binScript)
if not scrType in (TXOUT_SCRIPT_STANDARD, TXOUT_SCRIPT_COINBASE):
continue
address = hash160_to_addrStr(TxOutScriptExtractAddr160(txout.binScript))
if x.getValue() < 0:
category = 'send'
else:
category = 'receive'
amount = x.getValue()
confirmations = TheBDM.getTopBlockHeader().getBlockHeight() - x.getBlockNum()+1
blockhash = ''
blockindex = ''#x.getBlockNum()
txid = str(binary_to_hex(x.getTxHash()))
time = ''#x.getTxTime()
result = {'account':account,'address':address,'category':category,'amount':amount,'confirmations':confirmations,'blockhash':blockhash,'blockindex':blockindex,'txid':txid,'time:':time}
#result.append('}')
result2.append(result)
#print addr
#dumpObj(x)
#result = json.dumps(result)
return result2
# https://bitcointalk.org/index.php?topic=92496.msg1126310#msg1126310
def create_unsigned_transaction(self, bitcoinaddress_str, amount_to_send_btc):
# Get unspent TxOutList and select the coins
addr160_recipient = addrStr_to_hash160(bitcoinaddress_str)
totalSend, fee = long(amount_to_send_btc * ONE_BTC), (STANDARD_FEE * ONE_BTC)
spendBal = self.wallet.getBalance('Spendable')
utxoList = self.wallet.getTxOutList('Spendable')
utxoSelect = PySelectCoins(utxoList, totalSend, fee)
minFeeRec = calcMinSuggestedFees(utxoSelect, totalSend, fee)[1]
if fee<minFeeRec:
if totalSend + minFeeRec > spendBal:
raise NotEnoughCoinsError, "You can't afford the fee!"
utxoSelect = PySelectCoins(utxoList, totalSend, minFeeRec)
fee = minFeeRec
if len(utxoSelect)==0:
raise CoinSelectError, "Somehow, coin selection failed. This shouldn't happen"
totalSelected = sum([u.getValue() for u in utxoSelect])
totalChange = totalSelected - (totalSend + fee)
outputPairs = []
outputPairs.append( [addr160_recipient, totalSend] )
if totalChange > 0:
outputPairs.append( [self.wallet.getNextUnusedAddress().getAddr160(), totalChange] )
random.shuffle(outputPairs)
txdp = PyTxDistProposal().createFromTxOutSelection(utxoSelect, outputPairs)
return txdp.serializeAscii()
def jsonrpc_createCombinedLedger(self, wltIDList=None, withZeroConf=True):
"""
Create a ledger to display on the main screen, that consists of ledger
entries of any SUBSET of available wallets.
"""
start = RightNow()
self.combinedLedger = []
totalFunds = 0
spendFunds = 0
unconfFunds = 0
currBlk = 0xffffffff
if TheBDM.isInitialized():
currBlk = TheBDM.getTopBlockHeader().getBlockHeight()
wlt = self.waldlet
totalFunds += wlt.getBalance('Total')
spendFunds += wlt.getBalance('Spendable')
unconfFunds += wlt.getBalance('Unconfirmed')
self.combinedLedger.sort(key=lambda x: x[LEDGERCOLS.UnixTime], reverse=True)
self.ledgerSize = len(self.combinedLedger)
print totalFunds
return
class Armory_Daemon():
def __init__(self):
print "Reading wallet file"
self.wallet = self.find_wallet()
self.loadBlockchain()
use_blockchain = not CLI_OPTIONS.offline
print "Initialising server"
reactor.listenTCP(RPC_PORT, server.Site(Wallet_Json_Rpc_Server(self.wallet)))
self.NetworkingFactory = ArmoryClientFactory( \
func_loseConnect=self.showOfflineMsg, \
func_madeConnect=self.showOnlineMsg, \
func_newTx=self.newTxFunc)
reactor.connectTCP('127.0.0.1', BITCOIN_PORT, self.NetworkingFactory)
reactor.callLater(5, self.Heartbeat)
self.start()
def start(self):
print "Server started"
reactor.run()
def newTxFunc(self, pytxObj):
# Cut down version from ArmoryQt.py
TheBDM.addNewZeroConfTx(pytxObj.serialize(), long(RightNow()), True)
TheBDM.rescanWalletZeroConf(self.wallet.cppWallet)
# TODO set up a 'subscribe' feature so these notifications can be
# pushed out to interested parties.
# From here down is display purposes only, copied from ArmoryQt.py
message = "New TX"
le = self.wallet.cppWallet.calcLedgerEntryForTxStr(pytxObj.serialize())
if not le.isSentToSelf():
txref = TheBDM.getTxByHash(le.getTxHash())
nOut = txref.getNumTxOut()
recips = [txref.getTxOut(i).getRecipientAddr() for i in range(nOut)]
values = [txref.getTxOut(i).getValue() for i in range(nOut)]
idxMine = filter(lambda i: self.wallet.hasAddr(recips[i]), range(nOut))
idxOther = filter(lambda i: not self.wallet.hasAddr(recips[i]), range(nOut))
mine = [(recips[i],values[i]) for i in idxMine]
other = [(recips[i],values[i]) for i in idxOther]
# Collected everything we need to display, now construct it and do it
if le.getValue()>0:
# Received!
message = 'Bitcoins Received!'
totalStr = coin2str( sum([mine[i][1] for i in range(len(mine))]), maxZeros=1)
message += '\nAmount: \t%s BTC' % totalStr.strip()
if len(mine)==1:
message += '\nAddress:\t%s' % hash160_to_addrStr(mine[0][0])
addrComment = self.wallet.getComment(mine[0][0])
#if addrComment:
#message += '\n%s...' % addrComment[:24]
else:
message += '\n<Received with Multiple Addresses>'
elif le.getValue()<0:
# Sent!
message = 'Bitcoins Sent!'
totalStr = coin2str( sum([other[i][1] for i in range(len(other))]), maxZeros=1)
message += '\nAmount: \t%s BTC' % totalStr.strip()
if len(other)==1:
message += 'Sent To:\t%s' % hash160_to_addrStr(other[0][0])
addrComment = self.wallet.getComment(other[0][0])
#if addrComment:
#message += '\n%s...' % addrComment[:24]
else:
dispLines.append('<Sent to Multiple Addresses>')
else:
amt = self.determineSentToSelfAmt(le, self.wallet)[0]
message = 'Wallet "%s" just sent %s BTC to itself!' % \
(self.wallet.labelName, coin2str(amt,maxZeros=1).strip())
print message
def determineSentToSelfAmt(self, le, wlt):
"""
NOTE: this method works ONLY because we always generate a new address
whenever creating a change-output, which means it must have a
higher chainIndex than all other addresses. If you did something
creative with this tx, this may not actually work.
"""
amt = 0
if TheBDM.isInitialized() and le.isSentToSelf():
txref = TheBDM.getTxByHash(le.getTxHash())
if not txref.isInitialized():
return (0, 0)
if txref.getNumTxOut()==1:
return (txref.getTxOut(0).getValue(), -1)
maxChainIndex = -5
txOutChangeVal = 0
txOutIndex = -1
valSum = 0
for i in range(txref.getNumTxOut()):
valSum += txref.getTxOut(i).getValue()
addr160 = txref.getTxOut(i).getRecipientAddr()
addr = wlt.getAddrByHash160(addr160)
if addr and addr.chainIndex > maxChainIndex:
maxChainIndex = addr.chainIndex
txOutChangeVal = txref.getTxOut(i).getValue()
txOutIndex = i
amt = valSum - txOutChangeVal
return (amt, txOutIndex)
def showOfflineMsg(self):
print "Offline - not tracking blockchain"
def showOnlineMsg(self):
print "Online - tracking blockchain"
def find_wallet(self):
fnames = os.listdir(os.getcwd())
for fname in fnames:
is_wallet = fname[-7:] == ".wallet"
is_watchonly = fname.find("watchonly") > -1
is_backup = fname.find("backup") > -1
if(is_wallet and is_watchonly and not is_backup):
wallet = PyBtcWallet().readWalletFile(fname)
# Register all wallets with TheBDM
TheBDM.registerWallet( wallet.cppWallet )
print "Using wallet file %s" % fname
return wallet
raise ValueError('Unable to locate a watch-only wallet in %s' % os.getcwd())
def loadBlockchain(self):
print "Loading blockchain"
BDM_LoadBlockchainFile()
self.latestBlockNum = TheBDM.getTopBlockHeader().getBlockHeight()
# Now that theb blockchain is loaded, let's populate the wallet info
if TheBDM.isInitialized():
mempoolfile = os.path.join(ARMORY_HOME_DIR,'mempool.bin')
self.checkMemoryPoolCorruption(mempoolfile)
TheBDM.enableZeroConf(mempoolfile)
# self.statusBar().showMessage('Syncing wallets with blockchain...')
print "Syncing wallets with blockchain..."
print "Syncing wallet: ", self.wallet.uniqueIDB58
self.wallet.setBlockchainSyncFlag(BLOCKCHAIN_READONLY)
self.wallet.syncWithBlockchain()
#self.createCombinedLedger()
#self.ledgerSize = len(self.combinedLedger)
#self.statusBar().showMessage('Blockchain loaded, wallets sync\'d!', 10000)
def checkMemoryPoolCorruption(self, mempoolname):
if not os.path.exists(mempoolname):
return
memfile = open(mempoolname, 'r')
memdata = memfile.read()
memfile.close()
binunpacker = BinaryUnpacker(memdata)
try:
while binunpacker.getRemainingSize() > 0:
binunpacker.get(UINT64)
PyTx().unserialize(binunpacker)
except:
os.remove(mempoolname);
LOGWARN('Memory pool file was corrupt. Deleted. (no further action is needed)')
def Heartbeat(self, nextBeatSec=2):
"""
This method is invoked when the app is initialized, and will
run every 2 seconds, or whatever is specified in the nextBeatSec
argument.
"""
# Check for new blocks in the blk0001.dat file
if TheBDM.isInitialized():
newBlks = TheBDM.readBlkFileUpdate()
self.topTimestamp = TheBDM.getTopBlockHeader().getTimestamp()
if newBlks>0:
self.latestBlockNum = TheBDM.getTopBlockHeader().getBlockHeight()
LOGINFO('New Block! : %d', self.latestBlockNum)
didAffectUs = False
prevLedgerSize = len(self.wallet.getTxLedger())
self.wallet.syncWithBlockchain()
TheBDM.rescanWalletZeroConf(self.cppWallet)
self.wallet.checkWalletLockTimeout()
reactor.callLater(nextBeatSec, self.Heartbeat)
## {{{ http://code.activestate.com/recipes/137951/ (r1)
def printDict(di, format="%-25s %s"):
for (key, val) in di.items():
print format % (str(key)+':', val)
def delchars(str, chars):
"""Returns a string for which all occurrences of characters in
chars have been removed."""
# Translate demands a mapping string of 256 characters;
# whip up a string that will leave all characters unmolested.
identity = ''.join([chr(x) for x in range(256)])
return str.translate(identity, chars)
## end of http://code.activestate.com/recipes/137951/ }}}
if __name__ == "__main__":
from armoryengine import *
rpc_server = Armory_Daemon()
|
|
|
|
|
|
unclescrooge
|
|
October 19, 2012, 10:15:48 AM |
|
Ok so I updated the previous code which didn't work at all. Now I manage to get working "getbalance()", "listtransactions()" (only a few items that I think important are returned though), "sendtoaddress()" (create a file containing the unsigned transaction in the current directory and return a random string). Do not use in production server without testing though: ################################################################################
#
# Copyright (C) 2012, Ian Coleman
# Distributed under the GNU Affero General Public License (AGPL v3)
# See http://www.gnu.org/licenses/agpl.html
#
################################################################################
from twisted.web import server
from twisted.internet import reactor
from txjsonrpc.web import jsonrpc
import decimal
import os
RPC_PORT = 7070
STANDARD_FEE = 0.0005 # BTC
class Wallet_Json_Rpc_Server(jsonrpc.JSONRPC):
def __init__(self, wallet):
self.wallet = wallet
def jsonrpc_getnewaddress(self):
addr = self.wallet.getNextUnusedAddress()
return addr.getAddrStr()
def jsonrpc_getbalance(self):
int_balance = self.wallet.getBalance()
decimal_balance = decimal.Decimal(int_balance) / decimal.Decimal(ONE_BTC)
return float(decimal_balance)
def jsonrpc_getreceivedbyaddress(self, address):
if CLI_OPTIONS.offline:
raise ValueError('Cannot get received amount when offline')
# Only gets correct amount for addresses in the wallet, otherwise 0
addr160 = addrStr_to_hash160(address)
txs = self.wallet.getAddrTxLedger(addr160)
balance = sum([x.getValue() for x in txs if x.getValue() > 0])
decimal_balance = decimal.Decimal(balance) / decimal.Decimal(ONE_BTC)
float_balance = float(decimal_balance)
return float_balance
def jsonrpc_sendtoaddress(self, bitcoinaddress, amount):
if CLI_OPTIONS.offline:
raise ValueError('Cannot create transactions when offline')
return self.create_unsigned_transaction(bitcoinaddress, amount)
def jsonrpc_listtransactions(self, p_count=10, p_from=0):
#TODO this needs more work
result2 = []
txs = self.wallet.getTxLedger('blk')
#txs = json.dumps(txs)
#id_le_pairs = [[le] for le in txs]
for x in txs:
#result.append('{')
account = ''
txHashBin = x.getTxHash()#hex_to_binary(x.getTxHash())
cppTx = TheBDM.getTxByHash(txHashBin)
pytx = PyTx().unserialize(cppTx.serialize())
for txout in pytx.outputs:
scrType = getTxOutScriptType(txout.binScript)
if not scrType in (TXOUT_SCRIPT_STANDARD, TXOUT_SCRIPT_COINBASE):
continue
address = hash160_to_addrStr(TxOutScriptExtractAddr160(txout.binScript))
#wltID = self.wallet.getWalletForAddr160(a160)
if self.wallet.hasAddr(address) == False:
continue
else:
break
if x.getValue() < 0:
category = 'send'
else:
category = 'receive'
amount = float(decimal.Decimal(x.getValue()) / decimal.Decimal(ONE_BTC))
confirmations = TheBDM.getTopBlockHeader().getBlockHeight() - x.getBlockNum()+1
blockhash = ''
blockindex = ''#x.getBlockNum()
txid = str(binary_to_hex(x.getTxHash()))
time = ''#x.getTxTime()
result = {'account':account,'address':address,'category':category,'amount':amount,'confirmations':confirmations,'blockhash':blockhash,'blockindex':blockindex,'txid':txid,'time:':time}
#result.append('}')
result2.append(result)
#print addr
#dumpObj(x)
#result = json.dumps(result)
return result2
# https://bitcointalk.org/index.php?topic=92496.msg1126310#msg1126310
def create_unsigned_transaction(self, bitcoinaddress_str, amount_to_send_btc):
# Get unspent TxOutList and select the coins
addr160_recipient = addrStr_to_hash160(bitcoinaddress_str)
totalSend, fee = long(amount_to_send_btc * ONE_BTC), (STANDARD_FEE * ONE_BTC)
spendBal = self.wallet.getBalance('Spendable')
utxoList = self.wallet.getTxOutList('Spendable')
utxoSelect = PySelectCoins(utxoList, totalSend, fee)
minFeeRec = calcMinSuggestedFees(utxoSelect, totalSend, fee)[1]
if fee<minFeeRec:
if totalSend + minFeeRec > spendBal:
raise NotEnoughCoinsError, "You can't afford the fee!"
utxoSelect = PySelectCoins(utxoList, totalSend, minFeeRec)
fee = minFeeRec
if len(utxoSelect)==0:
raise CoinSelectError, "Somehow, coin selection failed. This shouldn't happen"
totalSelected = sum([u.getValue() for u in utxoSelect])
totalChange = totalSelected - (totalSend + fee)
outputPairs = []
outputPairs.append( [addr160_recipient, totalSend] )
if totalChange > 0:
outputPairs.append( [self.wallet.getNextUnusedAddress().getAddr160(), totalChange] )
random.shuffle(outputPairs)
txdp = PyTxDistProposal().createFromTxOutSelection(utxoSelect, outputPairs)
outfilename = str(int(time.time())) + '.unsigned.tx'
output = open(outfilename, 'w')
output.write(txdp.serializeAscii())
output.close()
result2 = []
result = {"Code":""}
result2.append(result)
return "success"+str(int(time.time()))
def jsonrpc_createCombinedLedger(self, wltIDList=None, withZeroConf=True):
"""
Create a ledger to display on the main screen, that consists of ledger
entries of any SUBSET of available wallets.
"""
start = RightNow()
self.combinedLedger = []
totalFunds = 0
spendFunds = 0
unconfFunds = 0
currBlk = 0xffffffff
if TheBDM.isInitialized():
currBlk = TheBDM.getTopBlockHeader().getBlockHeight()
wlt = self.waldlet
totalFunds += wlt.getBalance('Total')
spendFunds += wlt.getBalance('Spendable')
unconfFunds += wlt.getBalance('Unconfirmed')
self.combinedLedger.sort(key=lambda x: x[LEDGERCOLS.UnixTime], reverse=True)
self.ledgerSize = len(self.combinedLedger)
print totalFunds
return
class Armory_Daemon():
def __init__(self):
print "Reading wallet file"
self.wallet = self.find_wallet()
self.loadBlockchain()
use_blockchain = not CLI_OPTIONS.offline
print "Initialising server"
reactor.listenTCP(RPC_PORT, server.Site(Wallet_Json_Rpc_Server(self.wallet)))
self.NetworkingFactory = ArmoryClientFactory( \
func_loseConnect=self.showOfflineMsg, \
func_madeConnect=self.showOnlineMsg, \
func_newTx=self.newTxFunc)
reactor.connectTCP('127.0.0.1', BITCOIN_PORT, self.NetworkingFactory)
reactor.callLater(5, self.Heartbeat)
self.start()
def start(self):
print "Server started"
reactor.run()
def newTxFunc(self, pytxObj):
# Cut down version from ArmoryQt.py
TheBDM.addNewZeroConfTx(pytxObj.serialize(), long(RightNow()), True)
TheBDM.rescanWalletZeroConf(self.wallet.cppWallet)
# TODO set up a 'subscribe' feature so these notifications can be
# pushed out to interested parties.
# From here down is display purposes only, copied from ArmoryQt.py
message = "New TX"
le = self.wallet.cppWallet.calcLedgerEntryForTxStr(pytxObj.serialize())
if not le.isSentToSelf():
txref = TheBDM.getTxByHash(le.getTxHash())
nOut = txref.getNumTxOut()
recips = [txref.getTxOut(i).getRecipientAddr() for i in range(nOut)]
values = [txref.getTxOut(i).getValue() for i in range(nOut)]
idxMine = filter(lambda i: self.wallet.hasAddr(recips[i]), range(nOut))
idxOther = filter(lambda i: not self.wallet.hasAddr(recips[i]), range(nOut))
mine = [(recips[i],values[i]) for i in idxMine]
other = [(recips[i],values[i]) for i in idxOther]
# Collected everything we need to display, now construct it and do it
if le.getValue()>0:
# Received!
message = 'Bitcoins Received!'
totalStr = coin2str( sum([mine[i][1] for i in range(len(mine))]), maxZeros=1)
message += '\nAmount: \t%s BTC' % totalStr.strip()
if len(mine)==1:
message += '\nAddress:\t%s' % hash160_to_addrStr(mine[0][0])
addrComment = self.wallet.getComment(mine[0][0])
#if addrComment:
#message += '\n%s...' % addrComment[:24]
else:
message += '\n<Received with Multiple Addresses>'
elif le.getValue()<0:
# Sent!
message = 'Bitcoins Sent!'
totalStr = coin2str( sum([other[i][1] for i in range(len(other))]), maxZeros=1)
message += '\nAmount: \t%s BTC' % totalStr.strip()
if len(other)==1:
message += 'Sent To:\t%s' % hash160_to_addrStr(other[0][0])
addrComment = self.wallet.getComment(other[0][0])
#if addrComment:
#message += '\n%s...' % addrComment[:24]
else:
dispLines.append('<Sent to Multiple Addresses>')
else:
amt = self.determineSentToSelfAmt(le, self.wallet)[0]
message = 'Wallet "%s" just sent %s BTC to itself!' % \
(self.wallet.labelName, coin2str(amt,maxZeros=1).strip())
print message
def determineSentToSelfAmt(self, le, wlt):
"""
NOTE: this method works ONLY because we always generate a new address
whenever creating a change-output, which means it must have a
higher chainIndex than all other addresses. If you did something
creative with this tx, this may not actually work.
"""
amt = 0
if TheBDM.isInitialized() and le.isSentToSelf():
txref = TheBDM.getTxByHash(le.getTxHash())
if not txref.isInitialized():
return (0, 0)
if txref.getNumTxOut()==1:
return (txref.getTxOut(0).getValue(), -1)
maxChainIndex = -5
txOutChangeVal = 0
txOutIndex = -1
valSum = 0
for i in range(txref.getNumTxOut()):
valSum += txref.getTxOut(i).getValue()
addr160 = txref.getTxOut(i).getRecipientAddr()
addr = wlt.getAddrByHash160(addr160)
if addr and addr.chainIndex > maxChainIndex:
maxChainIndex = addr.chainIndex
txOutChangeVal = txref.getTxOut(i).getValue()
txOutIndex = i
amt = valSum - txOutChangeVal
return (amt, txOutIndex)
def showOfflineMsg(self):
print "Offline - not tracking blockchain"
def showOnlineMsg(self):
print "Online - tracking blockchain"
def find_wallet(self):
fnames = os.listdir(os.getcwd())
for fname in fnames:
is_wallet = fname[-7:] == ".wallet"
is_watchonly = fname.find("watchonly") > -1
is_backup = fname.find("backup") > -1
if(is_wallet and is_watchonly and not is_backup):
wallet = PyBtcWallet().readWalletFile(fname)
# Register all wallets with TheBDM
TheBDM.registerWallet( wallet.cppWallet )
print "Using wallet file %s" % fname
return wallet
raise ValueError('Unable to locate a watch-only wallet in %s' % os.getcwd())
def loadBlockchain(self):
print "Loading blockchain"
BDM_LoadBlockchainFile()
self.latestBlockNum = TheBDM.getTopBlockHeader().getBlockHeight()
# Now that theb blockchain is loaded, let's populate the wallet info
if TheBDM.isInitialized():
mempoolfile = os.path.join(ARMORY_HOME_DIR,'mempool.bin')
self.checkMemoryPoolCorruption(mempoolfile)
TheBDM.enableZeroConf(mempoolfile)
# self.statusBar().showMessage('Syncing wallets with blockchain...')
print "Syncing wallets with blockchain..."
print "Syncing wallet: ", self.wallet.uniqueIDB58
self.wallet.setBlockchainSyncFlag(BLOCKCHAIN_READONLY)
self.wallet.syncWithBlockchain()
#self.createCombinedLedger()
#self.ledgerSize = len(self.combinedLedger)
#self.statusBar().showMessage('Blockchain loaded, wallets sync\'d!', 10000)
def checkMemoryPoolCorruption(self, mempoolname):
if not os.path.exists(mempoolname):
return
memfile = open(mempoolname, 'r')
memdata = memfile.read()
memfile.close()
binunpacker = BinaryUnpacker(memdata)
try:
while binunpacker.getRemainingSize() > 0:
binunpacker.get(UINT64)
PyTx().unserialize(binunpacker)
except:
os.remove(mempoolname);
LOGWARN('Memory pool file was corrupt. Deleted. (no further action is needed)')
def Heartbeat(self, nextBeatSec=2):
"""
This method is invoked when the app is initialized, and will
run every 2 seconds, or whatever is specified in the nextBeatSec
argument.
"""
# Check for new blocks in the blk0001.dat file
if TheBDM.isInitialized():
newBlks = TheBDM.readBlkFileUpdate()
self.topTimestamp = TheBDM.getTopBlockHeader().getTimestamp()
if newBlks>0:
self.latestBlockNum = TheBDM.getTopBlockHeader().getBlockHeight()
didAffectUs = False
prevLedgerSize = len(self.wallet.getTxLedger())
self.wallet.syncWithBlockchain()
TheBDM.rescanWalletZeroConf(self.wallet.cppWallet)
self.wallet.checkWalletLockTimeout()
reactor.callLater(nextBeatSec, self.Heartbeat)
if __name__ == "__main__":
from armoryengine import *
rpc_server = Armory_Daemon()
|
|
|
|
|
|
mav (OP)
|
|
October 21, 2012, 04:25:07 AM |
|
Ok so I updated the previous code which didn't work at all.
...
Massive kudos for making this happen unclescrooge. Armory takes a fair bit of poking around, so it seems you've done well with this. I added the code and have written some basic tests, unfortunately time is unexpectedly short for me this weekend so later in the week I will try adding the missing fields from listtransactions (and properly confirm that the existing fields are correct). See the latest on github for the changes. I appreciate the efforts you have put in unclescrooge. It's nice to know people are looking at this, gives me motivation to continue on it. I was planning to use this in a project I was working on, but that has been put on the backburner and as such armory-daemon has languished. |
|
|
|
|
|
unclescrooge
|
|
October 22, 2012, 08:46:03 AM |
|
Ok so I updated the previous code which didn't work at all.
...
Massive kudos for making this happen unclescrooge. Armory takes a fair bit of poking around, so it seems you've done well with this. I added the code and have written some basic tests, unfortunately time is unexpectedly short for me this weekend so later in the week I will try adding the missing fields from listtransactions (and properly confirm that the existing fields are correct). See the latest on github for the changes. I appreciate the efforts you have put in unclescrooge. It's nice to know people are looking at this, gives me motivation to continue on it. I was planning to use this in a project I was working on, but that has been put on the backburner and as such armory-daemon has languished. Well thank you, you did the most part, and I had an interest in this as I'm launching a new bitcoin service using this script. I hope you tested it before pushing to git, I did but the more testing the better. Thanks for pushing it to github anyway Anyway I'll keep updating the code if I find so way to improve it with. But for now I have to focus on a way to generate an offline TX and push it to an offsite computer to be signed and broadcast immediately. Because if you don't broadcast your unsigned tx immediately, the next generated one should be incorrect :/ |
|
|
|
|
|
mav (OP)
|
|
October 22, 2012, 09:50:42 AM |
|
Anyway I'll keep updating the code if I find so way to improve it with. But for now I have to focus on a way to generate an offline TX and push it to an offsite computer to be signed and broadcast immediately. Because if you don't broadcast your unsigned tx immediately, the next generated one should be incorrect :/
This is an interesting point you make. On one hand it seems fairly certain you are right about this, but maybe etotheipi can confirm whether the wallet can handle the creation of multiple unsigned transactions at a time? It seems like a fairly likely situation to happen under most normal use cases, even with the gui client. If the daemon software can avoid having to implement a strict 'sequential' series of transaction where the server must create-a-tx-then-announce-before-creating-the-next-tx, it would make life much much easier. TBH I always assumed transactions could be grouped and processed offline as a batch. If armory does require one-at-a-time, that would become a very limiting factor for even a moderately popular web-based service. |
|
|
|
|
|
unclescrooge
|
|
October 22, 2012, 11:10:37 AM |
|
Anyway I'll keep updating the code if I find so way to improve it with. But for now I have to focus on a way to generate an offline TX and push it to an offsite computer to be signed and broadcast immediately. Because if you don't broadcast your unsigned tx immediately, the next generated one should be incorrect :/
This is an interesting point you make. On one hand it seems fairly certain you are right about this, but maybe etotheipi can confirm whether the wallet can handle the creation of multiple unsigned transactions at a time? It seems like a fairly likely situation to happen under most normal use cases, even with the gui client. If the daemon software can avoid having to implement a strict 'sequential' series of transaction where the server must create-a-tx-then-announce-before-creating-the-next-tx, it would make life much much easier. TBH I always assumed transactions could be grouped and processed offline as a batch. If armory does require one-at-a-time, that would become a very limiting factor for even a moderately popular web-based service. I tested several send transactions in a row (with same parameters), and for each transactions the inputs and outputs are the same. Which is logical since armory look at the existing adresses balances to construct a transaction, if a transaction is not broadcasted it can't be taken into account. I don't know if etotheipi is gonna change this behavior in the future, but for now you have to broadcast an unsigned transaction before creating the other one, no other way around it. What I'm thinking to get around this limitation is to have a computer, completely behind a firewall, which pull from production servers all unsigned txs when they are created, signed and broadcast them. Then it send a signal to the productions servers (delete the unsigned tx file?) to go ahead and create the next unsigned tx if needed. That's not as secure as offline signing, especially if signing is automated, but I think it's still relatively secure compared to existing solution with bitcoind. |
|
|
|
|
arsenische
Legendary
Offline
Activity: 1199
Merit: 1012
|
|
November 07, 2012, 09:54:05 AM |
|
There won't be any fireworks, but you might end up re-using an address by accident. I am sorry, I am misunderstanding something. I like the idea of deterministic address generation without private keys. But does Armory deterministically separate change addresses from normal ones? Let's say I don't want to use offline transactions. I just want to generate addresses with watching-only wallet (being run on the web server machine) and then spend the received bitcoins from the full wallet (being run on my desktop computer). I like this idea since it doesn't require any synchronization between wallets. But would it be safe, or would Armory generate change addresses that may conflict with addresses generated on server? Is there any chance to make this simple scheme work? |
|
|
|
arsenische
Legendary
Offline
Activity: 1199
Merit: 1012
|
|
November 07, 2012, 02:05:18 PM |
|
But would it be safe, or would Armory generate change addresses that may conflict with addresses generated on server?
I made a simple experiment and noticed that change addresses come from the same sequence. Is there a way around this feature? |
|
|
|
etotheipi
Legendary
Offline
Activity: 1428
Merit: 1093
Core Armory Developer
|
|
November 07, 2012, 04:09:07 PM |
|
You are right, there is no alternate chain for change addresses. It is possible to end up with accidentally re-using an address once, especially if your server is very active in giving out addresses.
However, in version 0.82 I added a new feature to the Expert interface (in the GUI) that lets you specify the change address to use. It allows you to send change back to the first input address (may be better than the alternative), or you can supply your own address. You can't specify another wallet yet, but I want to allow that, eventually.
However, the new wallet format that will come after beta, is based on endless discussion with Pieter Wiulle (bitcoin-qt dev), and that will result in wallets, by default, having two different subchains for each wallet -- the second chain will be used solely for change. The new wallet format is a ways off (hopefully Beta isn't!), but it is coming. For now, it sounds like switching your desktop to Expert usermode is your solution.
|
|
|
|
arsenische
Legendary
Offline
Activity: 1199
Merit: 1012
|
|
November 07, 2012, 07:00:04 PM |
|
You are right, there is no alternate chain for change addresses. It is possible to end up with accidentally re-using an address once, especially if your server is very active in giving out addresses.
However, in version 0.82 I added a new feature to the Expert interface (in the GUI) that lets you specify the change address to use. It allows you to send change back to the first input address (may be better than the alternative), or you can supply your own address. You can't specify another wallet yet, but I want to allow that, eventually.
However, the new wallet format that will come after beta, is based on endless discussion with Pieter Wiulle (bitcoin-qt dev), and that will result in wallets, by default, having two different subchains for each wallet -- the second chain will be used solely for change. The new wallet format is a ways off (hopefully Beta isn't!), but it is coming. For now, it sounds like switching your desktop to Expert usermode is your solution.
Thanks! Great software! |
|
|
|
|
mav (OP)
|
|
January 06, 2013, 03:50:14 AM |
|
I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py Unfortunately I am unable to load the blockchain, and am getting these errors. (ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661) Any ideas why these errors are being thrown? This is happening in TheBDM.run() The daemon still runs and correctly responds to getnewaddress however I haven't got a suitable wallet to test the other calls, namely: getbalance getreceivedbyaddress listtransactions sendtoaddress Please see https://github.com/thedawnrider/BitcoinArmory-Daemon/blob/master/armory-daemon.py for the source |
|
|
|
|
etotheipi
Legendary
Offline
Activity: 1428
Merit: 1093
Core Armory Developer
|
|
January 06, 2013, 04:18:51 AM |
|
I have recently pushed changes to the daemon to bring it up to date with the latest armoryengine.py Unfortunately I am unable to load the blockchain, and am getting these errors. (ERROR) armoryengine.py:11362 - Error processing BDM input
(ERROR) armoryengine.py:11363 - Received inputTuple: StartScanRequested [8, 28474661, False]
(ERROR) armoryengine.py:11364 - Error processing ID (28474661) Any ideas why these errors are being thrown? This is happening in TheBDM.run() The daemon still runs and correctly responds to getnewaddress however I haven't got a suitable wallet to test the other calls, namely: getbalance getreceivedbyaddress listtransactions sendtoaddress Please see https://github.com/thedawnrider/BitcoinArmory-Daemon/blob/master/armory-daemon.py for the source Oh boy. I forgot about this thread, and should've warned you that I converted Armory to a multi-threaded application. As such, some of the interfaces changed a bit. There is an example of blockchain loading in the extras/sample_armory_code.py file, but I'm not sure all the samples are up-to-date for the new multi-threaded code. On the upside, just about any function that doesn't return something (like blockchain scanning) you can use wait=False and it will run in the background allowing you to do other operations while you wait. You'll see what I mean if you look at the blocking vs. non-blocking methods, shown in sample_armory_code.py: ################################################################################
if run_LoadBlockchain_Block:
start = RightNow()
TheBDM.setBlocking(True)
TheBDM.setOnlineMode(True)
# The setOnlineMode should block until blockchain loading is complete
print 'Loading blockchain took %0.1f sec' % (RightNow() - start)
topBlock = TheBDM.getTopBlockHeight()
print '\n\nCurrent Top Block is:', topBlock
TheBDM.getTopBlockHeader().pprint() As above, if you set blocking to True, it should behave much like the original... the main thread always waits for a response from the BDM before continuing. On the other hand, if blocking=False (default): ################################################################################
if run_LoadBlockchain_Async:
start = RightNow()
TheBDM.setBlocking(False)
TheBDM.setOnlineMode(True)
print 'Waiting for blockchain loading to finish',
while not TheBDM.getBDMState()=='BlockchainReady':
print '.',
sleep(2)
# do other stuff while waiting...
print 'Loading blockchain took %0.1f sec' % (RightNow() - start)
topBlock = TheBDM.getTopBlockHeight()
print '\n\nCurrent Top Block is:', topBlock
TheBDM.getTopBlockHeader().pprint() In this case, the blockchain scan happens in the background and the main thread continues running immediately, giving you an opportunity to do other calculations or operations in the main thread. If you use the setBlocking=False, you should ALWAYS check theBDMState() before making a call like getTopBlockHeader() which you expect to return data -- anything that returns data is blocking by default. Use a conditional like this: if TheBDM.getBDMState()=='BlockchainReady':
# The blockchain is loaded and TheBDM is sitting idle waiting for something to do
elif TheBDM.getBDMState()=='Scanning':
# Currently scanning the blockchain, go do something else
elif TheBDM.getBDMState()=='Offline':
# Blockchain is not loaded and was not requested to be loaded (usually because of --offline)
elif TheBDM.getBDMState()=='Uninitialized':
# The BDM expects it will be online at some point, but the blockchain isn't loaded yet "Uninitialized" and "Offline" are pretty much the same, so I usually use "if TheBDM.getBDMState() in ('Uninitialized','Offline'): ..." Anyways, I'm sure there's other things that are new, but obviously ArmoryQt.py is functioning correctly, so there's lots of sample code in there. Really, 95% of it is the same, just some new/renamed calls and slightly different arguments. For this application, you probably want setBlocking(True), since you might just be adding complication to an app that doesn't actually need the multi-threading. If you run the latest Armory GUI, you'll see why I made the change. MUCH better user experience... Sorry about that! Please pester me about more errors, and I'll help you recover from this under-the-hood overhaul... (and sorry in advance, there's probably another one coming... upgrading to completely new wallets and unicode support...) |
|
|
|
|
