Not even sure how I came across this thread.... interesting, so, I'll throw in my two cents. Data over audio really isn't all that complicated folks. It's been done many many times. Modem stuff obviously is not the best to emulate, since everything about it is designed for use on POT lines, which have very limited frequency response. For example, there is an open source fax patch for Asterisk that emulates a fax modem over VoIP. ( http://www.zoiper.com/foip/ for those interested) But its limited to ~14.4kbps max I believe. PC sound cards and smart phone sound cards and the like are much higher quality in most cases. Granted, you probably would need a decent effort to get decent speeds, still, but, it's certainly not impossible. I'd say 64kbit/sec (4MB in ~8 min) would be reasonably easy to achieve even with mono audio, and even with single direction comm. Granted that this would work best as a two way link (speaker and mic on each device). I've actually considered using audio data transfer for another project, but, never got around to making the code. And I'll never write something like this for python, so, not going to be much help here anyways. Just pointing out that it's possible! -wk Well anything in C++ would work, as it's trivial for me to pull it into python. I just thought that python would be "easier" since it creates a nice platform-independent interface to the audio devices. But if you created a C++ implementation that works for one platform, I'm sure it can be adapted it to my needs. |
|
|
|
subSTRATA, you're killing me! You are both a blessing and curse: your rigorous attention to detail combined with the fact that I'm a perfectionist has led to me fixing lots and lots of stuff, but also spending tons of time focusing on little details. Not everything was little (I did find a couple odd crashes, and dialogs that work quite right), but we all know that the app is going to have bugs and/or crash occasionally  At some point I just have to release something ... So, while I do appreciate all the little things, this next thing is just a sanity check. I have all the QR code stuff implemented, fixed address sweeping subtleties, fixed some display issues, negative send amounts... Please test it out and I'll reward any serious bugs, but I'm expecting that I'll just release it as is unless there's something epic. 0.2 BTC for anything that seriously impacts usability -- 0.3 if you find something that actually makes me re-compile/re-package everything The links for the new version are on the Discussion thread. By the way, I addressed 6 slightly (long story), 7 and 8 and 9 are not things I'm going to fix right now (though 8 does resolve itself the next time the ledger is updated or you switch the filter... which happens occasionally and on new relevant tx). |
|
|
|
Alright, so I finally have a "release candidate" for 0.87 (it is 0.86.9-testing). I just uploaded the installers to the code.google downloads page. As soon as I get a sanity check, I'll turn it into 0.87-beta and finally release it with all the extra goodies... http://code.google.com/p/bitcoinarmory/downloads/detail?name=armory_0.86.9-testing_win64.msihttp://code.google.com/p/bitcoinarmory/downloads/detail?name=armory_0.86.9-testing_windows_all.msihttp://code.google.com/p/bitcoinarmory/downloads/detail?name=armory_0.86.9-1_amd64.debhttp://code.google.com/p/bitcoinarmory/downloads/detail?name=armory_0.86.9-1_i386.deb
Now, someone with networking/JSON/python-twisted experience please help me. I'm working on creating the JSON-RPC server, and I have all the pieces together except for the authentication. I know I just need very simple Digest authentication using username and password in a file only readable by the user. However, for such a "simple" thing, there's like 800 different ways to get there, and I have no idea how to do it. From my research, I think I have found all the pieces I need, but I'm not sure how to put them together yet ... it's like a jigsaw puzzle. I see that I need: twisted.web.guard.resource.Resource to be protected twisted.cred.portal.IRealm that creates "avatars" (incarnations of the user on the server) twisted.web.guard.DigestCredentialFactory specifying the realm twisted.cred.portal.Portal to that links the realm to a ... twisted.cred.checkers.FilePasswordDB object specifying the user/pass list twisted.web.guard.HTTPAuthSessionWrapper The server code and launch without authentication looks like: class Armory_Json_Rpc_Server(jsonrpc.JSONRPC):
...
reactor.listenTCP(RPC_PORT, \
server.Site(Armory_Json_Rpc_Server(self.wallet)), \
interface="127.0.0.1")
You can see the full source code as mav gave me here (with slight modifications, and adding all the complicated pieces above) Anyone want to help me figure out how to create and hook together all these things? Guarded resources, realms, avatar factories, a portals, etc? (sounds like I'm playing a RPG computer game with knights and wizards and magic, doesn't it?) |
|
|
|
Just to be clear, before I'd consider any effort a success for the purpose of my "bet" offer statement, I'd require that one successfully implement an audio-based bidirectional data interface that exceeds the utility of an RS232 port and that is not significantly more difficult to set up than a null modem connection.
Typical max of RS232 ports on computers is 11.5 kilobytes per second, with an error rate that is substantially zero. This would have to be achieved or exceeded, with a zero error rate. (RS232 can go higher, but 11.5 KBytes/second is the typical max that the hardware is commonly configured with as shipped on equipment, a capability that can be assumed to exist on pretty much every PC with a serial port, and therefore the value I will use).
I'm not offering to pay 20 BTC for a demonstration that someone could get some data over a sound card - since I have no doubt it's possible. Also, my statement that I'd be willing to "bet" is not a public bounty, but a private offer that I made to etotheipi and is technically contingent on him accepting it and being willing to pay me 20 BTC in case of failure... a restriction I would probably relax in the face of overwhelming evidence I was wrong.
Fair enough. You're right I was a little hasty in adding you to the bounty. I should remove that. Honestly, I wasn't sure this was going to work, and the original bounty was intended give some incentive for someone to try. I am not confident enough that this is so doable as to bet 20 BTC in the event that I was wrong -- I was simply willing to pay if it led me to something usable. But to be fair, to say it works better than RS232 is a bit extreme. The point of this exercise was to demonstrate that there is a reasonable alternative to RS232 -- one that is effectively isolated other existing modes of transfer (and thus have infrastructure/autorun stuff to assist with it that might be a security vulnerability). My goal was not to have it perform faster. It just has to transmit at least the same order-of-magnitude of data reliably (1/3 or more?), since that would enable me to implement the capability. You are saying "I bet no one can create a better xfer protocol on audio than RS232." That's not my goal. My goal is "create an alternative that has similar performance." But I'm not willing to bet 20 BTC that it's strictly possible, I was only offering 20 BTC for a working solution (with probably partial bounty for someone demonstrating that it can't). But I don't want to add another 20 BTC to the failure case. |
|
|
|
What platform/s would this need to support in order to collect the bounty?
It needs to support all platforms (since the goal is to get it to work in a platform-agnostic way). Python, as usual, is pretty good about handling the platform-dependent details, and gives you a uniform interface to the audio objects. I suppose to be supported on Android, it would need a custom java implementation, but that can be dealt with after the original implementation is demonstrated. This needs to be accessible from python. If it's a C++ implementation ,I can use swig to access it from python like I do a lot of other things. But my guess is that you would want to do it in Python, and SWIG-in libraries that will do quick encoding/decoding for you ...? (i.e. python handles the interface to the audio devices, but hands off all the data to a fast C++ backend for doing encoding/decoding). I'm not entirely clear about the best way, but it definitely needs to be supported on Windows, Linux and Mac... under the assumption that the audio devices are already configured correctly. I would recommend doing some research, but don't go too crazy with it yet. Report back any revelations you have before actually trying to implement it -- if this looks like it's going to terrible (like difficult hardware support, requires complicated algorithms for handling various line qualities, has problems with feedback, etc... then we should probably think about another way). On the other hand, there's nothing wrong with developing a pseudo-serial connection over audio. Technically, this doesn't have to be bitcoin- or armory-specific. As casascius suggested, perhaps the thing to is for me to write the communication assuming that there is a serial-like interface, and then it could operate over serial ports, IR, TOSLINK, or audio, as long as there's a "pipe" for it. |
|
|
|
|
Does bitcoind RPC interface broadcast arbitrary transactions for you? My understanding is that "stock" bitcoind will not accept such a 50 kB transaction without a substantial fee -- it would be DOA. Obviously other nodes might use different fee rules, but I want to understand why your own bitcoind took it?
Also, the latest version of Armory has coin-control -- but only at the address level. If your coins are split up between lots of addresses, you can do this house-cleaning, though you'll have to pay a full fee, because stock bitcoind won't accept it and forward it if you don't.
|
|
|
|
it should be able to handle 5 MB in under 2 minutes.
* Data to move 5 MB * Data to move 5242880 bits * Max audio date rate 9600 bits pr sec * Time to transfer data 546,1 sec * Time to transfer data 9,1 min Add some error correction and stuff we are talking 15 min to move 5 MB. It is not practical to transfer using audio. Yeah, I recanted that statement in a later post, I guess I should update the top post. If I do go with the audio solution, the rare 5 MB will just require patience on the part of the user... |
|
|
|
|
By the way, one the other reasons I was so set on audio, was because I also wanted to enable smartphones to be used as the offline signing devices. Take an old smartphone, disable WiFi, remove the SIM card, and then only connect it via audio. That seemed to be a nice bonus to the audio solution (despite having to buy some adapters).
Now, casascius is doing a good job of persuading me that the challenges of audio are too great, and the risks of serial ports can be mitigated. I'll be doing more research, but my question then becomes: is there any hope for the smartphones? Obviously, you can move the microSD card around, but that's what I'm trying to avoid in this thread/bounty. What other options are there besides QR codes?
Or is there a way to make QR codes work for real? If I have to move 1 MB, could I perhaps create a tagged list of 50 QR codes and play them in a video (looped), and let the other device just keep watching until it picks up all of them? This is less of a problem than I originally mentioned to CIYAM, because you only need one webcam/wire, and there's no confusion about which device to point where (obviously, you can't point a smartphone's camera at itself).
There's gotta be a better way!
|
|
|
|
why not a stupid old serial cable? why the sound stuff? sound stuff is much vulnerable, as it can be listened to.
My original concern was that most linux systems still have tty logins enabled by default over serial. Given that most users' username/passwd will be the same as one of the ones in their firefox/chrome saved passwords file, this could actually be the cause of many people losing coins. Lots of people will do the minimum amount of work to just hook up the serial cable and use it if it "works". Meanwhile, an attacker may be able to waltz onto the offline computer and have their way with it, while the user sits snugly on their couch thinking there's nothing that could go wrong. But Casascius rightfully pointed out that many of these SNAFU conditions are detectible. A user may not want to read even a simple set of instructions to set it up properly, but if I can actually disable offline Armory until the problem is fixed, then it makes it much more feasible. I'll be thinking about that, and doing some research on what it would take to detect and close this attack vector. Also, audio "listening" is not the problem. None of the data being transferred is secret. It's just transaction data that will end up in the blockchain, anyway. And even if it was, 99.9%+ of the threat vector is not threats in the same room* (if the threat has physical access to your system, you're probably already screwed). It's from having to link the offline computer to the internet (indirectly) and exposing it to the entire world of hackers and viruses. *EDIT: I shouldn't generalize: I'm sure there are applications/scenarios where physical threats are a concern -- but the vast majority of users are not them, and the ones that do care know they need some kind of extra security measures. |
|
|
|
Wasn't the original purpose of this idea to make it convenient for the user? If the user's got to go to Radio Shack to get an unusual cable to do this communication, then he ought to just pick up a null-modem serial cable, and know with 100% certainty it's going to work.
Well, I was planning to have links to all the correct stuff, and offer a real "offline bundle" for those that donate a certain amount (basically, sending them a set of the cables they need ready-to-go). The problem with the serial cable is that if they do it wrong, their wallet is exposed to everyone. They end up with less security than they thought they did, and the false sense of security be worse than not having had the option to begin with (they would've relied on something slightly less optimally secure, but with less user error). On the other hand, if they do the audio wrong... no one gets to spend the offline coins. Well, the authorized user will...eventually (even if they have to switch to some other method). You point out that they could blow up hardware... okay that's a valid risk to take into account here. You are clearly more knowledgeable on this stuff than I am. You seem to think this idea is DOA. I didn't think it was so crazy, but that's why I put up what I thought was a fairly significant bounty (I'm sure some grad student out there would like the challenge and make some money). @salfter: Thanks for that link. I'll look into it. |
|
|
|
For the -0.00000001 fee, possibly because I have not updated the armory in my offline computer for a long time. So that might have been fixed.
Ahh, that would explain it. Those things mostly showed up in offline mode when it tried to display balances that were defaulted to -1 when there was no blockchain info available. A couple months ago I went through and cleaned up all that that I could find. Obviously, the old version works and doesn't need to be upgraded, but I would recommend it since I added a confirmation window like you have when sending money online, to make it easier to confirm the transaction details (it shows you all recipients and fee before you sign it). It also has a couple extra warnings that may pop up if something about the tx looks suspicious. |
|
|
|
If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level. Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.
Having worked with modems extensively, and currently running a business that still supports 1200 baud dialup modems to this day (I wrote 100% of the code for that) I think this is going to be super difficult to get working right and reliably, with decent error detection and correction robust enough to give satisfactory results in varying real-world conditions.
The most recent generation (post-1990) of dialup modems will do things like periodically sweep the entire frequency range of a telephone connection to find out what portions of the audio spectrum are available on a particular call, and cram all of the data in the usable spectrum. That's the result of years of the innovation of multiple modem companies competing to make the fastest modem - something you're not going to be able to implement starting from scratch.
It's like trying to reinvent the entire 1980's and early 90's worth of modem technology all over again, but with unprecedented new constraints: audio hardware with wide signal ranges built to a loose standard, user-settable volume and equalizer controls, and audio jacks which were never meant to be cross-connected. And, for it to all be re-invented by one guy, rather than a whole industry.
Audio inputs on computers are rarely meant to be directly connected to the audio outputs of other computers. In most cases, the audio outputs are at a level for driving headphones, and this level will grossly overdrive any computer audio input, given that a headphone jack is meant to provide power to a pair of headphones, but inputs are meant to talk to microphones or line-level inputs (signals that are weaker by orders of magnitude than headphone output). The receiver will hear nothing but clipped static. A user can sort of get away with it for audio applications by turning the output volume down really low, but not if you're asking the user to do something that makes them unable to hear the signal, like not hook up speakers, and that's assuming the user is qualified to know what's the appropriate sound level for a data signal meant to be decoded by a computer.
Then you have to deal with things like ground loops, and crappy unshielded sound cards that pick up all kinds of unwanted noise from CPU that limit your bandwidth to nowhere even approaching 56Kbits/sec on random unpredictable occasions and... suddenly, this project has become more complicated than the whole of Armory itself!
By the time you start to write the troubleshooting guide for this, and try to teach your user how to figure out what line level his inputs and outputs are so his two machines can talk, you'll reluctantly fold this audio idea and go with something like RS232.
tl;dr: I'll bet you 20 BTC this plain isn't going to work even remotely as well as RS232 will - 20BTC you can add to the bounty just in case it does!
Isn't 80% of your criticism solved by using modem firmware/software to handle all the transmission/spectrum issues? I know it's complicated, and I know that it was all figured out in order to achieve 56Kbps, so that's why my original post suggested using old modem stuff. Perhaps I underestimated the effort level of starting from scratch compared to the modem option. But I think you're understating the fact that an audio connection like this should be an almost identical environment to what modems were designed for and modems did solve this problem. As for the connectors... What about an attenuating cable? It seems they're designed for this... |
|
|
|
|
Just pushed 0.86.8 to the testing branch. At this point will just be released as 0.87. Fixed tons of bugs I can't even enumerate (most of them tiny).
@ HostFat: I think you'll be pleasantly surprised. All QR codes are inflatable! It works in Linux, at least...
Also got permission from mav to add his JSON-RPC code to the project (committed as armoryd.py). The code is not up to my quality standards yet, but is fantastic skeleton for me to finally get that support in there. I may be posting questions here for others more knowledgeable about JSON servers. So far, it looks like what he's got works, but it needs some more security features, and some more commands. Will be making that a priority, actually...
Please test!
|
|
|
|
From my test of sending music over the audio link and recording it, I thought "hey, I just moved data... I wonder if I can re-encode my data as an audio file and have the standard .wav players do it for me...?" Obviously, would need a bit of error correction, but playing audio from python is stupid simple, and I'm sure recording is, too. It would certainly be minimalistic (using the pyaudio libraries already available rather than porting some ancient modem code...) I thought of it because that project you linked me has a "generate" function which "turns a binary file into raw wave data." Seems like I could play that wave in a loop until the receiving system declares that it has it. Unfortunately, I don't think that simplistic solution is fast enough. If all Bitcoin transactions were 250 bytes, it would be awesome (don't even need the audio cable, just put the speaker next to the mic). But it looks like we're only talking about 100s of bits per sec, instead of 1000s, and a single 50 kB transaction would take quite a while. @CIYAM: just FYI -- the 4 MB comes from the fact that I'm not just transferring the transaction to be signed. I'm also sending over every supporting transaction that supplies inputs to the transaction being signed. This is because Satoshi inconveniently didn't include the input value in the TxIn serialization or include it the signing algorithm, and the tx doesn't explicitly specify fee. So, the offline system has no way to compute the fee for sure without seeing the full transactions pointed to by the OutPoints in the TxIns. Although it seems like a small hole, it's one that could cause a lot of pain, so I had to "patch" it this way. Then I had a user complain that sending his tx was failing (for another reason), and when he sent it to me I saw it had 483 inputs. It was something like 250 kB, with 3.5 MB of supporting transactions. I'd say it's extreme, but there's actually quite a few people (especially SD players) that have tons of small inputs that will have to be combined if they try to clear out their wallet... |
|
|
|
The private key shown in testnet mode is not compatible with satoshi client. The leading symbol should be "c" instead of "5": https://en.bitcoin.it/wiki/List_of_address_prefixesNot sure if this counts because it's minor problem in the testnet mode. Just forget the bounty if you think it is not an important one Another bug is rounding error in transaction fee. Sometimes when I sign a transaction with 0 fee, the client shows something like -0.00000001 (yes, a negative number) as fee. It seems just a cosmetic problem without affecting usability. Is it related to the use of floating-point number? How about 0.1 for both? The testnet keys are obviously not that important, but it was silly/unnecessary to be hardcoding that. And after all, I complain about not having enough testers, so it's worth fixing to make it easier for testers! Where did you see the -0.00000001 fee? Usually this occurs when value variables that have been set to sentinel UNINITIALIZED (== -1) and then somehow end up getting used anyway. There shouldn't be any problems with rounding, because I only use ints/longs for all values (in satoshis), and have a coin2str and str2coin function that converts between value and visual repr. However, I suppose it's still possible to have some -1's laying around... is it consistent enough you can tell me how to reproduce it? I don't think I've seen this myself... (I mean, I've seen it in the past, but I thought I had fixed all of them) |
|
|
|
By the way, the idea is to use the audio link, no matter what. I was simply suggesting that software that used to be used for modems might save someone a lot of work, since they were designed to do this kind of communication over an audio channel. Building software from scratch would work, but probably significantly more work. I didn't mean actually creating a solution using physical modems :-)
Thanks for contributing to the bounty!
Ahh, ok. It seemed to me that a 56k modem connection couldn't handle 5 meg that fast anyway, but maybe if you also ran some sort of 7-zip API... You'd think I would've figured this out by now... you're right it would be about 10-15 minutes at that speed. It should be easy to wrap the transfer with compression, and if it takes 10 min for that once-in-a-blue-moon transfer, oh well. |
|
|
|
Have you also considered QR code support (you are welcome to use the CIYAM Safe scripts at no cost)?
You'll know why I passed up QR code the first time you have to deal with transferring 4 MB between systems. I really wanted to make it work, but it only work for 95% of transactions without starting to get complicated, and also I didn't like the idea of multiple webcams and screens. I'm envisioning a mess of wires and a confused user trying to figure out which webcam is supposed to be pointed at which monitor. By all means, it should be an excellent solution most of the time, for most users, but I didn't want to make that the core transfer mechanism for "regular" users. I'll add 5 BTC to this bounty for either modem or audio. Would be nice to see both options, however. It doesn't have to be open source, I just want to see it in Armory.
By the way, the idea is to use the audio link, no matter what. I was simply suggesting that software that used to be used for modems might save someone a lot of work, since they were designed to do this kind of communication over an audio channel. Building software from scratch would work, but probably significantly more work. I didn't mean actually creating a solution using physical modems :-) Thanks for contributing to the bounty! |
|
|
|
|
Bounty: Me (etotheipi) is offering 20 BTC. Raize is offering an additional 5 BTC.
Casascius offered a bet of 20 BTC that it can't be done. But that requires someone to be willing to pay him 20 BTC if it's not done. That's a bit steep for me, since I'm not convinced it is doable, but I was willing to give 20 BTC to someone who gave me usable code.
So, in my quest to improve offline wallets, I'd like to finally get started on my ultimate goal of using an audio link between two devices to get the almost-100%-secure link for moving transaction data. I have bought double-ended audio cables and tested this between online and offline computer, and it does appear to work - I can play music on one computer and record it on the other. Given that these are analog streams not normally intended for transmitting executable data, I think this is a perfect "side-channel" to exploit for communicating securely. Also because audio hardware is widely available and supported on all modern OS & HW.
I expect that this would be "easiest" by downloading and adapting some kind of modem firmware, and using it to handle the communication. My understanding is that the modem software is capable of testing the line quality and setting the communication rate and error correction automatically. The solution doesn't have to be modem-based, but it should be able to handle 5 MB in under 15 minutes. The vast majoriy of transfers will be on the order of a few kB, but I don't want a user who actually ends up with 5 MB to transfer to have to wait an hour to do it.
In order to get the bounty, I want python code or something that can be invoked from python (C++ can be SWIG'd in), which will give me the ability to pass data over the audio channel. The code should have some kind of initialization/handshake routine, and then only allow for sending and receipt of ASCII data. I basically want it to behave just like a serial port. Remote code execution on a not-yet-compromised-offline computer should be pretty much nil in this environment. This software would be set in "listening" mode on the offline side, and simply receive and return (to python) the string that was received. The calling application would then parse string, request a signature from the user, and then send the signatures back over the audio link (yes, I know I need a second cable for this).
Any person wishing to collect the bounty must also be willing to assign me all rights to use it in the Armory project, free of charge, etc. This is something I want to eventually integrate into Armory and I'm paying for it because I want to maintain full rights over my own project.
Discussion and improvements are certainly welcome, and partial solutions are also eligible for part of the bounty.
|
|
|
|
~~~~~
28. On "Transactions" tab, attempting to sort transactions based on incoming / outgoing column does nothing. Icon tooltip for both types of
transactions - green arrow pointing at door and red arrow pointing away from door - says "Bitcoins received".
~~~~~
Tooltip text for outgoing transactions is changed to "Bitcoins sent" but sorting still does not work. ~~~~~
33. Create wallet window title is "Create/Import Armory Wallet" but user can't import wallet using window in question here.
~~~~~
Not fixed. Thanks. I sent you 0.5 BTC last night. I wasn't quite sure how to count things up since I changed the rules slightly, and I don't know how to fit everything in. I hope you'll think that's fair... I never added sorting to the in/out column, because I didn't see the point. And you're right I forgot to change the "Create/Import wallet" window. I just fixed it now and it should be in the real release (coming soon!) |
|
|
|
I thought that computing the expected values of each bet would illuminate this but it's just confusing me more. Since the house edge is supposed to be 1.9% all of the EV's should all be around 0.981 but they're not, so unless I'm missing something satoshidice is fudging some numbers, making their house edge a little fatter than they're letting on.
I haven't had time to dig into the math, yet, but I wanted to point out that what you observed isn't fudging. You just haven't taken into account that you get 0.5% of your bet back even when you lose. Then your 0.976 payouts add up to 0.981. |
|
|
|
|
Show Posts
