[BOUNTY - 25 BTC] Audio/Modem-based communication library

[etotheipi]

Bounty:  Me (etotheipi) is offering 20 BTC.  Raize is offering an additional 5 BTC.  

Casascius offered a bet of 20 BTC that it can't be done.   But that requires someone to be willing to pay him 20 BTC if it's not done.  That's a bit steep for me, since I'm not convinced it is doable, but I was willing to give 20 BTC to someone who gave me usable code.

So, in my quest to improve offline wallets, I'd like to finally get started on my ultimate goal of using an audio link between two devices to get the almost-100%-secure link for moving transaction data.  I have bought double-ended audio cables and tested this between online and offline computer, and it does appear to work - I can play music on one computer and record it on the other.  Given that these are analog streams not normally intended for transmitting executable data, I think this is a perfect "side-channel" to exploit for communicating securely.  Also because audio hardware is widely available and supported on all modern OS & HW.

I expect that this would be "easiest" by downloading and adapting some kind of modem firmware, and using it to handle the communication.  My understanding is that the modem software is capable of testing the line quality and setting the communication rate and error correction automatically.  The solution doesn't have to be modem-based, but it should be able to handle 5 MB in under 15 minutes.  The vast majoriy of transfers will be on the order of a few kB, but I don't want a user who actually ends up with 5 MB to transfer to have to wait an hour to do it.

In order to get the bounty, I want python code or something that can be invoked from python (C++ can be SWIG'd in), which will give me the ability to pass data over the audio channel.    The code should have some kind of initialization/handshake routine, and then only allow for sending and receipt of ASCII data.  I basically want it to behave just like a serial port.  Remote code execution on a not-yet-compromised-offline computer should be pretty much nil in this environment.  This software would be set in "listening" mode on the offline side, and simply receive and return (to python) the string that was received.  The calling application would then parse string, request a signature from the user, and then send the signatures back over the audio link (yes, I know I need a second cable for this).

Any person wishing to collect the bounty must also be willing to assign me all rights to use it in the Armory project, free of charge, etc.  This is something I want to eventually integrate into Armory and I'm paying for it because I want to maintain full rights over my own project.

Discussion and improvements are certainly welcome, and partial solutions are also eligible for part of the bounty.

[Raize]

I'll add 5 BTC to this bounty for either modem or audio. Would be nice to see both options, however. It doesn't have to be open source, I just want to see it in Armory.

[CIYAM]

Have you also considered QR code support (you are welcome to use the CIYAM Safe scripts at no cost)?

[etotheipi]

Have you also considered QR code support (you are welcome to use the CIYAM Safe scripts at no cost)?

You'll know why I passed up QR code the first time you have to deal with transferring 4 MB between systems.   I really wanted to make it work,  but it only work for 95% of transactions without starting to get complicated, and also I didn't like the idea of multiple webcams and screens.  I'm envisioning a mess of wires and a confused user trying to figure out which webcam is supposed to be pointed at which monitor.  By all means,  it should be an excellent solution most of the time,  for most users, but I didn't want to make that the core transfer mechanism for "regular" users.

I'll add 5 BTC to this bounty for either modem or audio. Would be nice to see both options, however. It doesn't have to be open source, I just want to see it in Armory.

By the way,  the idea is to use the audio link, no matter what.  I was simply suggesting that software that used to be used for modems might save someone a lot of work,  since they were designed to do this kind of communication over an audio channel.  Building software from scratch would work,  but probably significantly more work. I didn't mean actually creating a solution using physical modems :-)

Thanks for contributing to the bounty! 

[CIYAM]

You'll know why I passed up QR code the first time you have to deal with transferring 4 MB between systems.

Ouch - yes that would be real pain (am only transferring a 1000 or so bytes at the most with my system).

By all means,  it should be an excellent solution most of the time,  for most users, but I didn't want to make that the core transfer mechanism for "regular" users.

Understood - it is more of a "duct tape" kind of solution (I did play in a few rock bands).

[Raize]

By the way,  the idea is to use the audio link, no matter what.  I was simply suggesting that software that used to be used for modems might save someone a lot of work,  since they were designed to do this kind of communication over an audio channel.  Building software from scratch would work, but probably significantly more work. I didn't mean actually creating a solution using physical modems :-)

Thanks for contributing to the bounty!  

Ahh, ok. It seemed to me that a 56k modem connection couldn't handle 5 meg that fast anyway, but maybe if you also ran some sort of 7-zip-like compression API.

ZMODEM rates for a modem connection do about 420K per minute. This might be a little tougher for someone to accomplish without having to rewrite a modem library, though you're correct in assuming that the clarity should be better than a typical modem connection so there is potential for someone to write something as a completely new communications channel.

[etotheipi]

By the way,  the idea is to use the audio link, no matter what.  I was simply suggesting that software that used to be used for modems might save someone a lot of work,  since they were designed to do this kind of communication over an audio channel.  Building software from scratch would work, but probably significantly more work. I didn't mean actually creating a solution using physical modems :-)

Thanks for contributing to the bounty! 

Ahh, ok. It seemed to me that a 56k modem connection couldn't handle 5 meg that fast anyway, but maybe if you also ran some sort of 7-zip API...

You'd think I would've figured this out by now... you're right it would be about 10-15 minutes at that speed.  It should be easy to wrap the transfer with compression, and if it takes 10 min for that once-in-a-blue-moon transfer, oh well. 

[Raize]

You'd think I would've figured this out by now... you're right it would be about 10-15 minutes at that speed.  It should be easy to wrap the transfer with compression, and if it takes 10 min for that once-in-a-blue-moon transfer, oh well.  

There is some code on this here: http://awesomegeekblog.blogspot.com/2009/04/file-transfer-over-sound-card.html

EDIT:
Ahh, and a github repo here:
https://github.com/vlofgren/file-transfer-over-soundcard/

[etotheipi]

You'd think I would've figured this out by now... you're right it would be about 10-15 minutes at that speed.  It should be easy to wrap the transfer with compression, and if it takes 10 min for that once-in-a-blue-moon transfer, oh well.  

There is some code on this here: http://awesomegeekblog.blogspot.com/2009/04/file-transfer-over-sound-card.html

EDIT:
Ahh, and a github repo here:
https://github.com/vlofgren/file-transfer-over-soundcard/

From my test of sending music over the audio link and recording it, I thought "hey, I just moved data... I wonder if I can re-encode my data as an audio file and have the standard .wav players do it for me...?"  Obviously, would need a bit of error correction, but playing audio from python is stupid simple, and I'm sure recording is, too.  It would certainly be minimalistic (using the pyaudio libraries already available rather than porting some ancient modem code...)

I thought of it because that project you linked me has a "generate" function which "turns a binary file into raw wave data."  Seems like I could play that wave in a loop until the receiving system declares that it has it.  Unfortunately, I don't think that simplistic solution is fast enough.  If all Bitcoin transactions were 250 bytes, it would be awesome (don't even need the audio cable, just put the speaker next to the mic).  But it looks like we're only talking about 100s of bits per sec, instead of 1000s, and a single 50 kB transaction would take quite a while.

@CIYAM:  just FYI -- the 4 MB comes from the fact that I'm not just transferring the transaction to be signed.  I'm also sending over every supporting transaction that supplies inputs to the transaction being signed.  This is because Satoshi inconveniently didn't include the input value in the TxIn serialization or include it the signing algorithm, and the tx doesn't explicitly specify fee.  So, the offline system has no way to compute the fee for sure without seeing the full transactions pointed to by the OutPoints in the TxIns.  Although it seems like a small hole, it's one that could cause a lot of pain, so I had to "patch" it this way.

Then I had a user complain that sending his tx was failing (for another reason), and when he sent it to me I saw it had 483 inputs.  It was something like 250 kB, with 3.5 MB of supporting transactions.  I'd say it's extreme, but there's actually quite a few people (especially SD players) that have tons of small inputs that will have to be combined if they try to clear out their wallet...

[payb.tc]

i think this would be interesting to use steganography to put the data into a piece of actual music as some kind of watermark, rather than just play data that sounds like noise.

not that that would have much practical application.

[casascius]

If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level.  Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.

Having worked with modems extensively, and currently running a business that still supports 1200 baud dialup modems to this day (I wrote 100% of the code for that) I think this is going to be super difficult to get working right and reliably, with decent error detection and correction robust enough to give satisfactory results in varying real-world conditions.

The most recent generation (post-1990) of dialup modems will do things like periodically sweep the entire frequency range of a telephone connection to find out what portions of the audio spectrum are available on a particular call, and cram all of the data in the usable spectrum.  That's the result of years of the innovation of multiple modem companies competing to make the fastest modem - something you're not going to be able to implement starting from scratch.

It's like trying to reinvent the entire 1980's and early 90's worth of modem technology all over again, but with unprecedented new constraints: audio hardware with wide signal ranges built to a loose standard, user-settable volume and equalizer controls, and audio jacks which were never meant to be cross-connected.  And, for it to all be re-invented by one guy, rather than a whole industry.

Audio inputs on computers are rarely meant to be directly connected to the audio outputs of other computers.  In most cases, the audio outputs are at a level for driving headphones, and this level will grossly overdrive any computer audio input, given that a headphone jack is meant to provide power to a pair of headphones, but inputs are meant to talk to microphones or line-level inputs (signals that are weaker by orders of magnitude than headphone output).  The receiver will hear nothing but clipped static.  A user can sort of get away with it for audio applications by turning the output volume down really low, but not if you're asking the user to do something that makes them unable to hear the signal, like not hook up speakers, and that's assuming the user is qualified to know what's the appropriate sound level for a data signal meant to be decoded by a computer.

Then you have to deal with things like ground loops, and crappy unshielded sound cards that pick up all kinds of unwanted noise from CPU that limit your bandwidth to nowhere even approaching 56Kbits/sec on random unpredictable occasions and... suddenly, this project has become more complicated than the whole of Armory itself!

By the time you start to write the troubleshooting guide for this, and try to teach your user how to figure out what line level his inputs and outputs are so his two machines can talk, you'll reluctantly fold this audio idea and go with something like RS232.

tl;dr: I'll bet you 20 BTC this plain isn't going to work even remotely as well as RS232 will - 20BTC you can add to the bounty just in case it does!

[etotheipi]

If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level.  Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.

Having worked with modems extensively, and currently running a business that still supports 1200 baud dialup modems to this day (I wrote 100% of the code for that) I think this is going to be super difficult to get working right and reliably, with decent error detection and correction robust enough to give satisfactory results in varying real-world conditions.

The most recent generation (post-1990) of dialup modems will do things like periodically sweep the entire frequency range of a telephone connection to find out what portions of the audio spectrum are available on a particular call, and cram all of the data in the usable spectrum.  That's the result of years of the innovation of multiple modem companies competing to make the fastest modem - something you're not going to be able to implement starting from scratch.

It's like trying to reinvent the entire 1980's and early 90's worth of modem technology all over again, but with unprecedented new constraints: audio hardware with wide signal ranges built to a loose standard, user-settable volume and equalizer controls, and audio jacks which were never meant to be cross-connected.  And, for it to all be re-invented by one guy, rather than a whole industry.

Audio inputs on computers are rarely meant to be directly connected to the audio outputs of other computers.  In most cases, the audio outputs are at a level for driving headphones, and this level will grossly overdrive any computer audio input, given that a headphone jack is meant to provide power to a pair of headphones, but inputs are meant to talk to microphones or line-level inputs (signals that are weaker by orders of magnitude than headphone output).  The receiver will hear nothing but clipped static.  A user can sort of get away with it for audio applications by turning the output volume down really low, but not if you're asking the user to do something that makes them unable to hear the signal, like not hook up speakers, and that's assuming the user is qualified to know what's the appropriate sound level for a data signal meant to be decoded by a computer.

Then you have to deal with things like ground loops, and crappy unshielded sound cards that pick up all kinds of unwanted noise from CPU that limit your bandwidth to nowhere even approaching 56Kbits/sec on random unpredictable occasions and... suddenly, this project has become more complicated than the whole of Armory itself!

By the time you start to write the troubleshooting guide for this, and try to teach your user how to figure out what line level his inputs and outputs are so his two machines can talk, you'll reluctantly fold this audio idea and go with something like RS232.

tl;dr: I'll bet you 20 BTC this plain isn't going to work even remotely as well as RS232 will - 20BTC you can add to the bounty just in case it does!

Isn't 80% of your criticism solved by using modem firmware/software to handle all the transmission/spectrum issues?  I know it's complicated, and I know that it was all figured out in order to achieve 56Kbps, so that's why my original post suggested using old modem stuff.  Perhaps I underestimated the effort level of starting from scratch compared to the modem option.  But I think you're understating the fact that an audio connection like this should be an almost identical environment to what modems were designed for and modems did solve this problem.

As for the connectors... What about an attenuating cable?  It seems they're designed for this...   

[casascius]

Isn't 80% of your criticism solved by using modem firmware/software to handle all the transmission/spectrum issues?  I know it's complicated, and I know that it was all figured out in order to achieve 56Kbps, so that's why my original post suggested using old modem stuff.  Perhaps I underestimated the effort level of starting from scratch compared to the modem option.  But I think you're understating the fact that an audio connection like this should be an almost identical environment to what modems were designed for and modems did solve this problem.

As for the connectors... What about an attenuating cable?  It seems they're designed for this...   

If you can get a hold of some modem software/firmware source code that can be retrofitted to work on audio signals from the wave audio feed of your choice instead of the hardware it was designed to talk to, I will consider this remarkable in and of itself.  Most of those are kept as proprietary as possible.

You won't get 56kbps by imitating a modem, since you can't get 56kbps peer-to-peer even with a real modem.  Achieving 56kbps is done by very carefully exploiting properties of the telephone lines they'd be used on.  For starters, achieving 56kbps is possible only under the most ideal of conditions, and in one direction only: from the ISP to the customer, and the ISP must have special head-end equipment that outputs the audio data in digital form directly onto a T1 or ISDN or similar line in order for it to even be possible, so as to cut out the loss going from analog to digital.

The modems themselves can push no greater than 33.6kbps to another modem, no matter how pure the connection.  The protocol simply doesn't allow for it.  And this is in the best case.

The attenuating cable you've mentioned as an example would be a nightmare.  There's no way to know how much attenuation is the right amount.  And just by looking at the cable, I can see a clear problem: it's meant for mono voice recording.  Rather than having a 3-point connector (ground, right audio, left audio), it has a two-point connector (ground and signal).  Plug this baby into a stereo jack, and you literally short the right channel to ground (compare it with a normal headphone jack and you'll see why - the right channel is the middle connection point, and the ground sleeve is shorter).  Your typical user is never going to know this without being taught.

Wasn't the original purpose of this idea to make it convenient for the user?  If the user's got to go to Radio Shack to get an unusual cable to do this communication, then he ought to just pick up a null-modem serial cable, and know with 100% certainty it's going to work.

[salfter]

You might be able to get some ideas from ADTPro:

http://adtpro.sourceforge.net/

It's a program for getting downloaded software transferred to an Apple II. It has several transfer modes to choose from according to what peripherals the destination machine has available; one mode is to run an audio cable from the line-out jack on your newer computer to the cassette-in jack on a II, II+, or IIe (or cassette-out to line-in if you need to go the other way). You can probably get a higher data rate between two newer computers than what ADTPro can manage when one end of the link is being driven by a 1-MHz 6502, but it should be a start.

[etotheipi]

Wasn't the original purpose of this idea to make it convenient for the user?  If the user's got to go to Radio Shack to get an unusual cable to do this communication, then he ought to just pick up a null-modem serial cable, and know with 100% certainty it's going to work.

Well, I was planning to have links to all the correct stuff, and offer a real "offline bundle" for those that donate a certain amount (basically, sending them a set of the cables they need ready-to-go).

The problem with the serial cable is that if they do it wrong, their wallet is exposed to everyone.  They end up with less security than they thought they did, and the false sense of security be worse than not having had the option to begin with (they would've relied on something slightly less optimally secure, but with less user error).

On the other hand, if they do the audio wrong... no one gets to spend the offline coins.  Well, the authorized user will...eventually (even if they have to switch to some other method).  You point out that they could blow up hardware... okay that's a valid risk to take into account here.  

You are clearly more knowledgeable on this stuff than I am.  You seem to think this idea is DOA.  I didn't think it was so crazy, but that's why I put up what I thought was a fairly significant bounty (I'm sure some grad student out there would like the challenge and make some money).

@salfter:

Thanks for that link.  I'll look into it. 

[paybitcoin]

You could use one of the protocols adapted for packet Ham Radio - there is already a lot of code and they are proven in noisy environments. Whoa I found this page and it would be sweet if we could encode Bitcoin transactions into this:

http://wb8nut.com/resources/olivia-1000-32.wav

(from WB8NUT's Digital Modes Information Page)

These are meant to fit in a very small bandwidth, though, of a few hundred - 1000 Hz. For short transactions it would probably work!!!

You probably wouldn't get more than about 8KHz in real life unless you were directly connected, forcing a max transfer rate of only a few kbits/sec. Directly connected would probably mean modem speeds depending on the ADC. Remember audio in PCs and phones rolls off at 22 kHz.

The problem with the serial cable is that if they do it wrong, their wallet is exposed to everyone.  They end up with less security than they thought they did, and the false sense of security be worse than not having had the option to begin with (they would've relied on something slightly less optimally secure, but with less user error).

On the other hand, if they do the audio wrong... no one gets to spend the offline coins.  Well, the authorized user will...eventually (even if they have to switch to some other method).  You point out that they could blow up hardware... okay that's a valid risk to take into account here. 

Not quite correct - remember it is still a digital codec on both sides, so no matter what you will still have the chance to introduce bugs. Even if the hacking now involves some serious sound equipment.

[Raize]

Yeah, looking at this more closely seems to indicate that RS-422 or RS-485 is going to be your best bet for very quick transfer rates. That means a null modem of some sort. And then they'd need to drop some serious cash on a USB converter. Come to think of it, I think the 56k modem option might actually end up being cheaper for them in the long run. Your risk then is possible unknown exploits in ZMODEM or XMODEM or KERMIT or whatever transfer mechanism you end up using. Like casascius says, maybe you could have the transfer mostly take place via null modem and then optionally do some sort of verification of it via audio?

Also, just because this stuff is decades old doesn't mean someone might not try to compromise it, your risk with custom code might actually end up being greater because it only has security through obscurity. I think I'd be willing to have a semi-airgapped connection via a modem though, because I could just only connect when I knew I wanted to send coin and feel relatively secure. I still like this idea, whatever you decide I'll honor the bounty I've posted.

[casascius]

The problem with the serial cable is that if they do it wrong, their wallet is exposed to everyone.  They end up with less security than they thought they did, and the false sense of security be worse than not having had the option to begin with (they would've relied on something slightly less optimally secure, but with less user error).

Having done serial comms for a very long time I don't think this is a serious issue.  I think at one point you expressed the fear that someone could unknowingly leave a getty process running on their serial port, making it a conduit where someone on the other side of the "moat" who breaches the security on one end could use the serial connection to log into the machine.

I don't see this as a realistic risk - first off, it's easily detectable by your application, you should have no problem finding out if any other processes have your desired port open.  Second, assuming you didn't detect it, the constant error message traffic generated by the getty as it complains about all the data it sees would clearly disrupt the communication so bad that the operator would notice something was wrong right away, and third there's many other obstacles, not the least of which is having to hack the owner's password through an interface that is already very hardened against brute force hacking, never minding the fact it's already slow.

You are clearly more knowledgeable on this stuff than I am.  You seem to think this idea is DOA.  I didn't think it was so crazy, but that's why I put up what I thought was a fairly significant bounty (I'm sure some grad student out there would like the challenge and make some money).

I have no doubt that someone could put together a proof of concept and get some data to go through their two computers' audio link.  But I think it's totally unrealistic to expect this to be reliable and more practical for even expert users as compared to RS232.  I think I am saving you lots of time and frustration by just saying don't mess with it.

[paybitcoin]

Meh, but if you're going to go the cable route, why not just a USB to USB transfer cable? Then you get mega bandwidth and better compatibility than with serial.

I personally have my bet on NFC.

[casascius]

Yeah, looking at this more closely seems to indicate that RS-422 or RS-485 is going to be your best bet for very quick transfer rates. That means a null modem of some sort.

RS-422 and RS-485 are far superior to RS-232, but also far less common, and far more expensive.  The only drawback to RS232 is you get ground loop problems if you want to run it at high speed at a distance of hundreds or thousands of feet.  In practice, nobody does this.

Most computers either have RS232 built in, or it can be added trivially with USB.

And then they'd need to drop some serious cash on a USB converter.

$1.99
http://www.ebay.com/itm/New-USB-2-0-to-RS232-Serial-9-Pin-9P-DB9-Adapter-Converter-/281027829591?pt=US_Parallel_Serial_PS_2_Cables_Adapters&hash=item416e905f57

Come to think of it, I think the 56k modem option might actually end up being cheaper for them in the long run. Your risk then is possible unknown exploits in ZMODEM or XMODEM or KERMIT or whatever transfer mechanism you end up using. Like casascius says, you could have the transfer mostly take place via null modem and then do some sort of verification of it via audio.

Realistically, ZMODEM/XMODEM/KERMIT aren't going to be used for this.  They are totally the wrong tool for the job.

Let me clarify that.  XMODEM is closest in function to what will probably get used.  XMODEM is a ridiculously simple protocol.  Here is the entire protocol: send a packet of up to 256 bytes with a simple addition checksum, and if the checksum didn't match, send CTRL-U to ask for a repeat, and if it the checksum matched, send CTRL-F to ask for the next packet.  End of file is signalled with CTRL-Z.  There is no room for vulnerability in that, it could be implemented directly in the application itself, hopefully using a hash algorithm as a checksum instead of simple addition.

Also, just because this stuff is decades old doesn't mean someone might not try to compromise it, your risk with custom code might actually end up being greater because it only has security through obscurity. I think I'd be willing to have a semi-airgapped connection via a modem though, because I could just only connect when I knew I wanted to send coin and feel relatively secure. I still like this idea, though, whatever you decide I'll honor the bounty I've posted.

A modem in the MOST LITERAL SENSE emulates a null modem cable while it is connected to another modem.  Think of it as a null modem cable that just happens to understand a command to pick up the phone and dial a call.

You can literally program a modem to blindly answer any incoming call and connect to whoever is on the other end, and then plug the modem into a serial device (e.g. a printer) that was meant for direct connection to a computer, not modem.  You can dial into it and start printing.  It's that simple.