[BOUNTY - 25 BTC] Audio/Modem-based communication library

[etotheipi]

Regarding smartphone<->PC communication: it's indeed an interesting problem. I'll think on it...

Smartphone <--> Smartphone, too.  The nice thing about the audio solution is that there's no reason it wouldn't work, you just need the right cables.  And of course an Android version of the audio xfer

[1713514160]

1713514160

[1713514160]

1713514160

[1713514160]

1713514160

[1713514160]

1713514160

[roman.z]

Hi W-M,

Thanks for the compliments!

Your comments are both correct:
- I will find a stereo cable and test it with my PC<->laptop setup.
- I will use now QAM-16, and maybe even QAM-64 if the SNR will be high enough.

Best regards,
  Roman.

Hello roman.z,etothepi and other people.

I am mostly a webdeveloper by trade, but I tinker a lot with audio as well (amongst other things, creating 100% code-generated song covers known as '/predfate2lq]Bytebeat'), and might aspire to become a full-fledged synthesizer developer or something of the likes in the future.



This truly is a very interesting idea. I would give it a shot myself, if it were not that roman.z was already doing such great work already; I don't want to make a competition about this and rather think that it's better to collaborate.

Some thoughts I stumbled upon while thinking about from your solution:
-Are you using mono or stereo audio cables right now? From the source cpde it seems to me you're not. Most computers have a built-in stereo input/output, and this would mean that you can effectively double the amount of carrier waves, and thus double the speed.
-You might want to look into QAM instead of using QPSK/8PSK, which basically adds amplitude-shift-keying on top of the phase-key shifting, greatly enhancing the amount of constellations (and thus the amount of data throughput)


Have a nice day,

~W-M

[roman.z]

Hi all,

Audio modem library v1.0 is ready!
I've moved it to GitHub, under https://github.com/romanz/amodem.

The README file (https://github.com/romanz/amodem/blob/master/README.md) is now more detailed, and contains calibration and usage demos.
There are even screencasts for the demos:

• calibration: http://youtu.be/iCg1tepGz10
• usage: http://youtu.be/94yS3IZmtho

The data from my test is at: https://www.dropbox.com/sh/2yai1xmntdqlwf1/AACvfzasKEHK0zVxdzI4jF7pa

A few more details about the version:
The modem is using OFDM over an audio cable with the following parameters:

    Sampling rate: 32 kHz
    BAUD rate: 1 kHz
    Symbol modulation: 64-QAM
    Carriers: (1,2,3,4,5,6,7,8) kHz

This way, modem achieves 6[bit/symbol]*8[symbol/baud]*1000[baud] = 48kpbs bitrate = 6.0 kB/s.
A simple framing with Reed-Solomon ECC is used, with (255,245) rate = ~4% overhead.
(it should take a bit less than 3 minutes for 1MB to be transmitted)

I would be happy if more people will try and test this library.
Questions, remarks and suggestions are welcome!

[justusranvier]

The process requires a single manual calibration step: the transmitter has to find maximal output volume for its sound card, which will not saturate the receiving microphone.

It should be possible to automate this step.

Both sides start with the volume low and slowly ramp up until they establish communication with each other, then keep ramping up until they stop getting positive replies from the other side.

[roman.z]

You are right, but this would require simulatenous bi-directional connection between the online and the offline computers, requiring two audio cables.
Since I have got only one audio cable, I am calibrating the sender manually.

[justusranvier]

You are right, but this would require simulatenous bi-directional connection between the online and the offline computers, requiring two audio cables.
Since I have got only one audio cable, I am calibrating the sender manually.

Since the deployed solution is going to need to send data in both directions, you should probably just consider simultaneous bidirectional communication to be part of the requirements.

[intron]

   Sampling rate: 32 kHz
    BAUD rate: 1 kHz
    Symbol modulation: 64-QAM
    Carriers: (1,2,3,4,5,6,7,8) kHz

64-QAM audio. Nice:)

[Ente]

I'm late to the party, but really like where this concept is heading to!

If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level.  Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.

Yes, I totally agree, a generic, modular solution would be sweet.
Both devices use a (virtual or physical) RS232 port, where users can decide what they put in between the two systems.

I would then not use an audio layer on top of that, but an optical:
- a green LED for sending data from online to offline, shining onto a phototransistor connected to the offline system
- a red LED for sending data from offline to online, shining onto a phototransistor connected to the online system

They flicker happily away, should be pretty fast too, and I immediately see when and how long they transmit data, and in what direction too!

Other users might prefer the audio link. Or a plain direct RS232 cable.

What do you guys think?

Ente

[markjenkins]

Thank you roman.z for contributing another soft modem to the world. They're pretty cool!

At first I thought this wasn't a good fit for bitcoin tx signing and was having the same "why not use RS-232" thought of others.

The one downside to using this is that it introduces yet another bus layer. Anyone using this should wonder if there is a security flaw in the decoder where a malicious encoder could break away from the bus layer and actually screw up the decoder software sufficiently to take over its execution environment. At least this can be audited for with this being a soft modem.

Ideal setup is probably to use RS-232 with the signing side RS-232 UART implemented in silicon (such as a super I/O chip), not by a firmware. Chances are that if the internet side of this is messing with the bus layer (if it were software defined and not a hardware UART) that it can't break past the signing side being hardware defined for the bus.

That way all security audit focus can shift to the data layer to make sure no malicious break-outs can happen there. Regardless of the link type, the quality of this layer of software will be the most important thing.

But, I appreciate that many people don't have the option of having a hardware UART on the signing side, so this is awesome.

Particularly because USB is not so very awesome:
http://www.wired.com/2014/07/usb-security/

Malicious firmwares on USB devices can do all kinds of nasty things to the host computer -- you don't want a compromised USB RS-232 adapter to be connected to your signing computer.

Let's go through some scenarios:
If you trust your RS-232 USB adapter for the your signing computer at the time of purchase, and if you can protect its integrity thereafter, consider two sub-cases:
 * Your internet side computer has a silicon defined UART -- there is probably no way it can mess with the bus layer aside from changing configured speeds and such and probably can not compromise the firmware of the software defined UART in the signing side RS-232 USB adapter

 * Your internet side computer has a software defined UART as well and can thus mess with things at the bus layer -- I'm going to say it is possible but unlikely that it could use that level of access to compromise the signing side USB device on the other end of the cable. There would have to be a security bug in the signing side USB firmware. Downside here is that these firmwares are mostly proprietary and thus costly to audit.


There very well may be situations where the continued integrity of the adapter can not be assured but the integrity of the signing computer can be (not simply by guarding it, but ensuring anyone who reaches it can't mess with it by using a TPM to ensure firmware and boot process integrity with the rest of the disk encrypted, fancy unique security seals on the case and regular internal inspections).

Write your fan fiction here folks where someone gets over the fence, drugs the dog, drugs the guard, hacks the walkie-talkie watchdog feature, hacks the security cameras sending video feed out by celluar to keep showing silent night, temporarily disables one alarm system with watchdog with fake watchdog, weaves through lasers of other alarm system that can't be disabled, cracks the safe, finds the USB adapter with the unique case, pops it open, desolders microcontroler that lacks flash memory (firmware uploaded on connect by computer), replaces with microcontroler that has malicious firmware flashed on, gets out of there by jetpack on the roof..... exchanges tumbled bitcoins for beachfront property in Caribbean...

[Ente]

I would always consider every single aspect of the online system to be compromised.
So the offline part better be good! :-)

As we inherently trust the offline system, we don't need silicon there. As we don't trust the online system, we couldn't trust the silicon there, as someone might have found a way to make it behave differently than expected.
For me, the border between "trusted" and "untrusted" is in the middle of the link (audio, cable, qr-code, LEDs), so I'm not sure any silicon would be worth it.
If the silicon is flashable, it's insecure.
If the silicon is "hard wired", it's custom and very expensive.
..I think.

Ente

[roman.z]

I'm late to the party, but really like where this concept is heading to!

If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level.  Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.

Yes, I totally agree, a generic, modular solution would be sweet.
Both devices use a (virtual or physical) RS232 port, where users can decide what they put in between the two systems.

I would then not use an audio layer on top of that, but an optical:
- a green LED for sending data from online to offline, shining onto a phototransistor connected to the offline system
- a red LED for sending data from offline to online, shining onto a phototransistor connected to the online system

They flicker happily away, should be pretty fast too, and I immediately see when and how long they transmit data, and in what direction too!

Other users might prefer the audio link. Or a plain direct RS232 cable.

What do you guys think?

Ente

This sounds very interesting!

I think that given two sound cards with optical input/output (such as http://www.dx.com/p/external-5-1-channel-usb-2-0-sound-card-optical-audio-adapter-black-41289),
it may be possible to use the digital audio interface as a uni-directional data stream...

The single problem is the additional cost (if existing sound card has no such interfaces), and maybe potential security implication (if USB sound card is used).

[roman.z]

Thank you roman.z for contributing another soft modem to the world. They're pretty cool!

Thanks for the compliments!
It was indeed fun

[2112]

* silicon defined UART
 * software defined UART

I'd like you to post links to the chip data sheets and computer I/O products that are the examples of two UART classes that you've just defined.

Otherwise I'm going to call baloney.

Edit: forgot to quote the full message into my collection.

Thank you roman.z for contributing another soft modem to the world. They're pretty cool!

At first I thought this wasn't a good fit for bitcoin tx signing and was having the same "why not use RS-232" thought of others.

The one downside to using this is that it introduces yet another bus layer. Anyone using this should wonder if there is a security flaw in the decoder where a malicious encoder could break away from the bus layer and actually screw up the decoder software sufficiently to take over its execution environment. At least this can be audited for with this being a soft modem.

Ideal setup is probably to use RS-232 with the signing side RS-232 UART implemented in silicon (such as a super I/O chip), not by a firmware. Chances are that if the internet side of this is messing with the bus layer (if it were software defined and not a hardware UART) that it can't break past the signing side being hardware defined for the bus.

That way all security audit focus can shift to the data layer to make sure no malicious break-outs can happen there. Regardless of the link type, the quality of this layer of software will be the most important thing.

But, I appreciate that many people don't have the option of having a hardware UART on the signing side, so this is awesome.

Particularly because USB is not so very awesome:
http://www.wired.com/2014/07/usb-security/

Malicious firmwares on USB devices can do all kinds of nasty things to the host computer -- you don't want a compromised USB RS-232 adapter to be connected to your signing computer.

Let's go through some scenarios:
If you trust your RS-232 USB adapter for the your signing computer at the time of purchase, and if you can protect its integrity thereafter, consider two sub-cases:
 * Your internet side computer has a silicon defined UART -- there is probably no way it can mess with the bus layer aside from changing configured speeds and such and probably can not compromise the firmware of the software defined UART in the signing side RS-232 USB adapter

 * Your internet side computer has a software defined UART as well and can thus mess with things at the bus layer -- I'm going to say it is possible but unlikely that it could use that level of access to compromise the signing side USB device on the other end of the cable. There would have to be a security bug in the signing side USB firmware. Downside here is that these firmwares are mostly proprietary and thus costly to audit.


There very well may be situations where the continued integrity of the adapter can not be assured but the integrity of the signing computer can be (not simply by guarding it, but ensuring anyone who reaches it can't mess with it by using a TPM to ensure firmware and boot process integrity with the rest of the disk encrypted, fancy unique security seals on the case and regular internal inspections).

Write your fan fiction here folks where someone gets over the fence, drugs the dog, drugs the guard, hacks the walkie-talkie watchdog feature, hacks the security cameras sending video feed out by celluar to keep showing silent night, temporarily disables one alarm system with watchdog with fake watchdog, weaves through lasers of other alarm system that can't be disabled, cracks the safe, finds the USB adapter with the unique case, pops it open, desolders microcontroler that lacks flash memory (firmware uploaded on connect by computer), replaces with microcontroler that has malicious firmware flashed on, gets out of there by jetpack on the roof..... exchanges tumbled bitcoins for beachfront property in Caribbean...

[markjenkins]

2112, you're right to call baloney.

A real hardware UART is a standard feature in microcontrolers. (for example)

I would be hard pressed to find a microcontroller datasheet that didn't have one. Even if I did, such a datasheet wouldn't necessarily go out of its way to explain that I the programmer of said micro could achieve the same functionality in software by bit banging general purpose IO pins. It would more likely be documented in a data sheet if such functionality were specifically included in firmware.

There are two commercial products I know of that had no UART and had RS-232 bit banged on general purpose (user) io pins, the low cost VIC-20 and Commodore 64. I think support for this was included in the firmwares.

With UARTs being ubiquitous and with bit banging being very inefficient, I would say there is next to no chance that any USB RS-232 adapter does that. I was wrong to raise it as a possibility. It's safe to assume they all have a hardware UART on the RS-232 side of them.

You only need to worry about the USB side of the signing machine. On reflection, that Wired article may have over-hyped the dangers. USB devices can't just arbitrarily read/write any system memory can they? I think a malicious USB device has to be more sneaky.

[Ente]

I'm late to the party, but really like where this concept is heading to!

If it were me, I would implement this as RS232 serial comms, and then try to go about seeing if someone wanted to write a generic audio-to-serial device driver that emulated a serial port at the kernel level.  Then the separation of duties in the code is appropriate, and the audio overhead is totally optional.

Yes, I totally agree, a generic, modular solution would be sweet.
Both devices use a (virtual or physical) RS232 port, where users can decide what they put in between the two systems.

I would then not use an audio layer on top of that, but an optical:
- a green LED for sending data from online to offline, shining onto a phototransistor connected to the offline system
- a red LED for sending data from offline to online, shining onto a phototransistor connected to the online system

They flicker happily away, should be pretty fast too, and I immediately see when and how long they transmit data, and in what direction too!

Other users might prefer the audio link. Or a plain direct RS232 cable.

What do you guys think?

Ente

This sounds very interesting!

I think that given two sound cards with optical input/output (such as http://www.dx.com/p/external-5-1-channel-usb-2-0-sound-card-optical-audio-adapter-black-41289),
it may be possible to use the digital audio interface as a uni-directional data stream...

The single problem is the additional cost (if existing sound card has no such interfaces), and maybe potential security implication (if USB sound card is used).

Actually I had a much simpler way in mind:
a serial cable coming from both the online and offline computer, meeting in a transparent box. There, they have LEDs and phototransistors hooked up. Much like an optocoppler. Now the two computers can effectively speak to each other over RS-232. The difference is that I can visually see there is data communication, how much, how long, and in what direction (via different colored LEDs in both directions).

How Armory speaks to the onboard RS-232 port, or to a USB-to-RS232 adapter is just software. And exactly this dynamic, modular software layer I was proposing! :-)

Ente

[filebit]

This idea is gold, while I was reading this several ideas came to mind regarding ways to use this technology to benefit us all.

Good luck!

[Ente]

You only need to worry about the USB side of the signing machine. On reflection, that Wired article may have over-hyped the dangers. USB devices can't just arbitrarily read/write any system memory can they? I think a malicious USB device has to be more sneaky.

I'm not worried at the moment.

- USB has no evelated rights per se. Whatever you stick in, runs with user rights. No matter if legit or fake keyboard or cam or flash. Only with holes in the USB stack itself this isn't true any more, but then that's a whole different problem.

- Firewire, pcmcia-cards, pci* and the like have *real* access. Like, a firewire device can read and manipulate all content in RAM without asking the OS or the CPU or them even noticing. I deactivated Firewire years ago in bios.

- Every USB device needs a custom evil firmware. There never will be malware which can just infect all USB devices it can get hold of. Many devices will be "immune" because their firmware can't be rewritten reasonably, or because the firmware flash memory is already full with the original firmware, no space for evil enhancements. At worst, malware might know and try to re-flash a few of the most common devices. All "sandisk" drives for example. This probably will kill more devices than turn them evil.

- We are, hopefully, not connecting random USB devices to our airgapped system. My initial plan was one USB stick, going back and forth from online to offline system. So *exactly this* device would have to be reflashed, and then this evil device must somehow gain control of the offline system.

- There are "USB switcher" thingies. Connect two computers and one device, and you can switch that device back and forth between those computers. Via button or software. I don't think someone could reflash a USB stick which isn't connected to my (infected online) computer when there is that "USB switcher" in between. Not 100% sure of course.


The bottom line is that we need to get data back and forth between the online and offline system. So it's not *offline* in the strict sense.
Because of how Bitcoin, Armory and transactions work, we can't predict how much data there will need to be transferred, and that data isn't human-readable to check directly.
So, no matter how clever the setup, in the end the user can only tell that, and how much data there is transmitted, and in what direction.

And that's why now, scrapping my initial USB and "USB-switcher" plan, I like that "red and green blinking serial cable" idea.

Ente

[Newar]

Haven't seen it mentioned yet, so I thought I'd let you know that I've successfully transmitted a tx to sign back and forth today using minimodem: http://www.whence.com/minimodem/
https://github.com/kamalmostafa/minimodem

As was mentioned, it took a while, but still quicker than USB back and forth  

It works for both mic / speaker as well as straight 3.5mm jack cable.

Alsa gave a lot better results than Pulseaudio.

[roman.z]

Hi etotheipi,

I've tested the quality of the "Smartphone -> PC" connection, by playing an WAV file (containing modulated audio at 48 kbps) from a Nexus 5, and recorded it from a PC microphone jack - using a standard 3.5" male-to-male cable. I've successfully demodulated a single 16kB file transmission, containing random binary data.

For the opposite direction, I need a special cable that allows me to connect to the microphone of the smartphone, so I've ordered the following cable, that seems to be fit for the job: http://www.aliexpress.com/snapshot/6288177801.html.
I will report my results when I get the cable

Regarding smartphone<->PC communication: it's indeed an interesting problem. I'll think on it...

Smartphone <--> Smartphone, too.  The nice thing about the audio solution is that there's no reason it wouldn't work, you just need the right cables.  And of course an Android version of the audio xfer

Regarding "Smartphone <-> Smartphone":
I think that it may be done easier by some kind of "QR-video" solution, since both of the smartphones should have a camera, and they are quite easy to operate (as compared to desktop screens and webcams). In addition, since we need to achieve ~6kB/s rate, we can use the larger versions of QR codes with quite low fps value. For example the largest version (#40 = 177x177) can contain 2,953 bytes = requiring 2 QR codes per second to achieve the required rate (see http://www.qrcode.com/en/about/version.html for details). The Android application itself will probably look very much like this one: http://stephendnicholas.com/archives/310 (which quite cool IMHO).

[roman.z]

BTW, I've took some time to improve the audio modem library a bit:

• a few optimizations so decoding can run "in parallel", while the audio is being recorded.
• unit test suite with good code coverage (integrated with https://travis-ci.org/romanz/amodem and https://coveralls.io/r/romanz/amodem)
• better (and easier) command line interface for sending and receiving data.
• make the library code to be PEP8 compatible.
• add support for Python 3.
• calibration process now checks all frequencies that are used for transmission.
• improve equalization process, with better handling of signal distortions.
• I am using a hexagonal constellation grid (instead of standard QAM), to improve SNR for existing bit rate -> thus decreasing error probablity

See https://github.com/romanz/amodem for details.