So each private key is a string which I sign with my wallet address to send from. Is that correct?
No.
Each private key is a random 256 bit number. These numbers are used by the wallet to calculate signatures for transactions after the wallet creates the transactions.
What is this 'signing' process?
It is an ECDSA signature using the Secp256k1 curve.
I guess it needs to be one way or else everyone can derive your private keys, so this must be shorthand for 'using a hashing algorithm with the private key as the input and verifying the output is the address you are trying to use'. Is this understanding correct? Or over-simplified?
Yes, it is one way as long as the signature is calculated properly using a random value for k.
Oversimplified.
Suppose Alice wants to send a signed message to Bob. Initially, they must agree on the curve parameters (
CURVE, G, n). In addition to the field and equation of the curve, we need
G, a base point of prime order on the curve;
n is the multiplicative order of the point
G.
Alice creates a key pair, consisting of a private key integer
dA, randomly selected in the interval [1,
n-1]; and a public key curve point
QA =
dA *
G. We use * to denote elliptic curve point multiplication by a scalar.
For Alice to sign a message
m, she follows these steps:
Calculate
e = HASH(
m), where HASH is a cryptographic hash function.
Let
z be the
Ln leftmost bits of
e, where
Ln is the bit length of the group order
n.
Select a random integer
k from [1,
n-1].
Calculate the curve point (
x1, y1) =
k *
G.
Calculate
r =
x1 mod
n. If
r = 0, go back to step 3.
Calculate
s =
k-1(
z +
rdA) mod
n. If
s = 0, go back to step 3.
The signature is the pair (
r, s).
What format is this .dat file?
levelDB
Is it simply a list of private key strings?
No, there is additional information in there.
What is the length of each key?
256 bits
Is it fixed?
It is not a string. It is a 256 bit number.
Is it some sort of open-source database structure?
levelDB
Or just a flat file?
No.
In the future could your private keys be something other than a list of strings?
They are not a list of strings. They are 256 bit numbers. This could change if the signature algorithm is changed, but so long as we continue to use the current ECDSA algorithm with the Secp256k1 curve, private keys will continue to be 256 bit numbers.
Like presumably you could hash the raw bytes of a favourite photo or something? Wildly insecure probably, but anyway, off-topic...
Yes, you can already do this. Hashing a favorite photo with SHA-256 will result in a 256 bit number that can typically be used as a private key.
As you said, this is "Wildly insecure".
Until I encrypt the wallet using the function in bitcoin-qt - is the wallet essentially wide open for anyone with access to the file to read my private keys?
Yes.
Why is wallet file encryption optional then?
Freedom of choice for the user to be as stupid as they like.
Truecrypt offers real on-the-fly volume encryption, so I could store my wallet in there indefinitely, but I can't find a way to point bitcoin-qt to use my encrypted volume to look for my wallet instead of the default /home/.~bitcoinqt or whatever it is. I see there's a datadirectory (or something?) parameter, but I don't want to carry around a 12Gb (and counting) volume and have to keep syncing the blockchain to it. Is there also a 'walletdirectory' or something parameter? That would be awesome...
No. At this time, there is no "walletdirectory" parameter.
I don't know much about Truecrypt. Perhaps you could create a symlink for the wallet.dat file between your datadirectory and your Truecrypt volume?