I was thinking about how to remove transactions when all the outputs have been used, but it seems to me that at least the transaction hash must be kept, because without it, it's impossible to tell the difference between an orphan transaction and a double-spend. Although since neither are fully valid, it might make sense to discard them both, and if the orphan later becomes a non-orphan, hope that it will be retransmitted.
I don't see why you need to keep that transaction. The entire state of the network (and everyone's balances) can be determined solely by the set of unused TxOut objects and the hash/index of their parent Tx object. And that's all the information you need to sign new transactions. All the TxIns and previous TxOuts are only necessary for blockchain verification, but that is done by the miners before they include them in a block. Your client can get away with storing just the TxOut information above, and trust that they are valid because they were part of the longest blockchain (which is difficult to fake), and would not be be there if they weren't valid.
You can store all the Tx's in a tree data structure, whose values are arrays of TxOut objects. As TxOut's are spent, you can remove them from the array, saving about 40 bytes for each of them. When the last TxOut in the array is spend, you can also remove the Tx node, which saves another 40 bytes (approx). You would only need to keep that data (and/or its hash) if you were concerned about verifying the transaction history.
