For basic P2PKH addresses (Bitcoin '1', Litecoin 'L') it should be possible to recover the funds by copying the private key from the Bitcoin wallet to the Litecoin wallet.
so in basic terms, if you send bitcoin to a litecoin address, its possible you can use the bitcoin private key to generate the litecoin wallet with the bitcoin in it? but you still wont get your bitcoin back?? you will now have some litecoin.
or am i reading that wrong? or could you then sign the litecoin wallet and send your bitcoin back to yourself? will the core allow that?
The former is correct. A private key is just a sequence of 32 bytes, and a public key hash (which the address is derived from) is just a sequence of 20 bytes, so it's up to the client software to check that the key and address are associated with the correct network. As stated above, internally the representation does not use anything identifying the network; it's only when base58 encoding a "human readable" address or WIF privkey that the network prefix is added.
This is explained (in some detail) at
https://en.bitcoin.it/wiki/Technical_background_of_version_1_Bitcoin_addresses . Steps 4 to 9 are solely for converting the internal 20 byte public key hash to base58 format, which
does denote the correct network, but the client (and blockchain) only use the result of step 3, which has no network information attached.
This means it's possible to use the same private key across multiple coins/blockchains (a brainwallet is one example of doing this) but of course the funds cannot be sent across blockchains.
it really seems that when altcoins were created, they core devs should have patched in a check that would make invalid tx's fail and those coins bounced back..
Bitcoin Core (and most likely Litecoin Core) does check that the address is valid when you attempt to send funds, so something else in the ATM system is decoding that address without verifying it is valid for the currency the user has selected. Since the ATM supports multiple currencies, it probably uses custom software. Or maybe it just phones home to a backend that does everything. Either way, it seems something is blindly decoding the receive address without checking that the first byte matches the network the user has selected.
