Personally, I don't recall ever receiving Bitcoin by accident, or due to a wrong transfer. Such cases may have happened to others, but I think it only happened because someone had saved the Bitcoin address, making it plausible. Well, I have experienced this before, but not with Bitcoin (with shitcoin address). At that time, my friend wanted to send some shitcoin to someone else, but instead, he sent it to my address. So, from that case, I figured it was impossible for an unknown person (out of nowhere) to send Bitcoin to another person (unknown) address without any prior interaction, or without having saved the destination address previously.
What you said makes sense and that is why you will hardly hear that somebody sent Bitcoin to a wrong address or that another person received Bitcoin from an unknown sources. If it happens it's always from a saved address where if you want to send to a particular address you will mistakenly send to another, aside from internal mistakes it is quite difficult to here where a Bitcoin is sent to an address that doesn't have any affiliate with you.
I've never received Bitcoin by mistake before but I've mistakenly sent to a wrong address which I copied and luckily it belonged to my friend and we settled the matter amicably. So it's not quite possible to send to an address that is not saved in your device, still there is need to be very careful because you can mistakenly send to somebody you know that won't be willing to send it back to you, remember that Bitcoin transactions are irreversible.