The public and private keys are both generated offline. So even before I generated the wallet, this public address already exists and coins can already be sent to it?
This public address is simply an identifier that does 'exist' even without anyone having a private key. For example, you could send coins to 1thisaddressdoesntexist but in reality it's highly improbable that someone will have the private key corresponding to that address to spend coins there.
Simply because I am the one who holds this particular private key that is associated with the public key? If so, what is preventing someone else who is also creating a wallet at the same time from generating the same keys? Just mathematical improbability? I think this is what I really don't understand. Thanks all!
Basically, you generate a private key and from there it undergoes a few mathematical functions (trapdoor functions to be specific) to produce a public key. They are linked together. From a private key one can work out an address, but with an address one cannot work out a private key (hence trapdoor function - google them, they form the basis of cryptography).
It is incredibly improbable. You'd be more likely to win the lottery 10 times in a row then having an address collision.
Statistically speaking, unless the protocol changes to accommodate more decimal places, only 2.1e14 addresses could contain at least one Satoshi, and that's only if everyone only had one Satoshi. If anyone has more (and pretty much everyone who has any has more than one Satoshi), then there are fewer occupied wallets.
Within the set of 2256 private keys, they only map to 2160 unique wallet addresses. So the question is how does 2160 compare to 2.1e14? One in a million? One in a trillion?
The answer is one in 6.9595 decillion. Since "decillion" isn't a commonly used word, I'll save you the bother of having to look it up: it's a one with 33 zeroes after it.
To put that 6.9595 decillion figure into perspective: The Earth has a diameter of 12,742 kilometers, giving it a surface area just shy of 50 million square kilometers. A square kilometer is 1 million square meters, and a square meter is one million square millimeters, meaning the surface area of the Earth, in millimeters, is just shy of 50 quintillion mm2.
So here's the game we'll play. I've got 140 trillion earth-sized spheres. On one of them, I have randomly selected a single square millimeter as the prize winning spot. Find it, and you'll get to spin the prize wheel to see how much you've won. The prize wheel currently has about 22 million spaces. 21 million of them contain less than a dollar. But you only get to spin the wheel if you can find the secret spot on the secret sphere.
Wanna play?