Let's say two miners are trying to mine for the next block. The first one finds a block, and broadcasts it. The second one, on the other side of the world, doesn't hear of it for a minute or two, and in the meantime, finds another block off the same parent. They broadcast, but it's too late; the network's already seen the first block, and soon enough it's got a good few blocks following it. The second miner found a block, but it didn't make it into the blockchain. That's whatcha call an "orphan".
And a particular miner's chances of getting into that situation? That's the "orphan rate".