Why did they choose such a format?

Why not free a top level IP (e.g 200 as an example) and then whenever that byte is selected, you can have variable length fields:
200.num_of_bytes.89.123.42.76.43.108.2

Or change each each number field to a 64 bit int.

7843843.9084394.12333232.89

Therefore since some sites need IP addresses, ISPs and router manufacturers would lose out by not providing access to those addresses. ipv6 is ridiculous made by academics. Do they honestly believe the whole internet to upgrade to a non-backwards compatible address?

I think that when they sat down and tried to figure out a neat way to do this they quickly realised that legacy support=more code=more bugs=more exploits and less reliability.

IPv6 was a chance to re-imagine the internet all over again. (And lets face it, the internet is still in it infancy, we don't even have a full set of working browsers yes. ), and they have done everything they can to make sure these problems don't come up again.

The whole Internet will upgrade. IPv4 is reaching its end of useful life. End of story.

Nah, I dont think its hard to imagine a cross compatible protocol with IPv4, it just would have needed to be implemented top-down, assigning "IPvX" a huge subnet of availabe IPv4 addresses, with ISP's providing transparent bridging. Eventually enough Devices would be in the new subnet and the Ipv4 bridging can be cut and the rest of the new address space opened up.

Anyway the moment is past, IPv6 is what we got, so any type of seemless migration is impossible.