I believe that what happens is that teams playing away from home have to travel to the opponent's city, which can tire the players and they don't have much time to rest from the trip because they need to train. Another point has to do with the pitch's grass, the weather, and lastly, the pressure from the fans who are cheering for the home team.
That's what I think; it's never just the fans, there must be those other variables you mentioned. But even if the home teams were the strongest, this always happens in general. Obviously, it's not a rule, but it's a tendency that always repeats itself.
The visiting teams have no other choice, this is the risk that all players are forced to be ready and able to bring full points, at least a draw is what the visiting team hopes for. The recurring tendency of traveling to the opponent's city reduces the physical condition of the players because of fatigue.
In the match, everyone relies on the home team while the visiting team does not, even though this season many visiting teams have won, in this case, of course, there are other variables that must be related.