Ah, so I guess the problem there isn't bytes vs. vbytes. It's that Core's fee estimation (like Electrum) is bad. Which leads me back to square one: How are people estimating their fees with Segwit? There's gotta be a better way.
Bad fee estimation must be part of the problem during periods of heavy spam and congestion.
On Electrum I always use manual fees as ALL fee estimators result in overpaying. I use the preview transaction function as it displays the size in bytes which is actually vbytes and multiply that by what fee per byte I want to pay. Then adjust the fee and preview again and the sat/byte number will be accurate.
(3.0.0 did slightly underestimate the size but that bug has been fixed.)
For actually working out what fee I want to use
https://dedi.jochen-hoenicke.de/queue/more/#2h shows what fee levels are currently being included in block far better than any other site I have found.