Fonts, from bytes to glyphs
Download, parse, match, shape, raster. The stage that surprises people is shaping, and it is the one you cannot skip.
A web font is not a picture of letters. It is a program of outlines plus tables describing how they combine, and shaping is the step that turns a string into positioned glyphs using those tables.
That is why subsetting by character range is safe and subsetting by removing tables is not: drop the wrong table and your ligatures, kerning or Indic scripts stop working while every letter still renders.
font-display controls the gap, not the cost
Swap, fallback and optional decide what a reader sees while the font loads. None of them make the font arrive sooner.