Recent phylogenetic analyses indicate that RNA virus populations carry a significant deleterious mutation load. This mutation load has the potential to shape patterns of adaptive evolution via genetic linkage to beneficial mutations. Here, we examine the effect of deleterious mutations on patterns of influenza A subtype H3N2's antigenic evolution in humans. By first analyzing simple models of influenza that incorporate a mutation load, we show that deleterious mutations, as expected, act to slow the virus's rate of antigenic evolution, while making it more punctuated in nature. These models further predict three distinct molecular pathways by which antigenic cluster transitions occur, and we find phylogenetic patterns consistent with each of these pathways in influenza virus sequences. Simulations of a more complex phylodynamic model further indicate that antigenic mutations act in concert with deleterious mutations to reproduce influenza's spindly hemagglutinin phylogeny, co-circulation of antigenic variants, and high annual attack rates.