The El Ni\~no-Southern Oscillation (ENSO) is a mode of interannual variability in the coupled equatorial ocean/atmosphere Pacific. El Ni\~no describes a state in which sea surface temperatures in the eastern Pacific increase and upwelling of colder, deep waters diminishes. El Ni\~no events typically peak in boreal winter, but their strength varies irregularly on decadal time scales. There were exceptionally strong El Ni\~no events in 1982-83, 1997-98 and 2015-16 that affected weather on a global scale. Widely publicized forecasts in 2014 predicted that the 2015-16 event would occur a year earlier. Predicting the strength of El Ni\~no is a matter of practical concern due to its effects on hydroclimate and agriculture around the world. This paper presents a new robust mechanism limiting the predictability of strong ENSO events: the existence of an irregular switching between an oscillatory state that has strong El Ni\~no events and a chaotic state that lacks strong events, which can be induced by very weak seasonal forcing or noise.