The recently observed correlation between HiRes stereo cosmic ray events with energies E ~ 10 EeV and BL Lacs occurs at an angle which strongly suggests that the primary particles are neutral. We analyze whether this correlation, if not a statistical fluctuation, can be explained within the Standard Model, i.e., assuming only known particles and interactions. We have not found a plausible process which can account for these correlations. The mechanism which comes closest -- the conversion of protons into neutrons in the IR background of our Galaxy -- still under-produces the required flux of neutral particles by about 2 orders of magnitude. The situation is different at E ~ 100 EeV where the flux of cosmic rays at Earth may contain up to a few percent of neutrons pointing back to the extragalactic sources.