I'm speculating, but it might be because HIFI headphone gear and amps are "affordable" within sub $5k. Once you get into passive speakers and accompanying amps for those, in the HIFI market, you're looking at a much greater barrier to entry. And, no one really wants to talk about mid-fi or consumer level passive speaker systems. The affordability of headphones also pushed it in the forefront of the audiophile community, and the advent of portable players helped with it as well. There are a lot more people building listening environments around the isolated experience of headphones and less and less about a listening room with speaker cabinets.