With all due respect, this electrical engineer begs to differ. :)
In your own words, "up in the signal chain", which translates to "DAC" these days, use of differential signals does make sense. With modern DAC's, availability of differential signals on the output is practically the norm. The driving force behind that trend is noise rejection to a certain degree, but even more the importance of the amp bridging for the mobile applications. Where their power efficiency rules. My inexpensive Sabaj DA3 is powered from USB, and "balanced" mode of operation makes a huge difference. The wet dream of every DAC manufacturer is to add their device to the BOM of a cellphone. IMHO, the modern source for the headphone amp, now and in the future, is differential.
When it comes to the construction of the amp, differential to single-ended signal conversion is not an issue, and has been solved long time ago. Except an interesting motive pops up in the construction of "M cubed" (M3), headphone amp by AMB, well respected DIY project. That amp has 3 output channels, the common ground is actively driven. And the amp, apparently, works well.
For a long time, the amp bridging was frowned upon because of the inherent distortion. Two channels controlling the differential output are never the same, whatever component matching we do. Hence the distortion. What was neglected for a long time is the fact that with amp bridging, the effective requirement for channel power output is halved. And every amp that I ever listened to showed better linearity and signal quality at lower amplifications. :)
With all due respect, I think that amp bridging, the "ballanced" amp architecture, is likely to be re-evaluated and perfected in the near future. As for the cans, there will be two trends. Either total elimination of the wire in mobile applications, or a drive to 4 wire cabling where the ground is soldered at the connector for 3 pin connectors/traditional amps.