Definitely, both from the perspective of high quality vs. freebies that come with a phone, as well as the differences between high quality headphones. (e.g., my HD6XX sounds quite different from my DT880).
Personally, I can pretty consistently tell the difference between, say a 128k mp3 and a 320k mp3 that I've ripped.
Where it gets hard for me is around 196 vbr mp3 and above. If I listen hard, I can do a little better than chance, but I mostly want to enjoy my music not focus on finding missing data bits.
Put another way, if I had the choice of spending $200 towards upgrading from 320k mp3 to lossless, or towards more/better headphones, it would be the headphones 100% of the time.