GronchoI used to do that too. I don't know about damaging it but I think it cause more wear on the spring. If you twist it the correct way it comes off smooth and easy. You can feel more resistance if you twist the wrong way. You can hear some grinding from twisting the wrong way.
This how I release with one hand. This only works if you're a righty. You'll have to take out the spring and invert it if you're a lefty.