For me, it happen too and it's annoying.
In my case, it's a kind of feedback from the usb. If I move the mouse, the sound will follow.
I know that Macbook are good with audio. For windows, it depends on the manufacturer of the motherboard.
To fix this, this is maybe a ground loop problem, so you will need a gound loop isolator.
If not, you can try something else or cover your cable and joint with aluminum foil : )
For me, I will purchase...