Well, at this point, I just don't know what the source of the noise is. Since playing the tape directly through your TV doesn't introduce the noise, we have to assume it's either in the Roxio video capture device, or a combination of that and the Mac. I was forgetting that you're on a Mac, and Goldwave is a Windows application. You've tried a different Mac, and a different USB power supply, so we've sort of isolated it to the Roxio device.
Once again, what we're forced to recognize is that no matter how "Easy" something may be described as being, there's often much too much to learn. At this point, I don't have a good solution for you, without you having to get into a little bit of audio editing. As I say, most audio editors can create a notch filter that would take out the objectionable noise from your video. But that's an extra couple of steps (extract the audio, filter it, add it back to your video).
You could try contacting Roxio and explain the problem (point them to this thread), but I doubt you'll get much satisfaction from Roxio. I'd be as tempted to look for another device, possibly find a less-expensive used one off ebay. Then we could see if it's your specific unit, or an overall design problem, but you're the first person to mention a high frequency noise problem, so there's hope it's the device, not the design of the device.
Sorry I don't have a better idea. Maybe someone else will put one forth!