Hmmm, It's hard to see what piece of hardware isn't working properly - xbox, RGC or maybe your computer. I used to have the same problem but found if i swapped the component cables around so the one that was originally going from roxio to T.V was now going from roxio to console, I have also seen a fair few of posts relating to the new slim version of the 360 having the "no signal" fuss, I have both xbox's and my slim seems to hate the roxio. Have you tried using a different t.v? or maybe have another computer to install the software on?
Im no epert on this but give it a few hours for jim to see your post and I'm sure you'll be filming in no time!! Is there a youtube channel for me to check out? always nice to watch people who are also using the RGC