I don't know if you've managed to fix your issues by now with PS3 capture over Component.
I have posted this before, but I will do it again just in case:
SETUP OF CABLES
Step 1: Check that you have a PS3 component cable. This cable contains 1 PlayStation plug, and 5 analog plugs, generally in the colors: RED, BLUE, GREEN for video, WHITE, RED for Audio.
Step 2: Just to be on the safe side, make sure that you did not mix up the red analog plugs and that all plugs are plugged in and secured firmly into your GameCap HD Pro.
Step 3: Connect a male-male component and audio cable from your GameCap HD Pro to your TV's component input
*Note: I noticed recently that the audio in and audio out on the gamecap pro are not mirrored! Make sure that with your audio cables, white mirrors red and red mirrors with white to make sure that your stereo mix isn't switched around!
(If this confuses you, I mean to say: make sure that all plugs are in the right color plug.)
Make sure your GCP HD is plugged in and correctly installed on your computer with the latest updates. the led light on the capture device should light up purple, not red.
SETUP OF PLAYSTATION 3 SYSTEM
Step 1: After turning your PS3 on, go to system settings, then display settings.
Step 2: Select output method and set it to Component. in the resolution selection, the most basic resolution is not unselectable. Apart from this, ONLY check 720p, then continue. If this displays the correct image then you should be good to go.
Step 3: Load up the capture software provided with your card, if needed set the input to Component, and check the options to make sure that 720p mode is selected.
Setting the resolution higher really isn't that useful. Most PS3 games don't go past 720p, with the exception of some titles. Sometimes, a game's box claims it's 1080p, but this isn't always true.
Besides, when your system's resolution is 1080i or 1080p, switching to a game that has 720p resolution might reset the capture resolution to 480p.
So again, setting the video output to 720p may look worse, but for games and capturing the footage, this is your best bet.
A word of advise: Make sure you always load up the game first, then check the capture resolution before you start playing and recording.
Do note that footage captured over component may look bland. To get the best results, I found these settings worked best for me:
Hue: 128 (default)
These settings generally work best when you set the game's gamma/brightness adjustment (back) to default.
As someone said earlier, there is a vast difference between hdmi and component and their quality.
As component is analog, you may see translucent artefacts in your footage. HDMI is a lot clearer.
To capture PS3 footage over HDMI, you will need to evade HDCP.
You can do this with either of the 2 options: a HDMI splitter, or the Monoprice DVI-D/Toslink to HDMI converter. The latter is more expensive, but is proven to work with all HDMI capable consoles.
With the monoprice you need to keep the following in mind: It only comes with a US power supply. For best results for people outside the US, you should use a travel power converter.
You will need the following cables: 1 HDMI to DVI-D cable, 1 toslink (or optical male-male) cable, 1 HDMI cable to run from the monoprice to the GCP, then 1 more HDMI cable to go from GCP to your TV.
You will also need to set your audio output to "optical" and disable all optional output methods. linear PCM is best for this.
If you wish to use a different sound source (like analog), you can set "multiple audio output" on from the PS3 settings menu.
However, a fair warning, the Roxio GCHDP has difficulty with transitions in games between loadscreens, cutscenes and gameplay while capturing over hdmi. There is no 100% fix for this issue. Monitoring your capture on a separate screen and stop the capture every now and then.
I am not responsible for purchases made by you based on my advice. What worked for me might not work for you. I will also not be held responsible for abuse of the HDCP evasion by others.