To play on a regular CD player, you need to create an Audio CD, which I believe you can do with Windows Media Player. Don't ask me how, because Windows Media Player and I do NOT get along well. But I believe if you put a blank CD into your computer, that is actually one of the options that comes up, to create a disc with Windows Media Player. Someone else here may jump in and give you the steps to do that.
By putting .WMA files onto a disc, you're creating a Data disc, which, as you've found, plays okay on your blueray player. That may not be true for every player, and certainly not for a plain-jane CD player. Of course, an Audio CD will only hold about 80 minutes of audio because it is decompressed to full CD-DA quality, where if you just put the .WMA files on as a Data CD, you can probably put many more hours on, but again, that won't be playable in every player.
To put your VHS onto DVD, you'll need some actual hardware to capture the video. You might consider Roxio's Easy VHS to DVD 3 package. Unless you've got a video card with a Composite signal input (unusual), your computer won't have the hardware to capture video. We can talk about that after you've got hardware to do it.