An Audio CD does not contain cover art. If Windows Media Player, or another player application is showing you cover art, it is because it is searching online, and finding a match for the CD that is being played, and it is then downloading that cover art, and possibly the track/title information. When you write an Audio CD, if you have it set to write CD-Text data, then the artist-title-track information is included on the CD, but Windows media player does NOT read CD-Text data. Again, it gets it from the internet after matching information about the CD (number of tracks, their length, etc.) and downloading it. Your CD, unless you've simply copied a commercial CD, won't match any searches.
If you check with some of the on-line databases (Gracenote, etc.), you can actually upload the information about your CD. I don't know what it takes to do that, it may involve money, it may not, I've never attempted, or wanted, to do that.
MP3 files can point to cover art. If you download an MP3 album, it may include cover artwork, and be pointed to by those MP3 files. But the Red Book Audio CD specification doesn't contain any provisions for cover art.
Hope that helps!