Yes, they play in the background. It's called TSR (terminate and stay running). Just like a smart phone. That specific one has "in product ads" such as a phony "Windows has issues, click here" box that I just ignore. I also use Grover Podcast from the Windows store, load my podcasts, and right now I am listening to The Skeptics Guide to the Universe (new episodes drop every Sat) and have 20 or so listed here, along with many of them on my Squeezebox radios. These newer tools even find the podcast for you and then mark it Subscribed, so the new episodes just show up. I don't download or store anything, either.

Pandora and Spotify rely on playlists and randomized content. You also can use TuneIn for free, which I set on my smart phone for the alarm app, to wake up to my MT radio station while in FL, for example. The Squeezebox system used radiotime, which turned into TuneIn. That also has a desktop app portal, and you can simply use their webpage portal, if you don't want to load anything on the computer. The apps have a nicer interface, though. My favorites from TuneIn on the Squeezeboxes show in the Favorites on the computer and the smart phone.

And I love podcast players that let me listen at 1.4 speed.

I am self-admittedly over-entertained.