I think I struggle with podcasts, too; not entirely sure why. I actually do enjoy doing some of those repetitive tasks (like knitting) to the sound of the voice of a crafty English lady on YouTube who's often just chattering away about her life but quite relaxing...
But otherwise I'm much likelier to listen to Dire Straits / Mark Knopfler when doing chores at home, than to spoken word... *shrug*

Plus, yes, I hadn't realised fully but there's a difference between audiobooks and radio plays for me as well, probably for the same reason you mentioned. :-)