It depends.

It isn't that yarn in itself is expesive, but if you're knitting/weaving, you're not doing it to save money on socks, you want to make something cool and unique. If you really get into it, you're going to eventually want that specialist wool/bamboo/elastane blend with a super specific colour grade and maybe a specific manufacturing method too. And that's expensive.

Similarly, if you're spinning your own yarn, you can get boring old for about half the price of boring old yarn, and even less if you dye big batches yourself. You can get a pretty nice wool for about a quarter of the price of the yarn, so far so good. But of course, if you're spinning your own yarn, you're going not doing that for production purposes, you want to make something cool and unique. So you'll want to blend in specifics, like glitter nylons, or maybe even metalic fibers, and that long-fiber, ultra-fine angora will go great with a slightly thicker cairngorn, etc etc. And before you know it, you're making yarn that costs maybe ten times what they sell at the local hobby shop.

And spinning wheels aren't exactly cheap either. Mine was something like 800 euros, but you can easily spend four times that on an electric wheel. You can buy a LOT of yarn for that money. And lets not talk about how much wool I've ruined due to lack of skill while learning.

Or, if you want to do it for historical purposes, you're going to want kinda-shitty, historically accurate materials like hemp or flax or wool from sheep that aren't really all that suited for wool-making, and are probably not even kept anywhere anymore outside of niche hobby flocks. And then you want to process it yourself. And it's surprisingly hard to fine someone who will just sell you flax-the-plant.

It’s not considering the value of my time; a decent (actually wearable) yarn is far more expensive than most people think.

I would consider it a waste of my time to spend a couple hundred hours on a garment that’s barely wearable because it’s uncomfortable and borderline not washable. That’s what you will get with any yarn that won’t cost you over $50 in materials for a simple pattern.

Cheap yarns are fine for beginner projects that aren’t made to be worn, but if you’re putting that much of your effort into a garment meant to be used, you should not be using bargain yarn. Your effort is worth too much to sabotage yourself that way.

eta: oh, if you’re wondering (like I did) why knitting something in polyester would be different from store-bought garments in what seems like the same material, it’s mostly in the weight of the yarn, and partly in how insanely uniform machine knitting is. That creates a radically different fabric than even the most skilled human could produce, and small deviations in either yarn weight or technique have radical differences in the fabric. There are knitting techniques that produce highly artistic texture by doing nothing but varying yarn tension.

I tried getting into knitting once, and really appreciated that the entry bar is so cheap. I never made the jump for more expensive yarn, life got in the way and stopped.