In "Priceless, the Myth of Fair Value," William Poundstone attempts to explain this phenomenon.
First he mentions the advent of the "99-Cent store" the first of which was created by a shopkeeper who noticed sales increased when prices were 99-cents despite the same item being even cheaper before.
One theory as to the beginning of this phenomenon dates back to British colonialism in America. Conversion from British shillings produced an odd-penny prices in local currency, so the strange prices were associated with higher quality imported goods.
Another theory comes from the invention of the cash register. Since change could only be made after the sales amount was punched into the machine (and recorded for later review by the shopkeeper), odd prices made it more difficult for employees to sell items on the sly and pocket the cash. Unless that employee had a pocket full of change.
Though he admits that neither of these explanations (if even valid) would explain why specifically prices ending in 9 (called "charm prices") are so popular.
A experiment carried out at the University of Chicago found that when different versions of women's clothing catalogs were sent to a random sample of people, the same item would sell better at $39 than at $34.
This is interesting especially because it partly debunks the "mental rounding down" that allegedly happens when you see a price ending in a 9.
Some people have come to associate it with things being marked down or somehow discount, and studies showed that higher end brands see less benefit from charm prices. A study showed that a charm priced item sold similarly well to a non-charm priced item that had an explicitly called out sale price (like "$40, reg $48").
I know when I worked at Circuit City, the status of the item was sometimes coded into the price. $x.99 was normal price while $x.97 was clearance, etc.
Ultimately, we do it because it works but there's no definitive answer as to why it works.