As an American, not usually. Depends on where I am. Around my own city or something like that I would assume they speak English in most cases.
If I was at a theme park or other touristy place, I probably wouldn't. I also wouldn't if I was on a farm out here, since a lot of the Mexican and Portugese workers don't speak English.
There's so much diversity where I live, I see "foreign" looking people every single day and yet they're usually still Americans just like me.