Honestly, I think unfortunately gender/sex does play a factor, in addition to race. If this administration has taught us anything, is that there is that much hate within our country.
Also think of cultures where historically their culture doesnt value women. Even if there are people who immigrated here, some still may never vote for a woman. Some will decline because they are racist. While we are all Americans, we are deeply deeply divided ATM :(
This is without even factoring the candidates political platform in yet.
He has a book called Paddle Your Own Canoe where he talks a lot about this.
I did the audiobook and its preachy but not a bad listen. Essentially, dont be a shitty human.