Don't beat yourself up. According to Princeton University professor, Susan Fiske, that feeling is hardwired into all of us.
When our primordial ancestors dropped from the trees and started walking across the African savannah on two legs, survival favored those who had an innate ability to work in small groups, as well as a deep hostility towards anyone not of the group.
Those characteristics were so critical that, over a number of generations, reinforced by tribal warfare, they became the norm. And they survive to this day in the biases that we all feel.
Fiske and other researchers have seen their tracks in surveys and fMRI studies around the world. Tons of research shows that bias operates unconsciously and automatically in almost everyone. That's the bad news.
The good news is that biology is not destiny. Research also shows that our prejudices are not inevitable; they are actually quite malleable, shaped by an ever-changing mix of cultural beliefs and social circumstances.
For example, when Fiske asked people to imagine that they were running a soup kitchen and had to decide what the homeless man above might like to eat, their feelings toward him changed. They began to see him an an individual, rather than as an exemplar of the "homeless."
"While we may be hardwired to harbor prejudices against those who seem different or unfamiliar to us," Fiske says, "it’s possible to override our worst impulses and reduce these prejudices. Doing so requires more than just good intentions; it requires broad social efforts to challenge stereotypes and get people to work together across group lines. But a vital first step is learning about the biological and psychological roots of prejudice."
An article she wrote for The Greater Good Science Center at the University of Berkeley is a terrific primer on the subject.