Emerson was not passionate about abolition. He wasn't a passionate person. He was a cool intellectual, and I think he probably was a little uncomfortable with passionate people, but he was against slavery.
In places like Germany or France the idea of black-white is not so much black-white but "our people and them," and "them" can be people from the near east like Turks or Muslims or North Africans, all of whom might well be considered white in the United States.