How to deal with this situation?

It sounds like he is afraid of the way he might be treated if he gets too close to you. I experience the same thing when people find out I am not Christian (I live in the U.S."Bible Belt".) My experience with cross-cultural relationships is that they have a very high mortality rate since the people involved tend to be shunned by the people of both cultures. They find they no longer have a home anywhere. Sorry I can't give you something more positive.