I disagree entirely with that. The possessive "my" implies that one is referring to his god, and if one is not a Christian, that could be himself, or Brahman, or the human race, or a hundred other things.
And I also disagree that the 10 Commandments, or any other precepts of Christianity, should apply to people who have not accepted it. Our beliefs are not those of the world in general, and to assume that everyone should arbitrarily follow our religious tenets is shockingly oppressive. Beliefs like that are what lead to one-party authoritarian governments that have a state-sponsored religion. If people are forced in to a religion, it ALWAYS ends up hurting the religion just as much as it hurts the people.
And to answer Gob's second question, I would refrain from using it if it offended people (and it does).