Sunday

For the first time in our adult lives, Brent and I had to learn what you say to someone, when they ask you to do something on Sunday.  I spent the last 7 years in Utah, and 6 of those years I was married.  No one asked us to do anything.  We were discussing what to say, and we realized saying, my parents don't let me do anything on Sunday, we have to go to church, no longer works.  But the last time anyone asked either of us to do something on Sunday, that was the answer.
I kept it simple.  I said something to the effect, Sundays never work for us.  We keep it a quiet day, we just go to church and relax at home.
I was originally asked to join them at a specific time, I said sorry I have church.  Then asked would a different time work.  Which is why I said Sundays never work.

Honestly though...  Any better suggestions?
How do all you non- Utahns who try to keep the Sabbath day holy, tell people you can't go on Sunday?

I decided I didn't want to say, we try to keep the Sabbath day holy by not shopping or spending money, because I didn't want to imply they were sinner.  I have no problem telling them we don't shop or spend money on Sunday, but I didn't want to imply they were sinners by not following one of the ten commandments, I think they are great people.  Growing up I never found it helpful to imply the person was a sinner.  For me the best method was telling me what I chose to believe, what I chose to do and not do, but not make implications for their own lives.  I found people were more receptive that way.
Any better explanations out there?