My rebuttal to this is (they usually include murder as the bad thing being done), “Are you telling me fear of hell is the only thing keeping you from murdering me right now? Says a lot more about your morals than mine.”