To the atheist, the conscience is the end product of a long line of evolutionary and social conditioning, so when it bothers them they are not really guilty of anything. As James Sire puts it, they are really only guilty of guilt. This is why we live in a culture of therapy. The goal is to rid ourselves of the feeling of guilt instead of the guilt itself. Unfortunately, we see this being translated into many churches as sin and the cross of Christ are ignored and replaced with a “Christian” self-help regimen. So even under the name of Jesus, empty deceit can hold sway.
See to it that no one takes you captive by philosophy and empty deceit, according to human tradition, according to the elemental spirits of the world, and not according to Christ. –Colossians 2:8-