There are atheist chaplains. Hell, there are probably atheist pastors, ministers, preachers and priests. Think of the class as a rigorous, specifically themed drama class--your job is to act a part. Lots of religion is theater. Whether or not any of the teachings are "true" is beside the point.
Look at it this way: religion is make-believe. Acting like you believe in religion is also make-believe. So pretending to believe in it is really not that different from actually believing in it; it just places the emphasis on the fiction of your belief instead of on the fiction of the religion itself.