The Nazis went after non-religious Jews, so it was always primarily about race. That was the argument Hitler was making, was that the "superior" "Aryan race" was being poluted by Jews and blacks and other "lesser races." Of course, Nazis went after anyone who didn't support them, too.