Yes, you are taking a position by "not" taking a position. By not doing anything at all, you are helping the pro choice result, which is abortion. By saying "it is your choice" you are pro-abortion.
If you were a German in Germany, during WWII, and you said to a Nazi who asked if it was wrong to kill Jews: "I am not involved in the decision, so I can't say whether it is right or wrong."
What you are doing is subjectifying, object morality. You may or may not think that it is wrong to kill Jews, but you don't want to get involved in the other German's business?
Similarly, you are saying: "I may or may not like abortion, but it's up to you because it is your choice."
Do you see how dangerous this position is?