I agree with you. You can still be anti-zionist and hold that the shavous are not binding.
Just to finish off the qoute from Rav Moshe tz”l:
?? ?? ?????? ??? ??????? ??? ?????????, ?”? ?? ?????, ????? ?????? ?? ????? ??????, ????? ?????? ?”? ????? ????? ????. ???? ????? ???? ????? ?? ????? ?????? ?? ????”? ????? ?? ???? ?? ??? ???? ?????, ????? ????? ??.