Exactly the opposite! You will get infinite Schar for being Machmir like Rav Schacter who holds all cows are Safek Traifos currently and does not drink milk.
On the contrary… because I’m not doing it lishmah. I’m not staying away from milk because I’m concerned about terifos… I’m doing it for my own selfish, petty reasons.
In fact, one can argue that I’m insulting HKBH (so to speak) by saying (metaphorically) that His milk isn’t good enough for me. For that, certainly, I’m not earning any points in Heaven.
The Wolf