Be proud! (not proud that you have an OTD brother) Proud you are frum. Proud that you know what is right.
Proud that you have a brother although OTD he is still alive– Your brother probably does not need a refuah (unless he has a mental issue)
If someone bothers you about your brother, reply by asking in a serious way, if they know of any way that they can help him. Maybe they know a Rov, a school or better a good job for him with a frum employer. You will see how fast they will back off.
Good Luck and stay strong (unfortunately there are too many people in your boat 🙁 )