You will always have those that are FOR you and AGAINST you. This is how it will be no matter if you want to accept or deny it. It is what it is and will inevitably be. Regardless of which it is, you can't allow it to stop who you are destined to be. Not all are capable of supporting you in the capacity that you need or desire.
It's completely natural to want to have a line up of a few or many that support you. Even me, that is telling you this desires some type of support. It could be support that looks something like "I got your back no matter what" OR something like "I know you can do better, so you need to keep going". True support holds you accountable while also desiring the very best for you. You can't want support and then completely dictate the process. That's just not how it works. The way you support others may not be reciprocated. In fact, the type of support you give may not be the support you need.
Now on to the absence of support. Do not let the support that you don't have overshadow the support that you do have. Think about it like this $1 is better than $0. Sure, you can't buy much with $1, but you know for a fact that you can't buy anything with $0. Got it?
Regardless, support shouldn't sway your goals. Do your best in the attempt to be your biggest fan, your loudest cheerer. Sometimes consuming your thoughts with the presence of support will cloud your judgement and potential to do greater.