I do understand how hard it is to get to that point, so I don't hold it against people who have a hard time saying no, but I just want to say that it is definitely worth it to do your best to *try* to become confident and comfortable with expressing "no." It is not selfishness. It is a gift.
When people can say "no" to my "do you want to do this thing?" that makes it safe for me to ask them anything. I don't have to calculate in how difficult it might be for them to say no and include disclaimers and offer them acceptable excuses to make it easier for them to say no. I can just ask, and be sure that if they can't or don't want to do it, they will say no.
It means that I don't have to worry that if I do something they don't like, they will start avoiding me or disliking me. If I can trust them to say "don't do that" or "I don't like that," then I don't have to be constantly watching their reactions to make sure that I am not hurting or upsetting or annoying them. I can just make a joke or be noisy or messy or silly or hyperbolic and know that if they dislike it, they will express "no" in some way that is easy for me to notice.