Why does everyone say this?

Everyone says that my marriage will not work out and they think that except for him and I. For the longest time we have been super happy and we just all around love eachother. We have been unfaithful before but forgave eachother and don't do those things any longer. We have our fights like every other couple and they worry about our offspring for genetically ping on hypertension and diabetes. I am obese and he is overweight but idk why everyone adds weight into why we will not work out. I love him so much! He has autism of some sort but it makes him even more fun and he is so special to me. I love to play peek-a-boo with him and blow raspberries on his stomach and tickle him and watch him laugh. We have both had depression before and we have been through just about everything and we are holding tight. Our backgrounds are different but not by much and our families are way different. Why does everyone say this if I feel like I have been so blessed and he does too? Why are people hurting us? What should I do?