It is because eye contact isn't important to us. It doesn't really enter into our minds to use it. I can't really remember what it was like when I was younger but I would never make eye contact and I would refuse to do something if people were trying to make me. There isn't really another connection shared between an autistic person and another person. At least it doesn't occur naturally. It has to be worked on.
Ask yourself why you make eye contact? I will take a guess that it makes you feel closer to a person. I know from a lot of reading that eye contact can make the area of the brain used for socialising light up, but this is non-existent in autistic people.
By the age of 10 I only made eye contact when someone yelled at me to look them in the eye. All I saw was a very angry person and I see that look every time I look someone in the eye.