Why does my age matter to you?

All you psychologists / social scientists etc out there (and anyone with an opinion), I have a question…
Why does my age matter to you?
Why do people’s attitude towards me change, sometimes quite dramatically when I mention my age?
I travel around a lot, change jobs and places frequently and therefore have to make new friends. It gets harder to make friends as you get older. Both my work and hobby have a majority age of 16-30. 
I often mention my age to people because I find their reactions funny (no way!! You are not 38!!!) I am not ashamed of how old I am.
I do get some very different reactions though.
The under 16s generally don’t treat me differently. I guess they already saw me as old and therefore nothing changes when I put a number on it.
30s + seem to talk to me more. It’s as if a barrier has been lifted. Do they maybe feel they have a little more in common?
16 – 30s majority age at work and theatre circles. The reactions just vary, but not in a good way. All of them say “I thought you were our age”. 
The strongest reaction was someone’s body language started becoming so closed off, they folded their arms and then you could tell they really didn’t want to talk anymore. 
The general reaction is that I initially get on well with people, start getting to know them, and I think I could have some friends, but then after mentioning my age, they talk to me less, and less in depth, more conversations feel as if they are just being polite. 

Anyone else would wonder what they did to offend these people. I often wonder if I did do something, but then I consider when the change happened. I know what I did, I mentioned my age.
So, why do people’s attitude change? Why do they perceive that I am so different? What differences do they think there are? Why does my age really matter??
Have you ever had the same thing?