i think its mostly personality. im a guy, and i ve met some guys that i just couldnt hang out with all the time, because of the way they act, and because they never wanted to do things i wanted to do. so i think a lot of it has to do with personality, and compatibility. just like a guy, looking for a girlfriend, or girl, looking for a boyfriend.
it could be the type of girls that you meet, or hang out with. you should go for the ones you feel most comfortable with, even if you dont think they wont wanna hang out with you, just go for it anyway, and be friendly with them, just talk to them. everyones different, so some girls will judge you on how you dress, some wont. same thing with guys, some will, some wont. you shouldnt have to change how you dress so much that dont feel comfortable though. you could change the way you dress, but not to the point where you dont feel comfortable anymore. i have a sister, and all her friends seem to dress about the same way though, i havent seen one that seems different enough for me to notice.