And this "fleeting material infatuation" is less important than the other things people base fandom on? Civic pride, the team your dad raised you to cheer for, your favorite player... how are any of these things more true, more important, more "real" than aesthetic appreciation? All sports fandom... repeat, ALL sports fandom is fleeting infatuation. They wear your city's name on their shirts? Not if the billionaire who owns them moves them next week. The team your father/uncle/grandfather loved? I bet there's a lot of things they tried to pass on to you that you rejected. And hero worship of an untitled 21 year old who wouldn't bother himself to acknowledge your very existence, and will leave at the first chance for more money? If there is a more childish reason to devote yourself to a team, I haven't heard it.
If you think basing your choice of favorite teams on their uniforms is childish and/or stupid, fine. I'm not even sure I can argue. But I'm still waiting for someone to offer up a reason that's any better.