Indeed. Though I doubt pithy classification schemes that pigeonhole a person into only one of two binary categories, I largely agree with you, and hereby publicly identify myself as belonging to the latter of your two schools of thought. However, I do have to take minor exception with your last statement - that the two schools "invariably" believe that the other is worthless. At least, as far as it pertains to me and my comment above. Again, this should be taken as exploration and elaboration of your stated premise rather than disagreement with it.
The people who belong to the former school - the "scholars", if you will - are those who have deliberately expended some time and effort in learning the social, artistic, political, etc. context in which the art was created. You're not born with that knowledge. Those who have not deliberately chosen to expend that time and effort fall into the latter school - the "appreciators" - by default. They approach the art naked, unencumbered. But also without the tools to appreciate some of the deeper aspects of art. Naturally they will miss some of the nuance that is available to the scholars.
For me, there is just too much art for me to have ever studied much of it in detail. Especially music. Having been a jazz musician in my youth, I have a quite deep understanding about the theory and practice of jazz, for example. But I do not have the same depth of knowledge of metal, because I have never spent much time studying it. I can't study everything. No-one can.
So when I listen to new music, I try to react on a purely aesthetic level at first. Sometimes, my enjoyment of a thing will warrant my deciding to spend some more effort understanding it in context. If I think it's good enough to trigger some intellectual motivation. Otherwise, I've got other things to spend my intellectual capital on.
Certainly there are people whom your statement does describe, and I in no way want to dispute that. But I thought it might be helpful to put some of my thoughts on the subject into the open.