You have to struggle alone and in silence against a distracting social environment which looks askance at your solitary quest. This quest engenders psychological powers [in and against the self] of an intensely dedicated and aggressive kind. From the point of view of the solitary quester with his inner direction and self-appointed goals and standards, it is society that is THE LONELY CROWD.
McLuhan’s use of masculine pronouns in regard to human beings in general (“the solitary quester with his inner direction and self-appointed goals”) is retained here because of its implication of his own situation. His use of ‘you’ has the same effect.