Immunity of intersubband polaritons to inhomogeneous broadening