You neatly summarized the whole mess with cameos given to the usual solutions acknowledging that none of them work. Technology, and managing to keep the lid on the problem groups within the general population is our only hope. Eventually the most difficult and recalcitrant groups die off but only to be replaced by newer difficulties and their recalcitrant groups.

The opportunities for innovation to our problems lie in hybridization by random matching of the best pieces of the least egregious failures from our past. This will need to be done with AI and ML for the less than obvious stuff that may be working but is invisible to the mass population.

With luck we will succeed in creating machines that can manage us while never letting us figure out that we are being managed. That will be the planetary administration of policies that we only deal with in the context of our own small, city states.

But the odds are that we have already screwed up too much and will suffer a significant die-off. That may trigger an epigenetic change that becomes a species bifurcation. That would be the ultimate effect of political polarization.

I don't expect to see that but it's fun to think about.