What will we become or, more accurately, what are we becoming? We are in process but our attention is being dominated by those who do not want to change or refuse to accept that the changes are the result of our previous decisions as a species. We are traveling into the ‘damned if we do’ and ‘damned if we don’t’ world. While the popular fear of AI/ML is that we will become irrelevant or a nuisance to be dispensed with, I think that is, itself, the problem we need to overcome. Not that those things can’t happen but I think the reality will be whether or not we choose to grow to accept our limitations. We are already there and dealing with the fact that our ML systems are beyond our ability to understand now. The logic that is plotted from a vast number of variables is beyond our ken. How do we make moral judgments when we can’t even count the variables involved in that judgement? Those issues are being tackled by developing a human audit trail made up of small decisions that would have been made by us in the process. Unfortunately the audit process may be invalid and we wouldn’t know. But that is beyond me . . .
From an historical perspective is this really different from the physical abilities that we acquire from systems that we build? Diving to the bottom of the Marianas's Trench put’s us totally at the mercy of the technology that we have built. Accepting administrative decisions made by an intelligent system for the management of society is the same type of dependency but different only in that the area is one that we hold as unique to our species. Whales can dive to great depths and that does not bother us. We go deeper with our technology and our proud. Handing over decision making to systems that we originally built hits our unique area of supremacy. But we learned to dive, fly, travel in space, and we will learn to trust decisions from our systems. If we don’t we simply can’t do what needs to be done for us to follow our desires.
The whole problem, and the inherent weakness exploited by fascism, is fear and cowardliness. Cultures collapse because of that. They become focused on the fear of others and follow a authoritarian leader that will sell them cheap strength by dominating the weak. The next stage of human evolution will be expanding on the the basis of vastly intelligent systems that we are able to integrate with to achieve our goals. Integration is more complex than rejection or domination. That integration is the challenge we need to meet and use or we won’t be able too survive.