AI Governance - The Trolley Problem

Wrt AI governance, let's talk how we govern subjective human beings. In the hunter/gatherers era, we would kick a renegade person out of the tribe/village/etc, which may mean certain death at the hands of predators/enemy tribes/etc. Later in our evolution, we created deities along with a set of divine laws, where the offender would pay a penalty for their misdeeds in the after life (as there were no police forces at that time). Now we have a criminal code which does not stop the person from behaving outside the 'norms', but if caught, results in increasingly severe penalties for acting in breach of those laws.

All of these measures are reactive... because humans are subjective. We cannot impose proactive rules that limit how we act because a) that would create a society of robots, and b) a human could act in a manner which has not even been considered by the creators of the laws. This is why we have courts, which take the individual circumstances of their subjective actions, and attempts to apply the laws to create new precedence. But this is an infinite process (NP-complete?) because, as stated, we will always create novel situations because we are subjective creatures.

So, for humans, governance is a reactive process. We can educate people on the laws both from a legal and moral ground, but at the end of the day, we must allow people to act subjectively, or we are a North Korea.

In the objective computing realm, we would test, test, test until we were sure(?) that this process will behave the same each and every time it is run. And we would pack into this algorithm all sorts of protection in order for this to happen. We would check (eg) input email addresses to make sure they are the correct format... DB transactions, try/catch constructs, etc. In fact, a considerable percentage of any piece of objective software is for error checking/handling.

With AI, we are now in the midst of a conversion from objective to subjective algorithmic computing. And unfortunately, we now have the same issues as we humans. How do we govern algorithmic subjectivity? We can have all the governance rules in place, but we face the same dilemma as in our subjective lives; if we create strict proactive rules, we limit subjectivity, and isn't this negating the point of AI?

With AI, how do we know that, in the millionth+1 time we run the process, it does not use a novel path which the creators of the governance rules had not anticipated, exactly like society? And what deems this novel path as 'in error', when in another computing scenario, this same path resulted in a new way at looking at a molecular structure for a cure of a disease?

This is why I say governance is AI's trolley problem. No algorithm that we ultimately use for governance will be optimal. Like the trolley either killing 1 person or 5 with no other options. If we want subjective algorithmic computing, this is the reality.

But AI is even worse, because our laws pertain to our own subjective actions. With AI, it is like the village asking Drog to 'go out and come back with some meat for the village'. Now Drog could hunt ethically/etc, or he could kill someone else who has just killed an animal ethically. Thus with AI, we must put boundaries into our use of AI not understanding at all where it will go to fulfil the request.

If we are eventually forced to govern subjective algorithmic computing which seems very likely, then we may have to model it on what we have within our current subjective social realm, which are reactive penalties in how the AI acted, creating a new precedence for future actions.