For the good of humanity, AI needs to know when it's incompetent | WIRED UK

AI doesn't inherently understand its own competency. If a human worker needs help, they can ask for it — but how do you build an understanding of personal limitations into code? ... forecast when it's going to reach a situation where it has no experience... it gently taps the human on the shoulder... to take control... while we can delegate decision making to machines, we can't delegate responsibility... Unless we have a strong notion of competency awareness in AI, it won't become mainstream....
What happens if incompetent AI is paired with an equally incompetent human? ... humans working with or accountable for AI need to fully understand the systems... Explainable AI is not enough, you have to have trusted AI ... have human decision making in the loop... systems and humans will work in tandem, helping each other with their blind spots...