A recent blog post by the Information Commissioners Office (the regulator upholding the interests of the public with regard to data) identifies AI topics that business is finding challenging with regard to regulation.
One of the topics concerns having "a human in the loop" which means including human review and/or input in a process carried out by an AI system. The GDPR has provisions concerning decisions made solely by AI systems and often it is challenging for businesses seeking to use AI systems to understand how to comply by adding a human in the loop.
If a human is introduced into the loop should the human be an expert who understands how the AI system works or is it possible to use a novice human in the loop? It could be challenging for a novice human to properly check a decision made by an AI system. It could also be very difficult for consumers to check that a human in an AI decision making loop was actually effective or simply cursory.
Other points were raised by businesses as indicated in the quote below.
The post on the topic of human reviews in non-solely automated AI systems attracted the most interest to date, and generated more than 20,000 views. A number of stakeholders highlighted the trade-offs that having a human-in-the-loop may entail: either in terms of a further erosion of privacy, if human reviewers need to consider additional personal data in order to validate or reject an AI generated output, or the possible reintroduction of human biases at the end of an automated process. Trade-offs is a key area of risk in our framework, and therefore we will make sure to reflect this feedback in the associated blog post which we will publish soon. Finally, some of the comments also highlighted an important shortcoming in our initial approach. We were aware that AI may be developed or run partially or completely by third parties, rather than in-house. However, the feedback strongly suggests that this is the case the majority of the time. As we finalise our framework, we will need to consider further the challenges this presents for data controllers in exercising adequate levels of oversight and control.