The recently published report (available here) provides some interesting recommendations and observations about one large provider's arrangements for data access, sharing and streaming infrastructure with some of the UK NHS.
The report flags a number of risks and concerns, including (as the reviewers see it) the potential for the supplier to be able to “exert excessive monopoly power” as a result of the data access and streaming infrastructure that is bundled with the provision of its app.
The reviewers also note the ongoing lack of clarity around the supplier’s business model — writing: “Given the current environment, and with no clarity about DeepMind Health’s business model, people are likely to suspect that there must be an undisclosed profit motive or a hidden agenda. We do not believe this to be the case, but would urge DeepMind Health to be transparent about their business model, and their ability to stick to that without being overridden by Alphabet. For once an idea of hidden agendas is fixed in people’s mind, it is hard to shift, no matter how much a company is motivated by the public good.”
It is clear that much has happened in the last 12 months to further undermine public trust in tech platforms and algorithmic processes. Whilst the UK government has been keen to position itself as a supporter of AI, there are still questions over how to manage and regulate public sector data and AI deployments, especially in highly sensitive areas such as healthcare.
One approach is for the health service itself to establish a clear and robust governance structure for AI and digital decision services that encourages competition by design by ensuring they have a requirement to support multiple independent options when procuring apps and services. Without such a structure, the risk is that platform dynamics will quickly dominate and control the emerging digital health support space, like big tech has dominated consumer tech.
The Independent Review Panel for DeepMind Health has published its second annual report today, focusing on business models, the relationship between DeepMind Health and its parent company, Alphabet, and the clinical utility of the applications it is developing. It also sets out twelve principles that the Panel will use to assess DeepMind Health’s activities in the future. These principles provide a useful set of expectations that could be broadly applied to other commercial organisations working with the NHS on health technology projects.