What is a TRE

A TRE is a secure physical or virtual environment designed for approved and named researchers to access sensitive pseudonymised data, where access to specific data sets is provided only to approved research projects. TREs differ from other data use models such as the more traditional data release model, where data are made available to approved researchers to download and analyse in their own data environments, hence risking losing control of their security and management. Instead, in a TRE, data are not released externally to data users for analysis on their own computers but placed on a server within a restricted, secure information technology environment, where the approved user is given secure access to carry out their project analysis. No row-level data leave the TRE environment. Traditionally, only aggregate-level results (eg, summary tables, graphs, statistical models) are released from a TRE at the end of the project, and only after a range of automatic and manual screening controls are applied to ensure that all outputs are non-disclosive of personal data.

The use of TREs is meant to address the challenges of using health and other forms of sensitive personal data to facilitate research that is assessed to be in the public interest while at the same time protecting privacy and ensuring trustworthiness.5 6 Importantly, their use does not eliminate the risk of disclosure of sensitive personal data but greatly mitigates it11 by providing assurances that data are handled securely, as data use can be tracked and technical and organisational measures are in place to check that no data leave the secure environment. Acting as data processors, TREs are meant to maintain a balance between:

• Confidence of data controllers (who determine the purposes for and manner in which any special category personal data are to be processed) through increased security.

• Benefits to the user/researcher (who can be from an academic, commercial or government setting) through improved access to larger data sets.

• Transparency for public and patients (whose personal data are made available in pseudonymised form) as to who has access to the data and for what purposes in order to ensure their continuing confidence and engagement.7

Robust data governance is key in achieving and maintaining such a balance. This means meeting all relevant legal obligations (eg, data protection, confidentiality, contracts and intellectual property), technical and cybersecurity requirements, and research ethics and data governance requirements.

There are several related frameworks used for providing guidance on TRE governanceiv most of which are based on the ‘Five Safes’ model. ‘Five Safes’ is an internationally recognised model introduced by the UK Office for National Statistics in 2003. It has been described as an ‘explicitly relativistic, subjective and empirical’ framework which has proved a ‘useful’ tool to frame, rather than prescribe, the crucial discussions around governance and management of sensitive data involving data providers, users and regulators.12 The ‘Five Safes’ breaks down the decisions surrounding data access and use into five related but separate dimensions:13 ,v

AI and the digitisation of healthcare: improving safety through the use of TREs

Healthcare has been identified as ‘one of the most important sectors for AI both for better services and for better efficiency’.14 15 This has paved the way for new, and arguably controversial, public–private–academic partnerships16 for the development of new AI technologies, including ML, which can be used in several healthcare areas such as diagnostics, therapeutics, population health management and administration, and for providing key infrastructure for the storage, maintenance and management of the data that underpin these technologies.17

Such developments align with the ongoing efforts since 2002 towards the digitisation of the NHS—from (missed) aims of achieving a ‘paperless’ NHS by 2018 to the renewed target for a ‘core level of digitisation’ by 202418—which has resulted in a rich and valuable wealth of healthcare data. The COVID-19 pandemic has reconfirmed and further accelerated plans for the digitisation of the NHS (ie, NHS apps, virtual appointments, online treatments, etc), along with recent plans to facilitate the more effective sharing of digital health and social care records and data.19 Further plans to personalise healthcare through the use of wearable technologies and apps will only enrich these data sets.

AI, and in particular ML, technologies for healthcare rely on the availability of big data for their training and development. As such, the extensive medical and healthcare data that result from the ongoing interactions between the UK public and the NHS have long been seen as a prime opportunity for the adoption of innovative AI technologies, for day-to-day patient care and for the further advancement of health research.

While the opportunities that the increasing digitisation of healthcare offers appear exciting, the risks and concerns that such developments entail are considerable. There have been a series of situations where public trust has been eroded in data sharing—some of which have attracted significant media attention and regulatory enforcement, while others may be more ‘routine’ infractions of contracts. Nevertheless, these instances cumulatively may instil a negative attitude in the public towards data sharing. Among these, past big data health initiatives, such as care.data—an English initiative designed to allow the repurposing of primary care medical data for research and other purposes—and more recently the postponed GP Data for Planning and Research programme,20 demonstrate the importance of public trust for major projects which seek to aggregate and centralise healthcare and related data, and the costly danger of losing it if legitimate public concerns are not taken seriously.21 22 Scandals, such as the ongoing case of DeepMind/Google and the Royal Free,vi along with a recent report in the BMJ that there are hundreds of organisations such as clinical commissioning groups, private companies and universities which have breached patient sharing agreements, some of them with little or no consequences23 (see also ref 24 25), demonstrate that what is often termed the ‘deficit of public trust’26 is not the result of public ignorance or badly publicised information21 27 (see also ref 28 29). Instead, it is an appropriate response of a public who, while willing for their data to be used for the benefit of patients and the NHS, are wary of a weak regulatory landscape that allows such data security failures.2 30 Nevertheless, this emerging picture has received limited attention in the academic literature and limited discussion as to how it may impact on data sharing and governance arrangements and policies more broadly as the NHS seeks to digitise and share more data.vii

To address the issue of the protection of personal data and the facilitation of research, TREs are positioned as a way to maintain and restore public trust.5–7 The technical and organisational safety measures that TREs offer can provide assurances that, not only will data not be leaving their secure environments but every interaction and subsequent analysis will be checked and tracked. Furthermore, their commitment to Safe Projects means that each project is assessed by an ethical and data governance committee for their potential to public benefit before a project approval is granted.viii In order to ensure public benefit and build public trust, the importance of patient and public participation alongside transparency of decision-making and data use has been further highlighted.7 31 32

However, while TREs may be identified as the appropriate way to address public trust concerns, our research shows that the increase in the development and adoption of AI, and in particular ML, in the medical and healthcare fields presents new challenges for the next generation of TREs which may threaten the ‘T’ of the TREs due to additional risks of disclosure of personal data by ML models trained on TRE data and a lack of clarity about chains of responsibility once the ML model has left the TRE environment.

TRE outputs and ML

Typically, TRE outputs take the form of aggregated results, graphs and tables. Before their release, these outputs go through both automatically and manually disclosive controls and are checked to ensure that no identifiable information is attached to them before allowed to leave the TRE. With the increased interest in ML trained on special category data such as healthcare and medical data, TREs are increasingly approached with requests to use their data for such purposes and to disclose new types of outputs, such as ML models trained on TRE data.

As the GRAIMATTER research demonstrates, the release of trained ML models from TREs introduces an additional risk for the disclosure of personal data.13 33 In other words, while on the one hand models are being constructed using training data, on the other, training data and/or a semblance or subset of it, or information about who was in the training set can also, in certain cases, be reconstructed from a model. This means that trained ML models may be considered as containing personal data and therefore constitute personal data sets, bringing them within the jurisdiction of data protection legislation.34 Personal data disclosure from trained ML models can happen inadvertently—for example, if the ML algorithm is overtrained, and the weights of the algorithm which are then exported from the TRE correspond to the data underneath—or, there can be malicious intent—for example, when a malicious researcher ‘hides’ individual-level data within the files (eg, sensitive data could be embedded in the weights of an ML algorithm which are then exported).33 ,ix

In order to mitigate any risk of direct or indirect personal data breach from the disclosure of trained ML models, and hence maintain public trust, key aspects of the technical, legal and ethical governance of TREs need to be reconsidered. Our project GRAIMATTER explored these challenges and proposed a range of measures and recommendations that need to be considered for the safe disclosure of trained ML models from TREs.13 We focused specifically on the ethical and legal governance issues arising from such practices recommending ways that they can be addressed. Here we present some broader issues that informed our thinking.

Legal and ethical challenges

While TREs have been positioned as the safer response to the riskier and controversial data release model, it is important to highlight that they are not a magic bullet. All their technical controls notwithstanding, they too are complex sociotechnical systems which, each in their own ways, bring together people, technology, regulations, institutional bodies, auditing and organisational procedures in a sophisticated but always precarious balance that seeks to facilitate research access to data while preserving privacy. The introduction of ML in such a setting requires us to rethink carefully whether and how a new balance can be achieved. In the paragraphs that follow, we highlight three critical areas that require further consideration if we want to ensure a ‘safe’ health and care data environment that truly serves the public, namely the possibility of disclosure of personal data by ML models once they have left the TRE, the dynamic nature of ML models and the impact that these and other factors have on the discussions around public benefit.