1 Introduction

Despite a focus in recent decades on improving mental health outcomes, the prevalence of mental illness has not decreased (Jorm 2018). To date, mental ill-health accounts for approximately one-third of all years lost to disability globally (Lake and Turner 2017). In response to what has been referred to as a “global mental health crisis,” (Torous et al. 2018) there are urgent calls to transform current mental health paradigms, including a "re-envisioning" of mental health care delivery (Lake and Turner 2017). We propose that the use of digital technology in the mental health field characterizes such a reimagining.

Digital mental health care, delivered via smartphone, increases the opportunity to democratize mental health care. The smartphone opens up an alternate pathway to effective mental health care for people who may not otherwise access face-to-face treatment (Srivastava et al. 2020). It also generates a possibility of scaling up mental health services and providing accessible, evidence-based treatment to large numbers of people.

However, despite the popularity of digital technology, particularly in adolescent and young adult populations, digital mental health interventions are characterized by low uptake and high attrition rates. The struggle to engage users, and sustain engagement over time, is one of the greatest obstacles facing the digital mental health field (de Beurs et al. 2017; Fleming et al. 2018). Torous et al. (2018) conducted a clinical review of user engagement with smartphone apps. The research reported that despite the potential of digital mental health delivery, digital interventions lack value to users because they "are not designed with service users in mind, do not solve problems users care most about, do not respect privacy, are not seen as trustworthy and are unhelpful in emergencies." These fundamental critiques hamper overall user engagement rates, and an increase in user uptake and sustained engagement may only be realized if these obstacles are addressed. Incorporating recommender systems into the digital mental health ecosystem presents a promising solution to Torous et al.'s (2018) finding that apps lack overall relevance to the user, thus leading to low engagement with what could otherwise be beneficial mental health care.

A recommender system is an information filtering system that uses algorithms to predict content or information that the system deems relevant to the individual. There are various ways that recommender systems could be used in mental health apps to determine what would be relevant to the user. They typically use information filtering algorithms to generate relevant content suggestions or recommendations based on a user’s personal information and previous usage data (Milano et al. 2020; Ricci et al. 2015). For instance, as demonstrated in Example 1, recommender systems can filter mental health topics and narrow down relevant content for the user based on what is known about an individual through the data they create through engagement with the Internet and their digital devices (e.g., smartphones), otherwise known as their 'data exhaust' or 'digital footprint’, as well as their app usage history and similarity to other users (Abdullah and Choudhury 2018; Paraschakis 2017). Recommender systems could be used in digital mental health technology to personalise the intervention, making it more applicable to the needs of the individual user and thus conceivably more engaging.

Example 1

X is still awake at 3 am and has been browsing social media on their phone for a few hours. X usually goes to bed before midnight on most nights but is having trouble sleeping for the second night in a row. The digital health app in question has made the inference that X is experiencing an episode of insomnia based on the following passive sensing data:

  1. 1.

    Device activity detected the use of the phone at atypical hours.

  2. 2.

    Accelerometer detected that the user had been lying down since 10 pm.

  3. 3.

    Ambient Light Sensor detected that the user had been in the dark.

Based on this sensing data, the app can generate personalised recommendations specifically relevant for the individual user. For instance, “Hi there. [The app name] notices that you are up at this odd hour and on your phone. Would you like to try some mindfulness to help you fall asleep?”.

If we are to accept the possibility that recommender systems can improve user engagement with mental health apps via increased personalisation, thus increasing relevance to the user (as demonstrated in Example 1.), then the use of recommender systems in the mental health field, particularly those driven by personal user data and artificial intelligence, present a range of important considerations. While the discourse surrounding the ethics of recommender systems is in its infancy (Milano et al. 2020; Paraschakis 2017), it is an area in need of careful consideration. Whilst by now, there is an emerging body of literature considering ethical and socio-technical aspects of recommender systems (Milano et al. 2020; Paraschakis 2017) and similarly, a body of such literature for digital mental health technologies (D’Alfonso et al. 2019; Gooding 2019), the novelty of this paper is the focus on considerations particular to the intersection of these two areas.

An ethical examination is of particular importance in the mental health field, as algorithmic systems that recommend digital mental health interventions are likely to have real-world implications on the mental health and wellbeing of those who use them. In line with BJ Fogg's 1998 seminal paper on persuasive technology and its impact on human attitudes and behaviours, researchers and developers in digital health must act as "watchdogs" charged with protecting the ethical integrity of the field (Fogg 1998).

We begin this paper with an introduction to recommender systems, with a particular focus on what we term ecological momentary recommendations (EMR). We then narrow our focus to the use of recommender systems in the mental health context more specifically. While a broader discussion of recommender systems is valuable, we suggest that there are both advantages and ethical concerns of using recommender systems in the mental health field that are particularly relevant to this context. It is these advantages and ethical concerns that have been identified by the authors and will be the focus of this paper.

Given the prevalence of mental health issues and the increasing use of apps and digital mental health technologies more generally, the considerations in this paper are of significant social importance as they will provide a greater understanding of how DMH apps can effectively and ethically implement recommender systems and thus increase their effectiveness.

The discussion of the advantages of recommender systems in the context of digital mental health and wellbeing will include (i) a reduction in choice overload, (ii) improvement to the digital therapeutic alliance, and (iii) increased access to personal data & self-management. While the discussion of ethical challenges relates to (i) lack of explainability (Zhang and Chen 2020), (ii) a privacy/personalisation trade-off and recommendation quality (Wang et al. 2018), (iii) and control of user history. Thus, our contribution to this literature is an examination of the advantages and ethical challenges of using recommender systems in the digital mental health field.

2 Recommender systems & ecological momentary recommendations

Recommender systems are based on algorithms that recommend to an individual, specific content items or pieces of information that the system deems relevant to that individual. There are various ways that recommender systems in an app or platform determine relevance, and they often use information filtering algorithms to generate relevant content suggestions or recommendations based on a user’s personal information and previous usage data (Milano et al. 2020; Ricci et al. 2015). For instance, Netflix and YouTube recommend videos to users based on their viewing history, ‘liked’ content, and shared interests with similar users, while Amazon recommends books to users based on their purchase history. In a similar vein to YouTube and Amazon offering items from a repository or library (of videos and books, respectively), digital mental health apps can offer a range of therapeutic content items, such as general mental health information, psychoeducational content, and suggested behavioural exercises or interventions. Such items can be for specific diagnostic conditions, transdiagnostic, or general mental health and wellbeing. Given a mental health app that contains a collection of such items, a recommendation system that delivers personalised item suggestions to users based on their mental health needs is a desirable feature, one that goes beyond expecting users to find what is relevant to them by manually browsing through a, possibly sizeable, list of items.

As mentioned, there are various ways that recommender systems can determine recommendations. They are generally based on techniques that use data from sources such as information about an item, a user’s profile, or a user’s usage history, including items they have browsed and ratings. Beyond such traditional recommendation system approaches, the use of smartphone and wearable technology’s capacity to collect usage and sensor data, both actively (requiring user involvement) and passively (not requiring user involvement), to make personalised recommendations has been an emerging area of interest in the digital health field (Abdullah and Choudhury 2018; Amft 2018; Boonstra et al. 2018; Chancellor et al. 2019; Cornet and Holden 2018).

The digital delivery of relevant, personalised health interventions in the moment based on contextual, behavioural, and physical information inferred from sensing data has been termed ecological momentary intervention (Schueller et al. 2017) or just-in-time adaptive intervention (Nahum-Shani et al. 2018) in the literature. To accommodate the possibility of delivering recommendations in general (rather than just health interventions) via such means, we here establish the term ecological momentary recommendations (EMR), defined as contextually relevant and personalized in-situ recommendations informed by sensor and usage data collected from ubiquitous personal devices, particularly smartphones and wearables. EMR thus rely on sensitive personal data about the individual and will generally involve forms of artificial intelligence (AI) to process and make sense of this data. This technology could address or circumvent some of the issues related to digital intervention, particularly concerning engagement.

Personal devices that make behavioural suggestions based on passive data are commercially available. For instance, Fitbit collects individual usage and sensory data via wearable technology and uses AI to make personalized in situ behavioural recommendations. For example, a Fitbit wearable collects passive sleeping data. If the user records lower levels of sleep than desired, the Fitbit app may recommend a mindfulness routine before bed to improve the quality and length of overall sleep. Most smartphones now come equipped with a range of passive sensing tools that collect behavioural and contextual information akin to these commercial wearable products (AWARE 2021; D’Alfonso et al. 2018). Sensors already incorporated within a typical smartphone and relevant to the mental health field may include the accelerometer (movement), applications (app use activity), Bluetooth (where the user is in relation to others), communications (texts, phone calls, and social media use), GPS (location and movement), keyboard use, and screen (Abdullah and Choudhury 2018). The collection of passive sensing data and identification of specific behavioural signals may have future potential to make clinical diagnoses, an area of research otherwise known as digital phenotyping (Boonstra et al. 2018; Trifan et al. 2019). For the foreseeable future, in the mental health context, it is most likely that passive sensing will be used to assess single sets of user and sensory data to make health recommendations.

Example 2

User X has been sedentary at home all day. For most of the day, they have been using their phone to browse social media and to play games.

Through passive sensing data, the app has made the inference that they have been sedentary all day, and that it would be good for their well-being to get moving.

Based on this sensing data, the app generates personalised recommendations. For instance, “Hi there. [The app name] has noticed that you’ve been using your phone all day at home. It looks like a lovely day today. How about going for a walk?”.

Personalised health recommendations based on personal sensing data are a step beyond traditional content recommendations that are just based on user preferences and previous usage history, as is the case with media streaming services such as Netflix and Spotify. Given that they involve collecting more sensitive personal tracking data and using this data in algorithms, EMRs require greater ethical scrutiny. It could be reasoned that recommender suggestions based on a user’s digital exhaust in the case of Netflix or Spotify are relatively harmless and of little consequence to a user’s health and safety. Mental health EMR’s, on the other hand, require access to sensitive user data related to everyday behavioural and contextual experiences and subsequently make real-world recommendations which an individual is apt to act on. If this occurs in a mental health context, the technology’s recommendations could significantly impact people’s everyday lives. Thus, the consequences of these recommendations and the actions they may produce must be cautiously considered.

3 Advantages of recommender systems

3.1 Self determination theory

According to self-determination theory (Deci and Ryan 2015), a theory of human motivation, development and wellness, users are motivated to engage meaningfully in activity that satisfies the three basic psychological needs of autonomy, competence and relatedness. Correspondingly, users will only participate under duress or avoid an activity that fails to fulfil these needs (Deci and Ryan 2015). Autonomy in an SDT framework refers to intrinsic or well-internalized motivation, and is typified by the capacity to control one’s experience and the offerings of “choice” and “rationale” are characterized as “autonomy-supporting behaviors” (Hu and Zhang 2017; Muñoz and Ramirez 2015). We present three advantages to the inclusion of recommender systems in the digital mental health ecosystem that supports the basic tenants of SDT theory in various ways. As well as increasing relevance to the user, recommender systems present a possibility of increasing user engagement by better satisfying the motivational needs of autonomy, competence, and relatedness.

3.2 Reduction of choice overload

While the element of choice is necessary for autonomy, an abundance of choice, or over choice, can lead to a phenomenon known as choice overload. According to the paradox of choice theory (Fernandez 2017) and the choice overload hypothesis (Scheibehenne et al. 2010), the experience of choice overload can psychologically overwhelm the user, acting to negatively diminish user autonomy and leading to lower levels of motivation and lower levels of satisfaction with their overall choice (Scheibehenne et al. 2010).

Example 3

User X is logging onto a mental health app to help manage high-stress levels at work. Because user X has searched for such information before, or because the system has information that the user experiences stress and their phone GPS has passively identified that they are at their work location, the recommender system provides several personalized recommendations on dealing with stress in the workplace. Thus, user X is presented with a series of relevant choices suggested by the recommender system, supporting autonomy, and avoiding choice overload.

The paradox of choice theory posits that a user will become overwhelmed by large catalogues of information and that this creates a psychological burden that decreases overall motivation to engage with an activity. This is particularly relevant to the digital mental health field, where high prevalence disorders such as depression and anxiety are often linked to cognitive difficulties such as trouble concentrating and rumination (American Psychiatric Association 2013). Thus, making this population of app users potentially more susceptible to choice overload. Recommender systems, however, can operate in such a way that provides sufficient choice for users to exercise their autonomy while not feeling overloaded with excessive recommendations, such that they become overwhelmed, their autonomy is diminished, and they disengage from otherwise helpful mental health support. Example 3 demonstrates how recommender systems can practically achieve this balance and avoid jeopardizing a user’s overall goal to engage with otherwise helpful material. Additionally, there is a connection between personalisation and the SDT principle of autonomy, as “personalization also creates a sense of ownership and choice beneficial to autonomy.” (Peters et al. 2018) Recommender systems can filter content and deliver customised mental health suggestions based on individual usage data, thus offering personalised options based on the user's needs but not overwhelming them with choices that vary in relevance. Thus, offering choice is more likely to satisfy a user's need for autonomy, while also offering a shortlist of personalized suggestions is less likely to overwhelm. The balance between no choice and too much choice needs to be meaningfully struck in mental health apps, as this balance increases the possibility that an individual will be motivated to engage in purposeful action.

Whilst it is important that the system genuinely determines recommendations to be relevant to the user, research has shown that the simple act of labelling content as a recommendation increases the likelihood that a user will engage with it (Paraschakis 2017). Whilst this phenomenon is of interest to recommender systems in general, it is of particular interest in the digital mental health context because when paired with the concept of balancing autonomy and choice overload, these two theoretical recommender system advantages could substantially impact a user's engagement with personalised therapeutic content. Greater engagement creates more opportunity for people to receive relevant mental health treatment, thus increasing the possibility that they will gain benefit from the technology.

Another aspect of personalised recommendations of particular significance to mental health content is that the recommendation of such content may foster a moment of psychological reflection for an individual simply by virtue of them receiving a personalised recommendation. Unlike say, the recommendation of a movie based on the simple fact that a user likes the movie’s genre or has watched similar movies, a mental health recommendation may provide the user with some deeper personal insights or pause for reflection on their own mental health or state (Gumley et al. 2020). On the other hand, such potential must be constrained and recommendations for serious mental illness must be delivered cautiously and appropriately; deliveries based on false positives or that cause distress, naturally need to be avoided.

3.3 Digital therapeutic alliance

The therapeutic alliance, or the relationship between a therapist and a client, is a reliable predictor of positive clinical change (Ardito and Rabellino 2011; Horvath and Luborsky 1993; Martin et al. 2000). As the mental health field expands to include digital health solutions and offer therapeutic interventions that do not include human therapists, the concept of a digital therapeutic alliance (DTA), or the relationship between a user and a digital therapeutic technology, has emerged (D’Alfonso et al. 2020). While there are strong indicators that the quality of the client-therapist relationship plays a significant role in therapeutic outcomes, surprisingly little consideration has been given to whether a person might create a therapeutic connection with a digital mental health app. According to SDT, DTA could help to encourage engagement with digital mental health technology by satisfying the need for relatedness. However, until recently, studies of digital mental health have tended to overlook the idea of a DTA while performing clinical trials and working on digital mental health technology; however, as this idea has consequences for engagement and outcomes, researchers must examine it in greater detail. The core components that comprise the traditional therapeutic alliance consist of agreement on treatment goals and tasks and developing a reciprocal bond between client and therapist (Horvath and Luborsky 1993). It is this notion of bond that could become analogous to the STD concept of relatedness. One could more easily conceive how agreement on goals and tasks could be accomplished between users and technology. Thus, it becomes the concept of the bond between user and technology, or some digital equivalent such as ‘connection’, that is of significant interest in the digital mental health context. We suggest that recommendation systems could play a role in strengthening the experience of relatedness between the user and a digital mental health app.

A narrative review conducted by Tremain et al. (2020) found that it was possible to cultivate a therapeutic alliance between people with serious mental illness and digital mental health interventions and that there may be “unique, yet-to-be-confirmed characteristics in digital contexts (Tremain et al. 2020).” These findings are supported by Lederman et al.'s (2019) who reported that creating engaging therapeutic interventions that combine various therapeutic mechanisms, such as personalized therapy, automated feedback, and social connection, can produce "a state that mirrors therapeutic alliance” (Lederman et al. 2019). To date, there is no scale that measures an established conceptualisation of alliance with mental health apps, although some incipient work has been done. One preliminary or exploratory attempt to fashion a measure of DTA, the mobile Agnew Relationship Measure (Berry et al. 2018), is an adaptation of the Agnew Relationship Measure for the traditional therapeutic alliance (Agnew-Davies et al. 1998). Brief consideration of some of the items in this 25-item scale further suggests the role recommender systems can play in fostering a DTA with mental health apps. For example, Item 5 is "I Have Confidence in the App and the Things It Suggests". Personalised and accurate recommendations accompanied by good explanations would be conducive to scores for this item. Or take Item 14, “The App Seems to Understand Me”. It stands to reason that personalisation of app content delivery will increase the chances of better scores for this item. In fact, a qualitative analysis conducted by Clarke et al. (2016) identified that automated personalization helped one user feel understood and that intelligent responses from an app fostered the perception of a relationship for another user (Clarke et al. 2016; Hillier 2018).

Personalising a user's experience increases the likelihood of feeling understood by the digital therapeutic intervention, and this experience more closely 'mirrors' the traditional client–clinician therapeutic relationship. Thus, recommender systems in this context are the mechanisms by which the personalisation occurs and, as such, offer a plausible way to positively impact the DTA and overall levels of engagement.

3.4 Increased access to personal data & self-management

Ecological momentary recommendations can give users greater control over their health care by providing immediate access to information passively collected through personal digital devices and personalised health recommendations (Lupton 2013). Quantified self-data informs the recommendations, as such ecological momentary recommendations could not occur without this two-stage data collection/recommendation process. For instance, consider the data tracking already commercially available via Fitbit and Apple Watch, such as heart rate and sleeping data. This sensory data is collected and provides users with immediate access to organised information previously collected primarily in the domain of health professionals and can work to promote the basic needs of autonomy and self-competence in accordance with SDT (Deci and Ryan 2015). This streamlined approach provides the individual with direct access to their personal information, theoretically increasing the possibility of personal decision-making and self-management (Sharon 2017).

As introduced earlier, research is underway to identify associations between behaviours or changes in behaviour and user data, particularly data collected via smartphones For instance, passive sensing is apt to collect and identify signals for depression, such as increased isolation levels, increased hours of sleep, and a decrease in activity via GPS and behavioural patterns concerning texting and app use (Boonstra et al. 2018; Trifan et al. 2019). Particularly in regard to predicting mental health issues with such data, this field of research has come to be known as digital phenotyping (Spinazze et al. 2019). In 2018, Abdullah & Choudhry classified sensing technology data relevant to severe mental health conditions into three categories—behavioural, physiological, and social signals. The authors posited that these data signals have the potential to correlate with the symptomology of specific mental health conditions. For instance, data associated with bipolar disorder could be linked to location and mobility data, speech patterns, general technology use, activity, social interaction, and communication patterns. However, to date, there is still limited research demonstrating a strong correlation between mental health states and user data (Trifan et al. 2019).

However, through collecting and responding to user data with timely and relevant recommendations (Schueller et al. 2017), effective sensing technologies could significantly improve a user's health and well-being and offer a plausible and cost-effective solution to the scaling up of mental health systems (Abdullah and Choudhury 2018) (Trusty et al. 2019). As the name suggests, passive sensing requires little active input from the user (Cornet and Holden 2018), as such, informative health data can be collected relatively unobtrusively, with limited burden to the user. In line with the paradox of choice theory, a selection of personalised EMR's could be suggested to the user to select from to address health conditions that may otherwise go unattended and does not rely on user engagement with the technology itself, which we already know to be typically low (de Beurs et al. 2017). As many mental health conditions are associated with high levels of comorbidity(Druss and Walker 2011; Wang et al. 2019), providing easily accessible and understandable personal health data such as resting heart rate, sleep patterns, and mobility data, with accompanying EMR’s, could benefit people from in this population.

4 Ethical challenges

4.1 Explainability

The concept of explainability (Zhang and Chen 2020), or the notion that the reasoning behind an AI or algorithmic process should be explained to a user alongside the output, is an important factor in recommender systems discourse as well as an “autonomy-supporting function” in SDT (Muñoz and Ramirez 2015). In recent years, explainability has taken on a pressing significance given the 'black box' nature of certain AI methods. Machine learning and deep learning models, in particular, may successfully produce a specific output without providing any insight into how or why that output was generated (Pedreschi et al. 2019). However, even in simpler cases where the process or mechanisms behind a recommendation can be explained or accounted for, questions can still arise regarding the presentation of recommendations containing sensitive content. In domains such as video streaming or book delivery, it is generally adequate to offer a factually informative explanation. For example, Amazon can explain a recommendation with statements such as “Recommended because you purchased Book X”. On the other hand, the nature of mental health content and the psychology of individuals using such content mean that careful consideration must be given to how the recommendation of such content is presented and that any accompanying explanations are affectively appropriate and sensitive to the user's experience.

This is particularly relevant given the often sensitive nature of mental health contexts. Research has shown that effective communication regarding a diagnosis is linked to improved client outcomes related to satisfaction, health, treatment adherence, and overall understanding of illness and treatment (Cegala and Broz 2002). Hence, the way a mental health diagnosis is introduced and initially discussed with a person can profoundly impact that individual's subsequent mental health journey and overall recovery and should be treated with utmost importance.

Although recommendation systems are designed to provide personalized content recommendations, they do not necessarily provide the affective nuance needed to accompany the delivery of sensitive personal content. Jarring diagnostic language or unanticipated recommendations could frighten or alienate the client, which, according to Cegala and Lenzmeier (2002), could reduce an individual's overall understanding of their illness, treatment, and treatment adherence. Thus, while explainability is a pressing topic more generally in recommender systems literature, there are unique complications in the case of mental health recommendations, as a diagnosis or mental health recommendation delivered without satisfactory explainability to the user could impact the user’s overall mental health recovery. Therefore, how recommendations are explained to users and which recommendations require human support requires careful consideration.

4.2 Privacy/personalisation trade-off and recommendation quality

To receive optimised recommendations from popular media platforms, a user must engage in a privacy-personalisation trade-off (Wang et al. 2018). If, for instance, a recommender system has little to no knowledge of an individual, then personalisation is not possible. To achieve greater accuracy with personalisation, more privacy-sensitive information is required. As a relatively innocuous example, a Netflix customer ‘trades’ their usage history, type of device, amount of time spent watching, user ratings, and similarity to other users to receive optimised program suggestions. This viewing information is a comparatively low-risk exchange of personal data to receive relatively harmless media suggestions, and the negative impact of an irrelevant recommendation is low.

However, when considering recommender systems in the mental health field and the usage of sensing and digital footprint data for EMRs, the balance between privacy and personalisation potentially becomes more of a high-risk situation. For example, a person may need to exchange a greater degree of private information to receive appropriate therapeutic suggestions. However, this privacy-personalisation trade-off raises several ethical concerns. The first pertains to the emerging nature of digital mental health technology and the subsequently unclear legal and regulatory frameworks (Gooding 2019). An individual's right to privacy and confidentiality regarding their mental health information is grounded in their therapeutic relationship with the health provider in question, for instance, a psychologist, pharmacist, or hospital (Hattingh et al. 2015). However, the therapeutic relationship between a person and mental health technology is not clearly defined, and responsible legislation protecting the user's rights is ambiguous. Studies have indicated that mental health apps in the commercial market do not conform to clinical guidelines and do not respect user privacy (Torous and Roberts 2017). The sale of private information to third parties without the user's knowledge or explicit consent is typical behaviour in the commercial market (Torous and Roberts 2017; Wang et al. 2018). Considering the private and often sensitive nature of mental health data and the information recommender systems would require regarding health symptoms, possible diagnoses, and mental health history to make appropriate recommendations about mental health creates an environment for obvious ethical concern. This private and sensitive information collected by a smartphone or digital device in the current context of unclear legal and regulatory legislation leaves the user open to security breaches and the commercialisation of their mental health data (Wang et al. 2018).

The second area of concern pertains to the detrimental impact of inappropriate mental health recommendations on a person's immediate or long-term health and safety, with the related consideration of trading off personal data for accuracy given the potential risks. There are various metrics for recommender systems, with precision being one of the basic mains ones (Gunawardana and Shani 2009):

$${\text{Precision }} = \# {\text{Relevant Recommendations }}/ \# {\text{Total Recommendations Generated}}$$

We can similarly define imprecision as follows:

$${\text{Imprecision }} = \# {\text{Irrelevant Recommendations }} / \# {\text{Total Recommendations Generated}}$$

With the nature of mental health content, considerations beyond basic relevance arise. By our definition of relevance, if a mental health recommendation is relevant, it is beneficial and not detrimental to the user. However, there are two types of irrelevant recommendations; those which are not detrimental but merely not useful to the user, and those which are potentially detrimental. Hence this is one domain where recommendation quality judgement goes beyond the dimension of relevance/accuracy. Given the potential costs of imprecision, a greater weighting might be placed on providing personal data for the sake of precision. Alternatively, a mental health recommendation system might err on the side of caution and place a higher threshold on when a possible recommendation is safe to deliver; in doing so, the metric of recall (the proportion of all relevant items that are actually recommended) would be diminished as follows:

$${\text{Recall }} = \# {\text{ Relevant recommendations }}/ \# {\text{ Total relevant items}}$$

Thus, the immediate ethical concern regarding irrelevant mental health recommendations is that they could deliver unsafe or inappropriate content for an individual’s condition. For instance, Torous et al. (2017) reported on a mental health app that irresponsibly suggested that people experiencing mania related to bipolar disorder should consume alcohol to assist with sleeping. However, a more ambiguous area is related to the potential for recommender systems to make imprecise recommendations that may impact the health and safety of the user if they have not traded sufficient data, which is depicted in Example 4.

Example 4

Via GPS and Bluetooth, the smartphone app detects that user X has not had any social contact for some time. As a result, the app recommends that user X visit a friend whom the app identified from their frequent contact list. Unbeknownst to the recommender system, the user has a limited social circle, all of which are substance users. User X is struggling to avoid this circle of friends with the goal to abstain from substance use. Thus, the recommender system has made a detrimental recommendation by encouraging the user to contact a friend from this social group.

Example 4 illustrates how recommender systems and inaccurate recommendations in the mental health domain can have a more significant consequence than an inaccurate program suggestion on, for example Netflix, where a user has the option to simply dismiss recommendations with virtually no consequence. However, mental health therapy recommendations could exacerbate vulnerabilities and have negative impacts on the health and well-being of the user. To return to Example 4, abstaining from substance use requires willpower and motivation that is extremely difficult for people to sustain over time; even minor perceived stresses are correlated with substance use and impulsivity ( Mooney et al. 2008). Thus, receiving a recommendation encouraging a user to engage with a friend that they are actively avoiding in order to refrain from drug use could have a detrimental impact on said users’ overextended motivation. Therefore, when it comes to mental health care, recommendations may appear relatively innocuous but have the potential for greater negative impact due to the sensitive and personal nature of the subjects they pertain to. Thus, in the calculus of privacy versus personalisation, it might be justifiable from this perspective to accrue more information about a client and safely trade-off a bit more privacy for better personalisation and safety.

4.3 Control of user history

Recommendation systems in popular sites such as Netflix and YouTube are largely driven by user usage histories, which in the cases of these sites consist of complete records of the videos a user has watched and ratings given (Amatriain and Basilico 2015). Given the understanding that users have the right to delete, or in some ways change, their usage history, such sites offer account interfaces that allow users to do so. For example, Netflix provides a page where users can alter the ratings of videos, which they have previously rated. YouTube has one of the most thorough and granular settings sections enabling users to delete some or all of the individual items in their watch and search histories. Of course, if this modification option were to be exercised by a user, it follows that the recommendations offered to a user would be influenced; in the case of a complete history erasure, the recommendation system would be returned to a blank slate. The ability to clear usage history is one form of user control, and in terms of digital health, participants in past studies have highlighted the desire to have control over the personal data derived from health apps they use. Such control was found to be conducive to a sense of user autonomy and empowerment, as well as perceived app trustworthiness (van Haasteren et al. 2019).

A principle that might be reasonably asserted from this previous work is that users should have the right to modify or erase their usage data, data that ultimately powers their recommendation system. However, with digital mental health applications, such a principle is not as straightforward and offering users the ability to erase their usage history could be problematic. This is because, unlike in video content consumption, book purchases or social newsfeed interaction, an individual's usage history on a digital mental health platform, particularly if integrated with clinical services, may very well need to be treated as a clinical record. Regulations and legislation that guide how data related to mental health interventions should be stored have not been established, though this is an area of emerging interest as technology use in the mental health field continues to expand (Gooding 2019). Under the freedom of information laws in Australia, patients of mental health services have the right to access their medical records (Paterson 2007). However, under the act, they do not have the right to change or destroy those records. From this perspective, removing app usage history would be considered akin to a user changing their mental health clinical records. In offering these hypothetical considerations, we acknowledge the structural difference between a personal mental health app and a controlled medical environment. However, as alluded to earlier, at the very least, regulations pertaining to medical environments would carry weight in the case of digital mental health platforms that power blended interventions tightly integrated into clinical services. The applicability of such regulation would preclude, or at least put constraints on, offering a settings section in such mental health apps that simply allows users to modify or erase their usage history. For example, in response to the COVID-19 pandemic, the Moderated Online Social Therapy (MOST, most.org.au) system (Alvarez-Jimenez et al. 2021), a mental health web platform for blended interventions that integrates with clinical services, has been implemented within headspace and specialist youth mental health services across Victoria, Australia. This platform incorporates online therapy modules, social networking among young people and clinical, vocational, and peer support. The young persons' usage patterns, completed therapy modules, therapy notes and engagement with online clinicians and peer workers can also be accessed by their face-to-face clinician. While legislation regarding the storage of this data over time may be unclear, it is also certainly not clear that young people using the system should simply be provided with an option to erase their usage history, particularly their interactions involving clinicians and subsequent therapeutic recommendations. However, an inability to delete or alter user data is at odds with current recommender system informed technology. This issue emphasises a gap in the current literature concerning medical records in the digital age and, moving forward, what obligations clinical services may have to maintain and make deliverable a user's digital mental health record if said service has incorporated a digital mental health element.

This restriction does raise a new design principle to consider, in line with the goal of supporting user control (Jannach et al. 2017; Valdez et al. 2016): users should have the option to detach the recommendation system offered to them from their usage history. This could be via some explicit decoupling mechanism that retains the user data as is within the live system or via a data deletion process that removes the user data from the live system and transfers it to an archive format such that it no longer forms part of the dataset in the live system. In this way, medical record usage history can remain unalterable, whilst users can control their usage history to influence their recommendations. This control would be beneficial in scenarios where a user who has been experiencing a specific condition and using therapy items to help with that condition ceases to want to engage with such content or has a change in their mental health interests. Since in the short term, the recommender system will be influenced by their immediate usage history, the ability to detach this history and generate fresh recommendations would be useful. A simple mechanism would be to have a ‘detach switch,’ which would prevent a user’s current history from influencing future recommendations. Additionally, users could have control over detaching individual items or items associated with some tag or theme. Alternatively, a similar mechanism, which is standard in popular recommender systems and influences recommendation generation, is to accompany each recommendation with an option not to receive that recommendation or recommendations like it anymore.

5 Summary & practical guidance

This paper identifies and discusses advantages and ethical concerns of introducing recommender systems into the digital mental health ecosystem. The factors presented will be summarised here, and guidance on implementing these factors when developing ethical digital mental health technology will be provided.

5.1 Advantages

  1. 1.

    Reducing choice overload. Recommender systems should provide a select number of relevant content suggestions in order to foster autonomy but not overload the user with choice. Select but relevant suggestions satisfy the user's need for autonomy and competence by providing an appropriate level of choice while simultaneously avoiding user disengagement due to choice overload and cognitive difficulties common in many mental health conditions. Using recommender systems to achieve this goal could lead to higher levels of engagement.

  2. 2.

    Increasing the digital therapeutic alliance between user and technology through relevant content recommendations. Therapeutic alliance, or relationship between therapist and client, is one of the strongest indicators of positive clinical change. Evidence suggests that it is possible to cultivate a relationship between a user and digital therapeutic technology if the technology is personalised and responsive to the user. Ensuring that the recommender system provides accurate recommendations that respond to the user's changing needs over time could serve to foster a digital therapeutic bond and increase engagement with the digital mental health technology.

  3. 3.

    Support users to access comprehensible personal health data and receive recommendations to self-manage health and well-being. Satisfying user autonomy is an essential element of user motivation. Digital mental health technology should consider how to provide users with access to the data collected about them in a comprehensible manner and consider how such data and subsequent recommendations could support the user to self-manage aspects of their health and well-being.

5.2 Ethical concerns

  1. 1.

    Recommender systems may have limited explainability regarding sensitive and personal information. When making recommendations, systems should err on the side of caution and be cognizant of technology’s limitations to always communicate in an affectively nuanced capacity. As such, rather than suggest a sensitive recommendation that the system does not have sufficient confidence in or it cannot explain sufficiently, either withhold the recommendation or make a safer, more conservative suggestion.

  2. 2.

    Privacy/personalisation trade-off and recommendation quality. Systems should be explicit regarding the use and storage of user information. Recommendations that could impact the health and well-being of users should be made cautiously, with a preference for avoiding imprecision over maximising recall. Furthermore, users should be made explicitly aware of the potential for erroneous recommendations.

  3. 3.

    Lack of user control over personal history. Commercial platforms have traditionally given users control over their personal data history and offer convenient ways to delete such data if desired. Thought must be put into the design of digital health apps to consider if and how users can delete their data history, or particular portions of their data history, and how the inability to do so may result in recommendations that are no longer relevant or have the potential to cause distress. Further, the user data that feeds recommender systems could be considered similar to a medical record. This emphasises a gap in the current literature concerning medical records in the digital age. Moving forward, we need to consider what obligations clinical services must maintain and make deliverable regarding a user's digital mental health record if said service has incorporated a digital mental health element. User data that feeds recommender systems should be conceptualised like a medical record. Thus, developers should err on the side of caution when collecting and storing mental health data, treating such data sensitively, despite the lack of legislative guidance.

6 Conclusion

This paper sits at the intersection of recommender systems and digital mental health technologies. While separate bodies of work have focused on these two areas, the juncture presented in this paper has not yet been examined to our knowledge. This paper highlights the opportunity to increase the relevance of digital mental health technology to users by incorporating recommender systems that suggest personalised content based on an individual’s data exhaust. Advantages and ethical concerns of incorporating recommender systems into the digital mental health domain provide a greater understanding of how DMH apps can effectively and ethically implement recommender systems. However, the use of recommender systems, particularly those driven by personal data and artificial intelligence, present a range of ethical considerations. As such, we have offered a series of guidelines in response to the advantages and challenges discussed in the hope of creating both effective and ethically sound digital mental health technology.