Quality of Life and Needs in Caregivers: Results From the Prospective Multicentric Open-Label Randomized Study of Informal Caregivers of Elderly Patients

Introduction

During recent decades, life expectancy in technologically-advanced countries has continuously increased, and a higher prevalence of chronic diseases has become a major public-health policy concern [1]. Health systems at different levels of health and social care need to coordinate their efforts to provide appropriate services in daily living for older people with functional limitations. Public support for informal care is one of the most important public policy measures for the future sustainability of health and social care in ageing populations [2, 3].

In Europe, caregiver incidence is rising steadily, and forecasts indicate that 25% of the people in employment will also assume caregiving responsibilities in 2030. Last updated estimates in France reported 9.3 million (14.8%) caregivers in 2021, including 4.3 million of caregivers of the elderly [4]. Informal caregivers usually include spouse, parents, adult children, friends, or neighbors providing any type of unpaid assistance with activities of daily living, regardless of the duration of daily time devoted to caregiving and accompanying [5]. Although the estimated number of caregivers varies between countries, depending on informal care definition and measurement, many European countries considered that almost 20% of the people aged 50 years or older were informal caregivers in 2017 [6]. This caregiving role is physically, emotionally, socially, and financially demanding, and about one third of the caregivers have admitted that they experience negative effects in their familial and social life, that they lack time to pursue leisure activities, and that their psychological and physical health may be greatly affected [7]. In 2017, the European Pillar of social rights makes explicit commitments to caregivers, including their right to flexible working and access to care services, and initiatives to support and preserve their employment, health, and wellbeing [8–10].

Caregivers may be supported through different interventions, ideally integrated into usual management. Recent approaches showed that home-based, nurse-led-interventions for caregivers incorporated into primary care improve health-related quality of life (QoL) in caregivers of patients with chronic or disabling conditions [11]. Whereas support to caregivers is integrated into early palliative care in oncology [12], other helpful contributions may be provided by community resources or support groups [13]. Social workers, involved in the management of older people, and experimenting in facilitating the caregiving role, can also be considered as privileged partners not only for patients but also for the specific population of caregivers, which suggests addressing the scheme provided through the ICE study. In addition, possible deployment using regional and nationwide networks is a tremendous asset and a promising and economically viable option.

In order to promote caregiver support and wellbeing, careful monitoring of caregiver QoL needs to be performed [14, 15]. Obviously, caregiver QoL is closely linked to the health status of the helped patient, and profound changes in caregiver lifestyle are regularly required, with practical, organizational, and potentially economic problems associated with them, according to the patient health status [16]. From initial diagnosis, through successive phases of stabilization, remission, or progressive decline and palliation, exposure to chronic stress may directly affect caregiver mental and physical health, and appropriate caregiver QoL assessment with access to support and respite is essential to prevent caregiving exhaustion. As part of the overall concept of QoL, monitoring caregiver anxiety and depression symptoms and burden is of particular importance, and their related scores are useful to detect early signs of mental health deterioration [17–21]. These scores are used in caregivers to further preserve their health, or to prevent health deterioration, which could subsequently jeopardize their ability to care for themselves and for the patient. Caregiver needs further require appropriate characterization to provide dedicated support with efficient, appropriate, and timely services including psychosocial services, respite, training, education, and long-term care. In addition, better identification of the caregivers the most in need is also necessary to relieve and ideally prevent burden associated with caring [17, 18].

This French prospective multicentric cohort of the Informal Carers of Elderly (ICE) study was initiated in 2015 in the Burgundy-Franche-Comté region (2.8 million inhabitants, 4.4% of the French population) [22], and included i) an observational study planned to enroll 7,604 caregivers of patients aged 60 years and older recently diagnosed with chronic disease for a 5 year period, with the primary endpoint defined as caregiver QoL assessment over 5 years; ii) a randomized trial focusing on the first 2 years of caregiving, to compare caregiver QoL at 1 year between caregivers receiving an information booklet alone and those receiving social worker support, according to the patient disease. It is necessary to demonstrate whether the social worker intervention benefits caregivers. The trial faced significant recruitment difficulties. Although several amendments were adopted to overcome barriers in recruitment and revision of the initial objectives, low accrual prompted the steering committee to stop enrollment in May 2019. The present work aims to report caregiver and patient characteristics and assess caregiver QoL, anxiety and depression, and burden at one and 2 years, as well as changes over the 2-year period in caregivers receiving an information booklet combined or not with support from social workers.

Methods

Design and Study Population

Physicians received patients diagnosed with one of the specified diseases and detected the potential primary caregiver during consultation. Caregivers were ≥18 years, identified by the patient or self-identified as primary caregiver, not employed by a healthcare organization, residing in the French region of Burgundy-Franche-Comté. Caregivers supported patients aged ≥60 years with a neuro-degenerative disease (idiopathic Parkinson’s or Alzheimer disease), cancer (breast, prostate, or colorectal), age-related macular degeneration, or neuro-vascular disease (stroke). Caregivers of patients living in institution, and caregivers under legal protection were not included. Inclusion/exclusion criteria were previously detailed [23].

A clinical research associate met the caregiver to gather all required information regarding the study (information sheet, informed consent), enrolled the caregiver, and indicated the allocation group. More details were previously reported [23]. All caregivers provided written informed consent.

Faced with low recruitment to achieve the initially planned sample size, and despite several amendments, the initial goal was unachievable, and the final sample size of the cohort ICE did not allow the statistical analyses initially planned [23] and specifically the lack of power did not allow a direct comparison of the supportive intervention and control groups.

Intervention Description

Caregivers were randomly assigned (1:1 ratio) to receive an information booklet and intervention from a social worker in the supportive intervention group (SIG) or to receive exclusively an information booklet in the control group (CG). Randomization was done by the data manager with an interactive web response system, using a minimization technique with stratification according to center, age (80 years or older versus below 80 years), gender, and stage (severity of the disease). Investigators and caregivers were not masked to group allocation.

The theoretical framework justifying social worker intervention is to prevent caregiver QoL deterioration and to better support their involvement and communication, thereby contributing to preserving the quality of care provided to the patient. Social worker intervention used the Linear Analogue Scale Assessment (LASA) questionnaire and semi-directive interviews to support the emergence of caregiver needs and specifically address their needs through counselling regarding home services, medical home care, community services (support group), proposing services to promote safety and assist in daily needs (meal delivery, medical alert service), counselling from a psychologist, admission of caregivers for respite care, and encouraging caregivers to take care of themselves and regularly attend consultations with their physician. Interventions from social workers were scheduled at 6, 12, 18, and 24 months (M6, M12, M18, and M24) from inclusion and consisted of a 1 hour visit at the caregiver’s home with the intention of evaluating the level of difficulties experienced by the caregiver using the LASA questionnaire [24], assessing caregiver needs, and detecting early signs of burden through a standardized semi-structured interview (Table 1). Booklets provided access to relevant external assistance structures, support programs, and included information regarding local legislation, administrative procedures, daily living management, and potential consequences related to the caregiving role.

TABLE 1. Social worker intervention, based on the linear analogue self-assessment questionnaire and semi-structured interview performed at caregiver home (Informal Carers of Elderly study, France, 2015–2019).

Variable and Instruments

Besides patient demographics (gender, age, disease), caregiver baseline characteristics mainly included: gender, age, marital status, patient-caregiver relationship, professional activity, profession and incomes, impact from caregiving on professional and financial situation, professional help requested, involvement (daily activities (grooming/dressing…); domestic chores (cleaning, grocery shopping, meals…); Administrative management (accounting, mails, decisions); Medical support (accompaniment to medical appointments, medical cares); Physical support services; Financial assistance; Moral and emotional support; Medical decision support.

Caregiver questionnaires were self-completed at home using paper-pencil or electronically assessed through a secure web platform according to their preferences. Questionnaires were sent to caregivers, regardless of the social worker intervention, in order to prevent any bias of desirability in the SIG.

The SF-36 questionnaire chosen to assess QoL is the most frequently used generic instrument translated and validated in French for a wide range of diseases [19, 25], providing quick answers on specific issues (5–10 min for full completion) and generating physical component summary (PCS) and mental component summary (MCS) scores, as well as scores per dimension (physical functioning, role physical, bodily pain, mental health, emotional role, social functioning, vitality, general health, and health transition). The Hospital Anxiety Depression Scale (HADS) questionnaire used to assess anxiety and depression in different pathologies, in hospitalized and non-hospitalized patients, and also in apparently healthy persons, was validated in French [20, 26]. The Zarit Caregiver Burden Interview (ZBI), also validated in French, has been reported as a reliable tool for the assessment of caregiver burden, i.e., degree of exhaustion or psychological fatigue in caregivers [21].

All questionnaires were self-administrated quarterly except for ZBI administrated each semester. Detailed procedures have been previously reported [23].

Scores for SF-36 subscales were computed on a 0 (worse QoL) to 100 (best QoL) point scale [27]. The minimal important difference (MID) was defined as the smallest change on any scale within an individual or at the group level. SF-36 MID was fixed at 5 points [28]. HADS reported a raw score from 0 (absence of trouble) to 21 (severe trouble) points [20], with an MID for HADS anxiety score fixed at 1.32 points, and for depression a score of 1.4 points [29]. The ZBI reported a raw score from 0 (no burden) to 88 (severe burden) points [21], with MID defined as half of the standard deviation observed at baseline, as usually performed for scores with no previously determined MID. Of note, increased SF-36 scores translated to an improved status, and increased HADS and ZBI a decreased status.

Endpoints

The primary endpoint was QoL SF-36 physical component summary (PCS) and mental component summary (MCS) score changes at M12 and M24 compared to baseline in each group. Secondary endpoints included i) QoL changes in SF-36 subscales, anxiety/depression, and burden at M12 and M24 compared to baseline; ii) longitudinal assessments of caregiver QoL, anxiety/depression, and burden over the 2 years. Exploratory analyses were performed to assess changes from baseline in caregiver involvement in activities of daily living, and to assess professional and financial support requested by caregivers, at M12 and at M24.

Statistical Analyses

Analyses included all caregivers randomized, with the completed questionnaire at baseline, and all data available were considered at each timepoint. Only descriptive analyses per group were conducted.

Socio-demographic, clinical patient characteristics, and caregiver characteristics were described in the global population and in each group. Categorical variables were expressed using number and frequency (n,%), and continuous variables used median (min-max). Mean (standard deviation)/median (min-max) scores were described at each timepoint in both groups. No statistical comparisons between randomization groups were done due to the limited sample size.

Reliability of the questionnaires was evaluated with the assessment of the internal consistency using a Cronbach’s alpha coefficient for each dimension of the SF-36, HADS, and ZBI at baseline. A Cronbach’s alpha of at least 0.70 was expected [30, 31].

To assess potential attrition bias, baseline characteristics of caregivers with early discontinuation at M12 and at M24 were compared to those continuing the study. p-values were provided to help in interpreting the results. Quantitative data were compared using a Student’s t-test. Categorical data were compared using a Chi-square test.

Mean changes in SF-36, HADS, and ZBI scores from baseline to M12 and M24 are presented per group with a paired t-test fixed at 5% for statistical significance.

Mixed models for repeated measures were used for longitudinal analysis including all timepoints up to M24 and including the following effects: randomization group, time, allocation-by-time interaction, adjustment on baseline score, and baseline score-by-time interaction. Random effects on intercept and time were used in order to reflect individual variations. Adjusted mean changes at M12 and M24 were reported with 95% confidence interval for SF-36-PCS and MCS scores, HADS anxiety and depression, and burden, per group. Statistical and MID clinical significances were indicated.

Analyses were performed using SAS version 9.4 software (SAS Institute Inc., Cary, NC, USA).

Results

From October 2015 to May 2019, 183 caregivers were recruited and 179 were randomly assigned in the SIG (n = 90) or in the CG (n = 89). Each caregiver completed questionnaires at inclusion and the completion rates reached reliable percentages (mainly >70%) for each follow-up timepoint. In the SIG, the rates were 59/75 (79%) at M12, and 44/56 (79%) at M24, and in the CG, 54/77 (70%) at M12, and 39/57 (68%) at M24. Details including reasons for early study discontinuation are presented in Figure 1.

FIGURE 1. Trial profile (Informal Carers of Elderly study, France, 2015–2019). Note: Only questionnaires with at least one scored dimension were considered for analysis. ^†Study discontinuation at Month 12 (M12) (Supportive intervention group: n = 15 [caregiver decision, n = 12, patient death, n = 3]; Control group: n = 12 [caregiver decision, n = 8, patient death, n = 4]).^‡Study discontinuation at Month 24 (M24) (Supportive intervention group: n = 34 [caregiver decision, n = 23; patient death, n = 9; relocation in institution, n = 1 outside the region, n = 1]; Control group: n = 32 [caregiver decision, n = 20; patient death, n = 10; caregiver death, n = 1; relocation in institution, n = 1]).

Caregiver Characteristics at Baseline

Caregivers’ median age was 65 (29–92) years, and more caregivers were women (n = 120, 67%) identified as spouse (n = 118, 66%), or as child helping mother/father (n = 42, 23%). The related patient population had a median age of 73 (60–94) years, including 92 (52%) patients with cancer, 66 (37%) with neurodegenerative disease, 11 (6%) with age-related macular degeneration, and 10 (5%) with stroke. Most of the caregivers were retired, however 42 (22%) caregivers still had a professional activity, among them 15 caregivers declared that supportive care had an impact on their professional life and required time arrangement (n = 10). Twenty-three (15%) caregivers declared average household incomes below €1500 per month. At inclusion, 5 (3%) asked for financial support and 25 (14%) for professional support (Table 2). At baseline, most of the caregivers were involved in five main support areas: moral and emotional support (n = 172, 96%), medical decision support (n = 172, 96%), medical support (n = 136, 76%), domestic chores (n = 120, 67%), administrative management (n = 119, 66%). Half of the caregivers declared to provide financial assistance (n = 94, 53%) (Table 2).

TABLE 2. Patient and caregiver characteristics in all randomized caregivers, and in the supportive intervention group and in the control group (Informal Carers of Elderly study, France, 2015–2019).

Reliability of the questionnaire was reached with a Cronbach’s alpha coefficient of at least 0.70 for each dimension (ranging from 0.79 to 0.90 for each dimension).

Caregivers with early discontinuation at M12 and at M24 showed more unfavorable characteristics at inclusion (Supplementary Material S1).

Health-Related Quality of Life

In SF-36-PCS and -MCS mean scores, no clinically significant decreases were observed at M12 or at M24 in each group. However, a statistical difference for PCS at M12 was noted in each group (Table 3).

TABLE 3. Short Form-36, Hospital Anxiety Depression Scale and Zarit Burden Interview, mean scores at baseline and mean score differences at months 12 and 24 in the supportive intervention group and in the control group (Informal Carers of Elderly study, France, 2015–2019).

Regarding the SF-36 subscales, the CG showed a clinically and statistically significant decrease in the mean score for physical functioning and bodily pain at M12 and M24, physical role, social functioning, and vitality at M12. In the SIG, a significant decrease in the mean scores in physical functioning were reported at M12 and M24, and in physical role and social functioning at M12 (Table 3).

Anxiety and Depression

While no difference in anxiety mean score was reported, depression mean score was clinically and statistically increased in the CG at M12 and M24. No differences in anxiety and depression mean scores were reported in the SIG regardless of each timepoint (Table 3).

Burden

Mean ZBI score was not clinically different from baseline to each timepoint in each group. Of note, a statistically significant increased ZBI score at M12 and M24 was observed in the CG (Table 3).

Longitudinal Analysis

The mixed models for repeated measures did not show clinically significant differences at M12 or at M24 for SF-36, HADS anxiety, and ZBI (Table 4). However, a significant change in HADS depression score at M24 was identified in the CG (adjusted mean change 3.4 [0.6–2.5]).

TABLE 4. Caregivers adjusted mean change over time (mixed model for repeated measures) in Short Form-36 physical component summary, mental component summary, in Hospital Anxiety Depression Scale, and Zarit Burden Interview in each allocation group (Informal Carers of Elderly study, France, 2015–2019).

Exploratory Analyses

Exploratory analyses showed several significant changes in caregiver involvement in medical support at M12, in financial assistance (M12, M24), and in medical decision support at M24 in the SIG. In the CG, the rates of caregivers providing help in medical decision support at M12, and in moral and emotional support at M12 and at M24 were significantly reduced compared to baseline. The rate of caregivers requesting professional and financial support was low, regardless of the allocation group. Only a slight increase in professional support was requested at M12 and M24 in the intervention group (data not shown).

Discussion

The present analysis of 179 caregivers showed no clinically relevant changes in QoL summary scores (SF-36-PCS and -MCS), nor in HADS anxiety and ZBI burden scores, regardless of the allocation group. HADS depression scores significantly increased at M12 and M24 exclusively in the control group, and the mixed models confirmed the mean change results in the control group at M24 (adjusted mean change: 3.4 [0.6–2.5]). No direct comparisons between groups were allowed based on the current sample size.

The deterioration in QoL physical sub-dimensions reported by caregivers in the first 2 years of the ICE study is consistent with previously reported results [32, 33]. Correlation between caregiving stress and physical impairments after 2 years was also previously underlined [32].

In the SIG, no significant changes in depression, and less QoL sub-dimensions with significant deterioration were reported, indirectly reflecting a benefit from social worker intervention. Previous studies investigating intervention in caregivers of patients with dementia also reported quite limited but significant decrease in caregiver depression and burden [34]. Similarly, interventions targeting self-care and interpersonal connections of caregivers, as well as symptom management in adult cancer patients alleviated depression and improved caregiver QoL [35]. The additional support in the SIG contributed to minimizing depression, which is consistent with lower depression scores observed in caregivers with more social relationships as reported by Stenberg et al. [33]. However, in contrast to results from Stenberg et al., our study did not evidence reductions in anxiety and burden.

Caregivers play a critical role and assume in turn several tasks in disease management of patients with cancer [36]. Earlier and better recognition of caregiver involvement in patient care needs to be encouraged. Further issues had arisen regarding the growing recognition of specific caregiver needs, and the most appropriate assistance to offer. Hence, compromised caregiver QoL will also adversely influence the delivery of effective patient care [37].

Clinicians underlined difficulties to appropriately identify the primary caregiver [38]. This finding may contribute to explaining the limited enrolment of caregivers in the ICE study. Caregivers have previously been qualified as “the invisible patient” [39]. The ICE study enrolled nearly 200 caregivers, and although this sample size—far below the pre-planned recruitment—did not allow comparisons in caregivers per patient disease and according to allocation groups, this prospective study gathered a substantial number of caregivers, favorably comparing with previous studies, and pointed to difficulties in reaching this specific population that still need to be overcome [40, 41]. The collaborative efforts to improve access to this informative mixed population of caregivers involved in the support of older patients with different diseases need to be emphasized. Better understanding of the working methods of social workers and improved coordination between social and health systems could help to identify and reach the caregivers more effectively. Further studies specifying caregivers needs, considering the initial pathology of their loved one, and assessing need evolution and changes according to each disease trajectory would be required. A strength of the ICE study is to clearly underline the increased depression over time in the control group, confirmed with the mixed model. This result is consistent with previous reports [32, 33, 42]. Moreover, the informative approach provided by the longitudinal follow-up of 2 years needs to be highlighted. However, the global interpretation of the changes in QoL should be cautious because of attrition biases. Indeed, caregivers having discontinued the study early were the most at risk for limited QoL with poor physical dimensions and high burden at baseline in caregivers withdrawn at M12. To what extent social support contributes to alleviating depression needs to be further investigated.

To date, no specific assessment of living conditions, including accurate identification of caregiving is performed at the initiation of patient management, and conducting research on caregivers remains a challenging issue [43], and other initiatives either considering patient-caregiver dyads, or family-centered approaches should also be encouraged and integrated in further clinical studies and in patient care. A framework to address the needs of caregivers has been recently proposed in early palliative care in cancer [12]. Further evidence-based studies with interventions from social workers, nurses, support groups, or integrated in early palliative care, and collaborations thereof are promising alternatives which need to be further explored [11–13].

The innovative and ambitious ICE program received funding from the National research agency and the French National Institute of Cancer and aimed to raise awareness and recognition of the role caregiver, improve self-recognition and support, and consider caregivers as part of the unit of care. Along with large public information campaigns in the last years, incentives to strengthen the relationship between health and humanities and social sciences structures are encouraged. The ICE study focused on the support to be delivered to the still difficult to reach population of caregivers, on an individual level. The study provided useful insights into caregiver characterization. While caregiver depression increased over time in this population, social worker intervention proved promising. These results underlined the critical need to better support caregivers of older patients and urge support to be adjusted with adequate methods, resources, and timing elements. Indeed, social workers reported that they mainly had a role of listening and returned that caregivers had not taken advantage of the full scope of opportunities they may provide. Several assumptions may explain such missed opportunities. The protocol scheduled semester visits that did not correspond to the current social worker practice, usually proposing more closely spaced visits. This may have contributed to limiting the ability of social workers to provide appropriate timely support. More frequent visits, organized at the request of caregivers, would have encouraged the emergence of needs; therefore, caregiver needs could have been more appropriately and efficiently addressed.

The term “informal” is largely used worldwide to qualify unpaid caregiving. However, this term may still lack conceptual clarity, and controversial views have been reported highlighting the negative connotation that “Informal” may include, faced with the substantial efforts in supporting and directly caring for patients over a long period of time that the caregivers, identified or not, supported or not, currently provide.

Health policy developments and local regulation have already evolved in recent years and have been recognized as essential to provide adequate support to this vulnerable population and new institutional structures have been created to alleviate caregiver burden in France [44], and other initiatives are currently being tested worldwide [45, 46]. Providing caregivers with an opportunity to rest and recover is essential for maintaining their capacity to care for their loved ones, and greater recognition of needs, issues, and rights during disease course is required. Caregiver respite and emotional needs are of particular importance at specific periods/timepoints, notably in assisting cancer patients in the first 12 months after diagnosis [47].

Conclusion

A global approach strengthening collaborations between social and health systems proved promising. Social support may contribute to preventing or reducing caregiver depression at 1 and 2 years and therefore prevent deterioration in global caregiver QoL. If randomized studies are needed to further define and investigate personalized reliable interventions, a greater use of the large currently available expertise of social workers, supported by already existing nationwide networks is also a contributory promising approach. This social involvement would allow to better reach the caregivers, in a complementary and consistent manner, to provide better identification and characterization of caregivers, and a better understanding of caregiver needs, especially early in the course of the patient disease. Timely organization of dedicated scalable support to the caregivers the most in need could be implemented. Tailored interventions from all available sources, including help provided by the social system, are required to adequately support this important, vulnerable, and still too invisible population.

Data Availability Statement

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Ethics Statement

This study received approval from the local Ethics Committee (Comité de Protection des Personnes Est-II, Besançon) on November 4th, 2014 and from the French advisory committee on information processing research in the field of health (Comité Consultatif sur le Traitement de l’Information en Matière de Recherche dans le Domaine de la Santé, CCTIRS) (protocol number 15.022-bis) on March 5th, 2015. Approval from the National authority for the protection of privacy and personal data (Commission Nationale Informatique et Libertés, CNIL) was obtained on June 23rd, 2015. Authors had full access to all data in the study, reviewed this manuscript and accepted responsibility to submit for publication.

Author Contributions

CL, MB, TS-DY, PF, MG, and PM participated in the design of the study. CQ ensured the coordination of social worker intervention and administrative support. MM was responsible for data collection. AM was responsible for the data management. SD drafted the manuscript. AA was responsible for the study methodology and drafted the manuscript. AP coordinated the study, realized statistical analysis and drafted the manuscript. VN was the principal study coordinator, participated in the design of the study and drafted the manuscript. All authors reviewed the manuscript.

Funding

The study received funding from the Agence Nationale de La Recherche (ANR) : ANR-16-CE-36-0006-01, Institut National du Cancer (INCa) : SHSESP13-007, La Ligue Contre le Cancer : CIR-GE-2013, Roche Foundation, and la Caisse Nationale de Solidarité pour l’Autonomie (CNSA). The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report.

Conflict of Interest

The authors declare that they do not have any conflicts of interest.

Acknowledgments

The authors thank all the investigators and all clinical research assistants involved in the study. We also would like to thank the Aidants Attitude organization for the booklet and all patients and caregivers that participated in the study. The authors want to mention that the large program ICE was initiated by the late Professor F. Bonnetain, responsible of the methodology and quality of life in cancerology unity of the hospital of Besançon, France. The Communal Centers for Social Action of Dijon and Besançon, the General Council of the Doubs and the Territoire de Belfort; the Caisse d’Assurance Retraite et de la Santé au Travail Bourgogne and Franche-Comté and the Agricultural Social Mutual provided social workers for the social support. We acknowledge the availability of an online pre-print via the link https://www.researchsquare.com/article/rs-1349746/v1 [48].

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.ssph-journal.org/articles/10.3389/ijph.2023.1605459/full#supplementary-material

References