Table 1 Crosswalk of Study Advisory Committee (SAC) recommendations, Coded Themes/Subthemes, Shippee framework phase and Institutional Review Board (IRB) modification

1

Investigate how caregivers cope

Scientific

Design advice

I.1

Implemented the short-form Zarit Caregiver Burden survey into screening tool at 3, 6, and 12-month follow-up surveys

Yes

2

Consider the geographic location differences of the enrolled sites. How do you reach patients in rural areas where outpatient palliative care clinics are not available?

Scientific

Conceptual suggestion

I.1

This recommendation was provided prior to the implementation of telehealth outpatient visits as enrolling sites are all primarily located in urban or suburban locations. In the patient enrollment forms the research team captured information related to the patient’s home address and the address of the outpatient clinic and the team is anticipating analyzing if distance to/from the clinic was a barrier. Additionally, the pandemic provided an opportunity to revisit care delivery techniques and allowed patients to receive outpatient palliative care via telehealth

Yes

3

Explore the potential barriers to palliative care access that may impact engagement across the intervention arms (e.g. distance from clinic/services, income etc.)

Scientific

Design advice

I.1, II.2, II.3, II.4

This recommendation was incorporated into data collection and analysis plans. For example, survey instruments were updated to ensure potential barriers were collected and analysis plans were updated to account for different barriers

Yes

4

Update materials to include more exciting terms so patients do not associate palliative care with death

Scientific

Recommendation for supplementary study materials

II.1

RCs were trained on the appropriate methods of describing palliative care with patients and caregivers

No

5

Implement a satisfaction survey for patients in an effort to improve the intervention

Scientific

Design advice

II.1

No patient satisfaction survey was formally collected as this was outside the scope of the research study. Nonetheless, any time study participants provided feedback it was brought to the research team’s attention and discussed

No

6

Expand inclusion criteria to Spanish Speaking participants

Scientific

Design advice

II.1

The recruitment criteria was expanded to include Spanish Speaking participants

Yes

7

Consider adjusting protocols for patients with disabilities specifically mental disabilities and other conditions that impact this group of study participants (ex. Dementia)

Scientific

Design advice

II.1

In the screening tool, a dementia question was added to assess whether dementia was included on the patients’ active problem list in Epic, signaling exclusion. RCs made accommodations for any patients who required by prioritizing certain questions within the follow-up surveys, requesting presence of caregiver during survey completion etc

Yes

8

Explore how the outpatient study arm will measure loneliness

Scientific

Design advice

II.1

The team incorporated the validated University of California Los Angeles (UCLA) Three-Item Loneliness Scale into the 3, 6, and 12 month patient follow up surveys to measure loneliness for both study arms

Yes

9

Ensure patients or caregivers are familiar with the study and a telephonic nurse may be calling them (if enrolled in that study arm)

Scientific

Recommendation for supplementary study materials

II.1, II.2

A welcome packet was developed and sent to each patient and caregiver enrolled

Yes

10

Provide each patient with a welcome packet that is bright and at an appropriate reading level

Scientific

Recommendation for supplementary study materials

II.1, II.2

A patient facing brochure was developed in partnership with the SAC members

Yes

11

Enrolled caregivers should feel part of the research process

Scientific

Design advice

II.1, II.2

A separate caregiver screener with questions specific to caregivers’ emotions, feelings, health was implemented

Yes

12

Shorten welcome letters, as drafts were too lengthy

Scientific

Recommendation for supplementary study materials

II.1, II.2

With the SAC’s feedback, the patient facing brochure was updated to include a shortened welcome letter. All patient facing materials were reviewed by the SAC

Yes

13

Have patient-facing visuals for palliative care at the time of recruitment to provide patients and caregivers with an overview of the study

Scientific

Recommendation for supplementary study materials

II.1, II.2

In partnership with the SAC members, the research team created a visual describing palliative care that is reader friendly

Yes

14

Include more images than text in patient facing materials

Scientific

Recommendation for supplementary study materials

II.1, II.2

RCs continued to assist patients where needed, and incorporated visuals of the study timeline alongside EMPallA descriptors that were required to be provided by regulatory guidelines

No

15

Replace the use of the word "caregiver" with other similar words that are more sensitive to the population

Scientific

Recommendation for supplementary study materials

II.1, II.2

Theae changes were implemented throughout study materials and communicated to research coordinators. Instead of “caregiver, the terms supportive care, companion, helper, support system were used

Yes

16

Use the terminology “stress” instead of “burden” when communicating with patients and caregivers

Scientific

Recommendation for language use

II.1, II.2

Language was adapted in the REDCap screener, scripts, and subsequent follow-up surveys

Yes

17

Implement telehealth video option during pandemic when clinics were significantly limiting in-person visits to reduce risk of COVID-19 for this vulnerable population

Scientific

Design advice

II.1, II.2

This recommendation resulted in a major protocol change. Originally, outpatient clinics only met patients randomized in this study arm in-person, but the study was adapted to allow telehealth visits

Yes

18

Create a video to introduce the study to the patient and caregiver to enhance recruitment

Scientific

Recommendation for supplementary study materials

II.2

Due to limited resources a video could not be developed, but the research team created resources summarizing the study for all patients and caregivers, including visual timelines and full descriptions

No

19

Shorten recruitment scripts

Scientific

Recommendation for supplementary study materials

II.2

As scripts were developed with IRB regulatory guidelines in mind and could not be modified as consistent recruitment processes had to be standardized across all sites

No

20

Be mindful of resources available in each enrolling Emergency Department

Pragmatic

Implementation advice

I.1

Research teams were encouraged to train RCs on respective sites’ available resources. RCs were also encouraged to share unique resources during peer-to-peer meetings that may be helpful to their partnering sites

No

21

Understand the impact of spiritual differences

Pragmatic

Implementation advice

I.1 II.1

A standard religion question was added in the screening survey. The specific question added was “Are you a member of a faith community?”

Yes

22

Be conscientious of where the patient is from and their unique healthcare needs

Pragmatic

Implementation advice

I.1, II.1

Each RC was provided open user rights within the electronic health record to better understand each patient enrolled (example, a RC was able to see if the patient was currently in the hospital, or if they had a recent encounter with a physician etc.)

Yes

23

Develop a phone script that adequately addresses caregiver duress

Pragmatic

Recommendation for language use

II.1

Research Coordinators (RCs) were trained by SAC members on appropriate language to use while speaking with caregivers

No

24

Implement pre-visit reminder calls

Pragmatic

Implementation advice

II.1

Before each Outpatient visit, the RCs would remind patients of their upcoming appointment. This data was documented in the REDCap Outpatient Log

Yes

25

Do not e-mail gift cards to participants

Pragmatic

Advice about dissemination

II.1

Given the unique patient population recruited for this study, the research team worked with their internal finance team to provide incentive gift-cards either in-person or physically sent in the mail

Yes

26

Provide more resources directly to caregivers to help them understand the benefits of palliative care

Pragmatic

Implementation advice

II.1

SAC members who served as caregiver advocates trained the RCs the most appropriate ways to interact with caregivers

No

27

Integrate the use of a translator services as they are used during emergency department visits, and Spanish-speaking patients should be able to use this mode of communication

Pragmatic

Implementation advice

II.1, II.2

The research coordinator protocol was updated with information related to using the translator phone

Yes

28

When recruiting in-person (with patient’s verbal permission), use their phone to contact their caregiver to see if they are interested in participating in the study. This was to avoid barriers related to robo and spam calls, and the caregivers not picking up their phone from an unknown telephone number

Pragmatic

Implementation advice

II.1, II.2

This suggestion was adapted and research coordinators across all recruitment sites when appropriate would use the patient’s telephone to directly call their caregiver to see if they were interested in participating in the study

Yes

29

Follow-up with each participant after study enrollment to keep patient engaged for the duration of the 6-month intervention and 12-month data collection

Pragmatic

Implementation advice

II.1, II.2, II.3

A postcard was created and was mailed and or provided to each participant after enrollment reminding them about the study benefits and expectations. Research coordinators would also call participants

Yes

30

Explain to the patient the definition of a caregiver before asking patient if s/he has a caregiver who wants to participate in the study

Pragmatic

Recommendation for language use

II.1, II.3

Recruitment scripts were updated, and discussions related to overcoming hurdles in recruiting caregivers were covered in biweekly peer-to-peer research coordinator learning collaboratives

Yes

31

Have research staff assist in coordinating with patient’s other physicians

Pragmatic

Implementation advice

II.2

During the recruitment process, each RC would enter the contact information for the patient’s providers in the study’s secure database REDCap. If the patient was enrolled in the telephonic study arm the nurse would assist with care coordination processes

Yes

32

Ensure that Outpatient Palliative Care clinics are well staffed

Resource

Staffing infrastructure suggestion

I.1

No changes could be made to the staffing infrastructures at each external outpatient clinic, but the research team worked with outpatient palliative care clinics to understand staffing trends, hiring updates and strategic interim plans to ensure that capacity will allow for EMPallA patient care

No

33

Hire additional Spanish speaking research coordinators at each enrolling site

Resource

Staffing infrastructure suggestion

II.1, II.2

Due to budgetary constraints and inability to find appropriate staff this was not feasible at each enrolling institution. Nonetheless, the research team recommended that bi-lingual qualifications be included in hiring criteria. Where this could not be accommodated, research teams made use of hospital translator phones to ensure that the Spanish-speaking population could accurately be reached

No

34

Ensure all Spanish patient facing materials are culturally sensitive

Resource

Patient-centered suggestion

II.1, II.2

All materials were directly translated from original documents to maintain a standardized tone and reviewed by Spanish speaking Latinx SAC member

Yes

35

Implement mock Spanish recruitment trainings

Resource

Patient-centered suggestion

II.1, II.2

Leveraging our SAC member with expertise in recruiting Spanish-speaking participants, the research team implemented focused trainings on recruiting Spanish-Speaking participants between the SAC member and each individual research coordinator responsible for recruitment

Yes

36

Implement mock recruitment training for new enrollment sites that were added to the study design

Resource

Patient-centered suggestion

II.1, II.2

The research team scheduled time between SAC members and not only new enrollment sites, but also newly hired RCs to conduct mock recruitment

No

37

Involve Patient stakeholders to train new research team members via WebEx conferencing on the research study pitch language used to describe palliative care

Resource

Patient-centered suggestion

II.1, II.2

SAC members trained RCs on a routine basis to ensure that staff on the ground was provided with updated resources throughout study enrollment

No

38

Assess the feasibility of covering patient’s physician costs

Resource

Patient-centered Suggestion

II.2

Patient Centered Outcome Research Institute (PCORI) funded project cannot cover physician related costs, but this conversation was the impetus of adding “insurance status” as a variable in the patient screening process

Yes

39

When reviewing data internally (SAC and research team), update bar graphs to include patient and caregiver enrollment numbers on the same bar graph

Dissemination

Advice about dissemination

II.4

The research team’s Data Analyst updated this figure to consistently report patient and caregiver enrollment targets and goals on the same bar graph for visual ease

No

40

When reviewing data internally (SAC and research team), always include ethnicity enrollment breakdown for patients and caregivers

Dissemination

Advice about dissemination

II.4

The research team’s Data Analyst consistently provides race/ethnicity data when presenting enrollment updates during the internal team meeting

No

41

Disseminate study progress to patients while the study is ongoing. One Suggestion included having a website for patients to see enrollment numbers and target goals, results etc

Dissemination

Advice about dissemination

III.1

Once primary data collection and analysis concludes, study results will be disseminated to study participants

No