Article Text
PDF
Abstract
Objectives To compare user experiences of 8 regional urgent and emergency care systems in the Republic of Ireland, and explore potential avenues for improvement.
Design A cross-sectional survey.
Setting Several distinct models of urgent and emergency care operate in Ireland, as system reconfiguration has been implemented in some regions but not others. The Urgent Care System Questionnaire was used to explore service users' experiences with urgent and emergency care. Linear regression and logistic regression were used to detect regional variation in each of the 3 domains and overall ratings of care.
Participants A nationally representative sample (N=8002) of the general population was contacted by telephone, yielding 1205 participants who self-identified as having used urgent and emergency care services in the previous 3 months.
Main outcome measures Patient experience was assessed across 3 domains: entry into the system, progress through the system and patient convenience of the system. Participants were also asked to provide an overall rating of the care they received.
Results Service users in Dublin North East gave lower ratings on the entry into the system scale than those in Dublin South (adjusted mean difference=−0.18; 95% CI −0.35 to −0.10; p=0.038). For overall ratings of care, service users in the Mid-West were less likely than those in Dublin North East to give an excellent rating (adjusted OR 0.57; 95% CI 0.35 to 0.92; p=0.022). Survey items relating to communication, and consideration of patients' needs were comparatively poorly rated. The use of public emergency departments and out-of-hours general practice care was associated with poorer patient experiences.
Conclusions No consistent relationship was found between the type of urgent and emergency care model in different regions and patient experience. Scale-level data may not offer a useful metric for exploring the impact of system-level service change.
- HEALTH SERVICES ADMINISTRATION & MANAGEMENT
- PRIMARY CARE
- ACCIDENT & EMERGENCY MEDICINE
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first nationally representative study exploring service user experience of urgent and emergency care systems.
The study context facilitates the comparison of regions that have undertaken extensive system reconfiguration with those that have not.
The analysis identified specific aspects of care that were negatively associated with patient ratings of their experience.
Few significant differences were found between regions, suggesting either that patient experience was not linked to system reconfiguration, or the survey measure used was not suitable for making comparisons at system level.
Introduction
Urgent and emergency care consists of all the services which manage patients seeking immediate attention for a health condition, and the processes for referring and transferring patients between services.1 Studies of patient experience in this field have focused on interactions with individual services rather than whole episodes of care.2 ,3 Such studies fail to account for the fact that patients often use multiple services before receiving definitive care.4
Policy initiatives designed to improve urgent and emergency care often focus on improving system operation. These include the differentiation of hospital functions, specification of ambulance bypass protocols and the introduction of alternative care models.5 Such initiatives generally aim to ensure that patients receive care in a setting that is appropriate to the severity of their condition as quickly as possible, while also preserving local access and reducing the duplication of services within a critical proximity. In a well-functioning system, patients should choose or be directed to the service that is most appropriate to their condition, progress through the system smoothly and feel as though their care is well co-ordinated.1 Urgent and emergency care services have been reconfigured to varying degrees across the Republic of Ireland since 2006 (outlined in table 1). These initiatives have been planned at a regional level and, where implemented, have attempted to direct patients to settings that are appropriate to the severity of their condition. Common features of these programmes include centralisation of specialist urgent and emergency care at a ‘hub’ hospital, and integrated ambulance and general practice (GP) referral protocols for given conditions.
Regional characteristics and service reconfiguration summary, 2006 to present
In this study, we use a recently developed survey methodology4 to compare patient experience in different regions. We hypothesise that service users in regions which have undertaken reconfiguration of urgent and emergency care will report a better experience than patients in regions which have not. We also seek to identify aspects of patient experience where improvements may be possible.
Methods
Design and setting
A cross-sectional survey of the general public across eight regions covering the Republic of Ireland was conducted from March to June 2015. The characteristics of each region and measures taken to reconfigure services are presented in table 1. Two regions (South and Mid-West) have implemented significant reconfiguration of urgent and emergency care. Four regions (West, North-East, Dublin South, South-East) have introduced some measures designed to reconfigure care but these do not cover all services. Two regions (Dublin Midlands and Dublin North-East) have undertaken no major changes since 2006. Further details on urgent and emergency care provision in Ireland are provided in online supplementary file 1.
supplementary file
Sampling
Data were collected by a market research company using computer-assisted telephone interviewing. Random-digit dialling was used to contact landline and mobile telephone numbers. Quota-controlled sampling was used, whereby the market research company was required to contact a sample in each region that was representative of the age and sex profile of the national population. Approximately 1000 interviews were conducted in each region in order to fill these quotas. A screening question was used to identify recent users of urgent and emergency care.
Recent users were defined as adults or parents of children (under the age of 16) who self-identified as having used a healthcare service on an urgent or emergency basis in the previous 3 months. An ‘urgent and emergency basis’ was explained to respondents as a problem where help or advice from a healthcare service was needed on the same day.
On the basis of a previous study in England, it was estimated that a survey of 1000 respondents per region would yield ∼150 recent users of urgent and emergency care.2 A sample size of 150 service users would provide 80% power at the 5% significance level for regional patient experience comparisons on the entry into the system scale of the Urgent Care System Questionnaire (UCSQ), assuming an SD of 0.8 and with the intention to detect a difference of half an SD on this scale.4
Measures
An adapted version of the UCSQ was used to assess service users' experience of their most recent contact with urgent and emergency care services. Cognitive testing and piloting were conducted to assess and adapt the survey instrument to fit the Irish healthcare context prior to the main data collection phase. Cognitive testing and piloting were conducted to assess and adapt the survey instrument to fit the Irish healthcare context prior to the main data collection phase. Cognitive testing involves testing quantitative questions through a series of qualitative in-depth interviews to see how respondents understand, retrieve information for, decide on and ultimately arrive at responses to those questions. While the cognitive testing interviews indicated that the original questionnaire was working well overall, a number of potential changes were highlighted by the process. A cognitive testing report was created which included minor wording changes, clarification for answer responses and additional or more descriptive interviewer instructions and notes were added to the questionnaire. The full version of the UCSQ instrument is available in online supplementary file 2.
supplementary file
This instrument measures patient experience using 18 items across three scales: entry into the system, patient convenience of the system and progress through the system. Survey items were derived from qualitative research with users of urgent and emergency care and were designed to capture users' recent experience with the system.6 Participants were asked to indicate their agreement with each of the 18 items along a five-point scale from strongly disagree to strongly agree. Scoring was reversed for negatively worded items. A separate item was used to assess participants' overall rating of the care they had received, with six response options ranging from ‘very poor’ to ‘excellent’.
Participants were asked to indicate which of the following services they had used in their episode of care: pharmacist, GP (in-hours and out-of-hours), ambulance service, emergency department (ED), local injury unit, mental health service, public health nurse or other (patient-defined). Finally, participants were asked to indicate the condition that had precipitated their contact with urgent and emergency care service from the following options: illness, injury, adverse reaction, infection, mental health issues, other, not stated.
Statistical analysis
We compared patient experience ratings across the three UCSQ scales and overall ratings of care across regions. Item-level responses were also described at national and regional levels.
Linear regression was used to identify regional differences in the three UCSQ scales, with entry into the system, convenience of the system and progress through the system as the outcome variable, adjusted for sex, age group, services used and condition precipitating care episode.
Logistic regression was used to identify regional differences in the proportion of patients who rated their overall care experience as excellent, adjusting for sex, age group, services used and condition precipitating care episode. It has been suggested that evaluations of patient experience should focus on optimal responses (eg, strongly agree, excellent) to provide a guide to the amount of improvement that may be possible and this is the approach we have followed in our analyses.11 ,12
In all regression models, we used the region with the best performance as the base comparator. All statistical tests were two-sided and p values <0.05 were considered to represent a statistically significant result. SPSS V.22 was used to analyse the data.
Results
In total, 8002 interviews were conducted, with 1205 interviewees self-identified as recent service users. Since a non-probability quota-based sampling method was used, it is not appropriate to report an overall figure for response rate. There was little variation in the characteristics of respondents across regions (see table 2). There was some variation in the services used, especially out-of-hours GP care which varied from 28.9% of participants in the South region to 12.2% in Dublin South. Participants in the Mid-West used the most services on average (2.48), with participants in Dublin North East using the least (2.05). In addition, there were noteworthy differences in the conditions that precipitated contact with urgent and emergency care services. For example, in the South, illness accounted for 73.2% of contacts, compared with 59.8% in the West.
Demographic characteristics and services used by participants across regions
Table 3 outlines the percentage of respondents who endorsed the optimum response option to each item and the scale scores for each UCSQ domain. At the item level, optimal response proportions varied from as low as 17.6% to as high as 45.8%. The three lowest rated items, when adjusted for negative wording, were: ‘I was told how long I'd have to wait’, ‘Services did not seem to talk to each other’ and ‘Services understood that I had responsibilities, like my need to look after my family’. The three highest rated items were ‘I felt that the first service I tried was the right one to help me’, ‘I did not know which service to go to about this problem’ and ‘I was made to feel like I was wasting everyone's time’. There were no instances where the proportion of patients in a region that endorsed the optimal response differed from the national figures by more than or <10%. Additional detail on item-level responses is available in online supplementary file 3.
Percentage response to optimal option for survey items (scores reversed for negatively worded items) and mean values for UCSQ domains
supplementary file
There was little regional variation in UCSQ scale scores. Dublin North East had significantly lower ratings on the entry into the system scale than Dublin South (adjusted mean difference=−0.18; 95% CI −0.35 to −0.10; p=0.038). Use of GP in-hours (coefficient=−0.13; 95% CI −0.24 to −0.03; p=0.013), GP out-of-hours (coefficient=−0.18; 95% CI −0.30 to −0.62; p=0.003), a public ED (coefficient=−0.31; 95% CI −0.41 to −0.20; p<0.0005), a private ED (coefficient=−0.38; 95% CI −0.61 to −0.15; p=0.001) and other services (coefficient=−0.63; 95% CI −0.99 to −0.27; p=0.001) was independently associated with lower entry into the system scale scores.
No significant differences between regions were found for patient convenience. Use of a public ED was the only significant independent predictor of variation in this scale and was associated with lower ratings (coefficient=−0.55; 95% CI −0.67 to −0.41, p<0.0005).
For progress through the system, the West region was found to have significantly lower ratings than the South region (−0.18; 95% CI −0.36 to −0.01; p=0.043). No other regional differences were identified. Use of GP in-hours (coefficient=−0.23; 95% CI −0.34 to −0.13; p<0.0005), GP out-of-hours (coefficient=−0.20; 95% CI −0.33 to −0.80; p=0.001), public ED (coefficient=−0.42; 95% −0.50 to −0.31; p<0.0005) and other services (coefficient=−0.43; 95% CI −0.80 to −0.06; p=0.024) were significant and negative independent predictors of variation in this scale. Use of an emergency ambulance (coefficient=0.18; 95% CI 0.02 to 0.34; p=0.032) and an episode precipitated by illness (coefficient=0.21; 95% CI 0.02 to 0.39; p=0.027) were independently associated with higher ratings on this scale.
Of the total sample, 35.9% rated their care as ‘excellent’. One statistically significant difference in ratings of excellence across regions was found: service users in the Mid-West were less likely than those in Dublin North East to give an excellent rating (adjusted OR 0.57; 95% CI 0.35 to 0.92; p=0.022). In addition, users of a public ED (adjusted OR 0.60; 95% CI 0.44 to 0.82; p=0.001) and GP out-of-hours (adjusted OR 0.66; 95% CI 0.47 to 0.92; p=0.015) were less likely to report an excellent rating of care than those who had not used these services, as were younger participants (adjusted OR 0.81; 95% CI 1.10 to 1.37; p<0.0005).
Discussion
Overall, 35.9% of service users in this study rated their care as excellent, slightly lower than figures from two studies conducted in the UK where ratings of 38.5% and 41.5% were reported.4 ,13 UCSQ mean scale scores were also slightly lower than those reported in the UK.1 ,4 ,13
The study has not revealed a strong association between regional care models and patient experience scale scores despite notable regional differences in service users' reasons for contacting urgent and emergency care, and the types of services they used. Indeed, when assessing overall ratings of care, service users in the Mid-West, which has recently undertaken a wide-ranging series of changes to urgent and emergency care services, reported significantly lower ratings than those in Dublin North East, which has undertaken little change. The Mid-West is a predominantly rural region with a relatively low population density, where emergency care is centralised to a single centre, while Dublin North East is a densely populated urban area containing three large public EDs. Participants in the Mid-West used more services than those in Dublin North East, implying a longer pathway to definitive treatment. This has previously been shown to be negatively associated with patient experience.1 The Mid-West reconfiguration programme was designed to ensure that patients received care in a setting that is appropriate to the severity of their condition as quickly as possible:10 our study suggests that this may not have been achieved, but a definitive conclusion is not possible without data from the period before reconfiguration occurred. The other regions where reconfiguration took place, the South and North East, also have relatively low population densities but participants in these regions used comparatively fewer services than those in the Mid-West and gave overall ratings of care that were above the national average. In these regions, some centralisation of emergency care has taken place but at least two EDs remain in operation, compared with just one ED in the Mid-West. In the regions where little or no reconfiguration has taken place, several EDs continue to operate, often with little formal integration of governance structures across sites and services.
It is possible that ongoing public negativity around the reconfiguration of acute hospital services in the Mid-West may have influenced participant responses. Changes to healthcare provision often meet with public opposition, characterised by a rejection of the justifications for change advanced by policymakers.14 ,15 For example, previous studies exploring patient experiences of healthcare reconfiguration have failed to identify consistent positive patient responses to changes in gynaecology services16 and GP out-of-hours provision.17
The respondents in this study were particularly dissatisfied with aspects of their care relating to communication and most satisfied with aspects of care around accessing the appropriate service and feeling cared for. These patterns of item endorsement are very similar to those found in a UK study using the same instrument.4 Previous studies of patient experiences of primary care and EDs have also identified communication around waiting times and physician empathy as important predictors of patient perceptions.18 ,19 In the Irish context, this may be a signal that while there have been many changes directed at individual urgent and emergency care services, there is still insufficient attention to the delivery of a whole system approach to communication between providers. Our study suggests that there is a particular problem with care episodes that involve contact with public EDs and out-of-hours GP care.
Strengths and limitations
This is the first national study of the patient experience of different urgent and emergency care models. For patient experience studies to be worthwhile, they must produce recommendations on attributes that are modifiable by policymakers, and in this paper, we present data on the aspects of care that are in greatest need of attention.18 The survey method employed in this study is an innovative way to gain access to people who have had an urgent or emergency care episode. Since recruitment did not require the cooperation of individual service providers, we are able to provide a comprehensive analysis of the experience of service users in all parts of Ireland over the reference period. However, the survey method has several potential sources of bias. First, some participants may have accessed services outside of their region of residence, potentially impacting on the accuracy of regional data. Second, patients who died following contact with urgent and emergency care services are not covered by the study. Finally, only individuals with phones were sampled, potentially excluding some service users.
The survey instrument identified few significant differences between regions. It is possible that policy initiatives designed to create streamlined regional care systems have not been successful. It is also possible that it is still too early in the implementation cycle to judge the success of these initiatives, in which case the current study provides a useful baseline for future patient experience studies. Alternatively, it is possible that UCSQ scale scores are not sensitive to the signal of regional differences in patient experience. This may be because the instrument covers a very broad spectrum of severity: it may be more useful to focus only on patients with more serious healthcare episodes where a well-functioning system is a strong determinant of experience.
Finally, it is important to note that the generalisability of study results may be somewhat limited outside of the Irish context, which has its own unique configuration of urgent and emergency care provision. However, comparing various models of care situated within the same national healthcare context may be viewed as a strength of the study, as it facilitates comparisons that are not always possible when comparing across national boundaries due to varied system compositions and policy contexts.
Conclusion and implications
There does not appear to be a consistent relationship between the type of regional care model implemented in different parts of Ireland and patient experience. Scale-level data may not offer a useful metric for comparing different models of care as few differences were detected at this level. Instead, it is recommended that policymakers in Ireland use the item-level data produced in this study as a baseline against which future efforts to improve urgent and emergency care experiences are measured longitudinally. Inability to explore ‘before and after’ comparisons hampered the ability of this study to make meaningful inferences about the impact of changes in healthcare provision in specific regions. It is thus recommended that health system administrators intending to implement changes identify and measure relevant outcome factors prior to implementing changes in order to facilitate examination of their impact on factors of interest.
References
Footnotes
Contributors JB and OH conceived of the study. ED and MB were involved in piloting, adapting and conducting the survey. CF performed the analysis. CF drafted the initial manuscript and all authors contributed to drafting the final manuscript.
Funding This work was supported by the Health Research Board, Ireland (CARG/2012/28).
Competing interests None declared.
Ethics approval Ethical approval for the study was granted by the Clinical Research Ethics Committee of the Cork Teaching Hospitals.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.