Service evaluation of a digital behavioural change programme
ABSTRACT
The purpose of this service evaluation was to analyse the efficacy of OurPath, a UK-based digital behavioural change programme, based on real-world data gathered in a commercial setting. In total, 98 potential participants with a mean body mass index of 31 kg/m2 (obese) paid money to enrol on OurPath, a digital behavioural change programme that combines a private online social network, daily structured educational content, health coaching, wireless scales and an activity tracker. Participants underwent a core 6-week intensive lifestyle change intervention and moved onto OurPath's Sustain programme, with follow-ups after 3 and 6 months. A total of 77 participants met the inclusion criteria for the core programme and achieved a significant weight loss from baseline of 5.3% in 6 weeks (p<0.01). At the date of data analysis, 69 participants were over 3 months from their start date. Of these 69 participants, 42 (61%) had submitted a weight reading after 3 months and had achieved a mean weight loss of 6.7% (p<0.01). Data available for 15 participants after 6 months (52% of participants who had passed the 6-month mark at the date of analysis) demonstrated a mean weight loss of 8.2% (p<0.01). Weight loss data were not available for all participants at the 3 and 6-month mark. The OurPath online behavioural change programme achieved clinically significant weight loss results in a real-world setting. Online delivery platforms like OurPath could therefore offer an effective and scalable solution to tackle the UK's obesity and diabetes epidemics.
Introduction
Type 2 diabetes (T2D) is a growing concern for western healthcare systems. In the UK, the NHS spends £9 billion a year on its treatment,1 close to 10% of the entire health budget. Over 5 million people in England are at high risk of developing T2D;2 80–90% of that risk is associated with obesity.3,4
Fortunately, there is strong evidence that lifestyle interventions focused on improving diet and increasing physical activity can help service users to lose weight and significantly reduce the risk of developing T2D. Landmark clinical trials such as the National Diabetes Prevention Program in the USA showed that an intensive lifestyle intervention outperformed both placebo and metformin, and reduced the development of T2D by 58% after 3 years,5 and 34% after 10 years.6 Furthermore, there is growing evidence to support the use of digital behavioural change and lifestyle improvement programmes, which have been shown to be as effective, and often more feasible, than in-person interventions, and increase physical activity and healthy eating habits.7–9
OurPath's 6-week digital lifestyle improvement programme provides people with structured education, healthcare tracking devices and a support network of mentors and peers to help them change their behaviours and adopt healthier lifestyles. OurPath is delivered to people remotely through an online website platform, as well as through smartphone apps.
This service evaluation was designed to validate OurPath by benchmarking real weight data against clinically significant weight loss standards, as well as evaluating the inclusion of OurPath's Sustain programme, which was absent in previously reported trials.
Methods
Research design
A quasi-experimental research design was used, which included longitudinal and pre–post tests of weight, activity levels and programme engagement outcomes.
Participants
Participants were recruited online through digital advertising on Facebook and Google, using diet and weight loss-related keywords. The participants paid an average of £120 to enrol in the 6-week programme, which included the smart weighing scales, activity tracker and 6 weeks of health coaching. The exact cost paid varied across the recruitment period from £99 to £150, as pricing sensitivity tests were also being conducted in this period. The ongoing Sustain programme was charged at £10 per month.
Participants were required to be 18 years or older, be based in the UK and to have a minimum body mass index (BMI) of 23 kg/m2. This BMI cut-off was chosen because the overall goal of the OurPath programme was to achieve a certain level of weight loss (defined by starting BMI) – to not encourage further weight loss for those in the lower-middle range of the ‘healthy’ BMI weight definition.
Eligible participants completed an online account setup process, in which they agreed to terms and conditions and privacy policies, and completed health and demographic questionnaires. Participants were then able to access the OurPath programme from web and mobile devices.
Intervention
The OurPath programme consisted of a core 6-week digital intervention that encouraged sustained behavioural change and lifestyle improvement, followed by a less intensive ‘Sustain’ programme to keep people on track with their goals.
The OurPath programme was originally designed to drive sustained behavioural change and lifestyle improvement among those at risk of developing T2D and other lifestyle diseases. The intervention was developed in several stages, following the steps recommended in The behaviour change wheel: a guide to designing intervention.10 It delivered several behavioural change techniques specifically recommended for diabetes prevention programmes by the Behavioural Insights Teams within Public Health England and the Department of Health.11
The OurPath programme included four major intervention components:
small-group support
health coaching
an education curriculum (aligned with National Institute for Health and Care Excellence guidelines)
digital heath-tracking hardware.
OurPath's primary diabetes prevention programme was 6 weeks long, and was delivered digitally through users’ smartphones and computers/laptops (Fig 1). The specific duration of the primary phase was designed to maximise engagement and sustainability, and is supported by numerous academic studies.5,12
In the week preceding the programme start date, users received information that enabled them to mentally prepare for the weeks ahead and to understand the key nutritional principles of the OurPath programme. Users also received a ‘Healthbox’ containing a Bluetooth-enabled wearable activity tracker, as well as a set of 3G-enabled smart weighing scales (Fig 2), which ensured an accurate and reliable weight reading before the programme started. People were able to start clearing their cupboards, get into the right frame of mind, and garner support from family and friends before they officially began.
From day 1, users were assigned a trained health coach (a registered dietitian), put into a support group of around 10 similar users and received daily educational content on nutrition, exercise, sleep and stress management. The educational content provided in the OurPath programme was informed by multiple UK guidelines, including:
NICE guideline PH38 – Type 2 diabetes: prevention in people at high risk
National Diabetes Prevention Program education content specification.
The daily education content was specifically designed to achieve the following objectives:
supporting and encouraging users to achieve the broader UK dietary recommendations as detailed in the Eatwell Guide
supporting and encouraging users to increase their daily activity levels to meet, or surpass, the England chief medical officer's recommendations of 150 minutes of moderate aerobic activity per week
supporting users to regularly monitor their weight and progress towards an achievable weight loss goal.
In order to encourage long-term behavioural change, the OurPath programme was designed to be stimulating and enjoyable. A user-centred design development and feedback process showed that individuals responded well to a daily or weekly task or competition, such as taking a photo of their meal for their support group or writing a blog post. The health coach and support network ensured that users felt comfortable to reach out and ask questions.
The OurPath programme also provided educational content on cognitive behavioural therapy, which lies at the heart of enabling long-term change. Negative patterns of thought, high levels of stress and insufficient sleep can be as harmful to weight loss as eating the wrong foods, hence why cognitive behavioural therapy was a core pillar of the education content within the programme.
The new generation of healthcare sensors (ie 3G-enabled smart weighing scales and Bluetooth-enabled wearable activity trackers) used as part of the OurPath programme allowed data to be continuously captured, providing motivation to users as well as equipping mentors with real-time information about the progress of their users. These data were also intelligently analysed to drive early warning intervention systems to keep people on track and further tailor the programme according to a user's progress. This allowed the OurPath programme to be adaptive and proactive in helping its users achieve their desired results.
After the 6-week intensive programme, users moved on to OurPath's less intensive ‘Sustain’ programme, where they continued using OurPath's mentorship network, tracking tools and content to help sustain the positive changes they had made. The Sustain programme was designed to maximise sustainability in the long term, providing regular support when a user needed it most.
The programme overhauled historically core components of T2D prevention programmes (such as the requirement for in-person attendance and calorie counting) and was designed to consider 21st century lifestyles and innovations. OurPath's technology and user-focused approach destigmatised T2D interventions and allowed individuals to make the necessary changes to their lifestyles in a way that worked for them.
The programme was designed and validated by a core development and clinical advisory team consisting of diabetes specialist clinicians, GPs, psychologists, behavioural science academics and registered dietitians.
Measures
Demographic and health information was collected at baseline. Programme engagement was assessed via lesson completion, which was tracked through the online interface. The primary outcome measure was body weight, which was tracked via the wireless scale participants received. Participants were encouraged to weigh themselves daily during the core programme and weight data was automatically recorded online (avoiding the need for participants to self-report).
Feedback on participant satisfaction was gathered through user-submitted ‘blog’ posts, detailing their experience of the OurPath programme. Examples of this feedback can be found on the OurPath website, but because of the qualitative nature of the text-based feedback, results are not being reported here.
Analyses
The Jamovi open-source statistics package was used to calculate statistical tests and p-values. One-way Student's T tests were used, with the null hypothesis being an average weight loss of 0% (ie no weight loss) and the test hypothesis that the population mean was greater than zero.
P-values reported in this publication also held true when a null hypothesis of 4% weight loss was used.
Results
Demographics and participation
A total of 98 participants signed up to the OurPath programme, completed the initial assessment and online setup process, and began the intervention on staggered weekly start dates from 4 July 2016 to 20 February 2017. The participants were demographically diverse, with an average age of 43 years, 70% women and 33% black and minority ethnic. The average BMI was 31 kg/m2 with a QDiabetes risk score of 9.2%.
In total, 77 participants (85%) met the inclusion criteria for completing the full 6 weeks of the core programme. Inclusion criteria were defined as having at least five weight measurements, reading over 25% of the education content and sending at least three messages to the mentor or group chat. The results included in the analysis are from these participants. A full demographic breakdown of participants can be found in Table 1.
Outcome measures
Weight loss at each time point is reported in Table 2. Participants had significant weight loss from baseline to 6 weeks. Participants achieved a statistically significant weight loss of 5.3% (p<0.01) at 6 weeks.
Of the 69 participants who had passed the 3-month mark at the date of analysis, 42 (61%) had submitted a weight reading after 3 months and had achieved a mean weight loss of 6.7% (p<0.01). Data available for 15 participants after 6 months (52% of participants who had passed the 6-month mark) demonstrated a mean weight loss of 8.2% (p<0.01). A flow chart of the users has been provided in Fig 3. Further data on the users who dropped off after 3 and 6 months (with no additional submitted weight readings) were not available.
Regarding engagement, over the 6 weeks each participant sent an average 35 (standard error = 6.5) messages to the group and mentor, and weighed in an average of 22 times (standard error = 1.4).
Discussion
The results of this service evaluation showed that those participants that remained engaged with the programme were able to achieve and sustain clinically significant weight loss in a real-world setting through the OurPath programme. These results met and exceeded the recommended 5–7% weight loss outcomes that are associated with a ≥50% reduction in risk of T2D.
More broadly, such results suggest that the digital delivery of evidence-based treatment can be effective. The average weight loss achieved at 6 months was greater than the average 12-month weight loss shown in the meta-analysis of 22 National Diabetes Prevention Program lifestyle intervention translations,9 indicating that digital interventions can be as effective, if not more so, than in-person interventions.
If that is indeed the case, online behavioural change programmes like OurPath have the potential to simultaneously relieve the strain and cost of in-person interventions on public health, and increase public access to those interventions, to positively impact the incidence of diseases like T2D.
The use of digital tools allows for the combination of the most successful aspects of in-person lifestyle interventions while eschewing their limitations. A programme like OurPath can combine the effectiveness of group support, dedicated health coaching, a structured learning syllabus and regular health tracking, without compromising on usability, accessibility and engagement.
Online groups provide instant, 24/7 access to mentor and peer support, whereas in-person groups require participants to meet at a regularly scheduled time and place, usually on a weekly basis. A number of factors can interfere with participants’ ability to attend – whether it is their availability, transportation or even convenience. By comparison, online groups only require internet access and can therefore integrate seamlessly into people's lives, providing immediate support around the clock.
Mentors can be employed full-time, trained and supervised by OurPath to deliver a suitable and consistent standard of care. They can support several different groups at once, with a single mentor supervising between 100 and 200 participants simultaneously. Digital lifestyle interventions are thus significantly more cost-effective and scalable, and can be rolled out to reach millions of people at risk with much less effort.
The structured educational content can be delivered in daily, bite-sized chunks, to be read at the user's convenience. This maintains user engagement and enables them to better retain the material than they otherwise might in a classroom setting. Service-side, it means greater intervention fidelity and standardisation, alongside the ability to provide tailoring where needed and recommendations through the mentors.
Digital interventions enable remote monitoring of adherence and outcomes through digital health hardware such as wireless scales and activity trackers. Both participants and mentors are therefore able to track their progress in real time. Participants can be motivated and encouraged by observing their progress towards their goals, while mentors can personalise their interactions according to how the participant is doing and intervene if adherence drops off, for instance. Furthermore, the use of smart weighing scales means weight readings were automatically sent to a user's account and stored, removing both the need for users to self-report and the possible temptation to manipulate their data. As a result, the validity of the weight data was greatly increased.
There are a number of limitations to the current service evaluation that should be taken into account when evaluating the broader conclusions. Firstly, given users had to pay to enrol on the programme and were reached using digital advertising, the evaluated population will not be a completely representative sample of British adults at risk of obesity and T2D. In addition, those willing to pay over £100 for an online lifestyle improvement programme are likely to be more motivated to make a change.
The second major limitation is that the weight data was incomplete for those participants who had reached 3 and 6 months after their start date at the time of the analysis. This meant that the analysis of weight loss achieved after 3 months was confined to 42 participants and after 6 months to 15 participants – representing 55% and 19%, respectively, of the original cohort of 77 who met the inclusion criteria for the evaluation.
Nevertheless, this service evaluation indicates that the OurPath digital behavioural change programme has the potential to effectively and sustainably reduce obesity and diabetes risk factors in British adults. Digital programmes like OurPath should be considered in future national initiatives to address the obesity and diabetes epidemics, offering effective and highly scalable solutions to deliver evidence-based treatments to people at risk.
Conflicts of interest
CE is a director of OurPath Ltd. JH is a minority shareholder of OurPath Ltd.
Author contributions
Study conception and design: CE, JH and EA
Acquisition of data: CE
Analysis and interpretation of data: CE and JH
Drafting of manuscript: CE
Critical revision: JH and EA
- © Royal College of Physicians 2017. All rights reserved.
References
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- ↵
- Edwardson CL
- ↵
- ↵
- ↵
- Michie S
- ↵
- Public Health England, NHS England Behaviour Insight Team
- ↵
Article Tools
Citation Manager Formats
Jump to section
Related Articles
- No related articles found.