Resilient GP: bad research, and an inappropriate demand on GPs’ time

Resilient GP’s survey report Inappropriate demands to GPs is a poor quality piece of research – bad enough that it can’t tell readers anything much. I’ve already blogged about the report’s ethical problems In this post, I’ll raise concerns about the sampling used in this project, the survey itself, the reporting of this work, and the report’s analysis (or lack thereof). I’ll argue that getting GPs – busy as they are – to participate in low quality research like this is an inappropriate use of their time

Sampling

Resilient GP say that they

conducted a survey on a large, private online discussion group composed entirely of GPs…We received over 200 unique responses.

Unfortunately, the survey report doesn’t specify much the characteristics of the group or those who responded – for example, if all the respondents were in England their answers may not reflect the situation in Scotland. The report also doesn’t specify the response rate: if these 200 responses came from a group of 200 GPs then they clearly reflect the group well; if they came from a group of 50,000 then the minority who chose to respond may be very different from the group as a whole (for example, may be responding because they’re annoyed by particular demands). Without this information, it’s impossible to know to what extent issues with the sample may bias the report’s findings or limit how much one can generalise from it.

The survey

Resilient GP’s report says almost nothing about the survey they used. They state that

We asked for examples that were considered by that GP to be an inappropriate use of their time and skills.

However we don’t know, for example, whether participants received a large number of questions to elicit these responses or how the questions were phrased. Without this information, the readers of this report can’t really interpret the survey results.

Reporting

The way the survey report presents the survey results makes it near-enough impossible to draw any robust conclusions from this work. The report presents a list of what were viewed as inappropriate demands. It states that

We excluded very similar responses or those we considered might have conceivably have been a presentation of underlying illness.

However, there is nothing beyond this (for example, no way of knowing how Resilient GP verified judgements as to what “might have conceivably have been a presentation of underlying illness” or how they decided what was/was not similar enough to exclude).

There is no way of knowing how recent or regular these demands are: for example, for a GP to see one patient who wants a fake sick note in a 40 year career then this would be unfortunate but not exactly shocking; if this happens every day that would be a very noteworthy finding. If there were regular demands for pet medicines 20 years ago but never today this would mean something very different from regular demands for pet medicines today.

It is also not clear whether Resilient GP have reviewed the available evidence on demands on healthcare professionals and healthcare systems. If they have, this isn’t apparent in this survey report.

Analysis

Bluntly, there isn’t much. The survey report reads like a long list of inappropriate demands reported by GPs, loosely divided into five categories. It is not clear how or why Resilient GP chose all of these five categories, although the survey report states that this was done

For ease of reference, and to help stimulate ideas for alternative solutions

There might be interesting information that can be drawn from the survey data Resilient GP collected. If there is, though, this analysis fails to do so.

Conclusions

One can conclude very little from Resilient GP’s survey report. Some GPs (about whom we know very little) were asked something (we don’t know what). They responded with a number of reports (we don’t know how many) about what they remember as inappropriate patient demands. We don’t know when these demands were made or how frequently they arise. This survey report therefore doesn’t tell us much at all.

Finally, while I’ve already blogged about ethical problems with this survey report, I’d like note one more ethical issue. Doing low quality research is often seen as unethical – among other problems, it wastes the time of participants and may make things harder for future researchers. Asking GPs – busy as they are – to participate in low quality research like this is an inappropriate use of their time.

 

* To be fair, the survey report does state that one demand – a “letter stating patient is unable to attend their tribunal or ATOS assessment” – is “a very common request”

Advertisements

Tags: ,

5 responses to “Resilient GP: bad research, and an inappropriate demand on GPs’ time”

  1. Kojacked says :

    Resilient GP have shown such poor judgement in collating and releasing this “survey” in the first place, why on earth would anyone trust their clinical assessment over whether these patients had an “underlying illness”?

    How do we know that these patients aren’t being fobbed off as “timewasters” and “abusers” of the NHS when in fact they are really unwell.

    Also I’m not sure if you’re aware but the group and its members that produced this survey are totally visible on social media. That’s why it’s so unethical, any patient could identify that their GP is part of the group that produced the survey and therefore identify themselves.

    I’m not sure why this massive abuse of patient confidentiality isn’t a disciplinary issue?

    • jonmendel says :

      Thanks for the helpful comment, but I’m not on Facebook and therefore can’t access that group.

      • abetternhs says :

        Have you seen the comments here by Dr Jaswit Atwal
        http://www.resilientgp.org/inappropriate-demands/#comments

        and here by me (and others)

        http://www.resilientgp.org/see-no-evil-speak-no-evil/#comment-91

        Petra Boyton’s Facebook post:

        There’s nothing like a really bad piece of ‘research’ to help you a. feel better about your own work and b. learn more about how to do research effectively and ethically.
        Here is a disturbing survey run by a website for UK GPs (Family Practitioners). Members of the site were asked to describe consultations with patients where the doctor believed the issue the patient was asking about was ‘inappropriate’. You can see their reporting on their findings here: http://www.resilientgp.org/inappropriate-demands/
        You can use this to flex your research muscles. Firstly, have they done a survey? And how have they gone about organising their data? Are there any ethical issues you can spot in the work they have undertaken? Could you re-analyse their list of inappropriate demands to see whether particular kinds of patients are being represented – and if so, how are they being described? What kinds of additional lenses might you bring to this kind of analysis – perhaps you could think about it in terms of class, poverty, health literacy?
        If you don’t have time to do some or all of these reflections. You might want to read the excellent critique of the work by Jon Mendel https://jonmendel.wordpress.com/…/resilient-gp-an-ethically…
        Having received a fair amount of criticism on Twitter the website defended the work undertaken. Do you think their response attends to some of the issues outlined above? http://www.resilientgp.org/see-no-evil-speak-no-evil/
        Resilent GP (the group who originally did this work) are now suggesting they haven’t done research, a study or a survey. But instead are starting an important discussion on practitioner stress/workload/appropriate use of services. What’s your view on this? If a conversation needs to be had within health/social care, development, education or so on do we begin this by listening to diverse experiences? Have conversations that stem from reviewing the available evidence first? Or run a not very good survey and use that to start a discussion? Is it okay to have practice-based discussions that are underpinned by really bad research – even if that research supports our preconceived ideas and generates a lot of interest?
        What are the possible harms that might come from doing work in this way? Is this a new issue for practitioners or one that has always been a problem? If practitioners are overworked and stressed how might work support them (in research, advocacy and activism?). What happens if patients are alienated from practitioners by research of any standard?
        Feel free to chat about this below if it interests you. Or adapt for your own teaching (where you might bring in additional evidence about workload, patient need, survey design etc). Alternatively you might want to comment directly on their blog or on Jon Mendel’s one about the wider ethics/research issues raised by the ‘inappropriate demands’ poll.

        And my responses:

        As a (very new, part-time) academic GP, with an interest in resilience e.g. https://abetternhs.wordpress.com/2014/06/12/lessons/) and a paper coming out in Medical Education next month, I’m aware that many GPs have a pretty dim view of academic General Practice as being ‘an easy option’ and out of touch with the ‘real world’. Certainly there’s little going on in academia that is concerned with the well-being of GPs on which patient safety and compassionate, high quality care depends. It’s the subject, not the rigour that attracted attention.

        Furthermore, most GPs don’t have time to read ‘research’, instead they/we read ‘guidelines’ or research summaries. Most decent qualitative papers that deal with the complexity of general practice are hidden behind paywalls in journals most GPs have never come-across. I try on my blog to highlight these, but it’s time consuming and won’t advance my academic career (assuming I have one)

        I think the academic GP community should reflect on this and come up with some useful research that helps to highlight the complexity and pressures of our work, that can be used to make things better, rather than just coming up with more guidelines and more things to do

      • jonmendel says :

        Thanks – that’s very helpful. I had seen some of the comments on the Resilient GP site (just got up-to-date on train last night) but hadn’t seen the Facebook discussions.

        Are Resilient GP now suggesting this wasn’t research or a survey? I’ve had people (not Resilient GP themselves) tweeting me suggesting similar, but that’s not how I’d read what they’ve posted.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: