Data collection is well under way now; it started in November last year, and it will continue until perhaps May or June this year. It’s pretty full-on!
There are 29 participants in the trial (we had 30 but one participant dropped out), all doing a baseline assessment and then a follow-up assessment, approximately three to four months apart. Each assessment lasts somewhere between one hour and two-and-a-half hours, depending on what sub-groups the participants have been allocated to. Everyone completes a questionnaire and does some measurements during the assessments, which takes slightly less than an hour; but if the participant is also an interviewee, they also spend about half an hour talking to me about their experience of the trial processes; and if they’ve been randomly allocated to wear a CardioSTAT® heart monitor, it takes about 20 minutes more to fit that in addition. Then they get the symptom diary – everyone completed one of these, giving details of their perceived symptoms over the seven days following each assessment – I have to talk everyone through it and show them what to jot down. And if there’s a lot of chat, or any technical difficulties, or the interviewee has a lot to say, it can take longer!
The original plan for the assessments was that we’d invite participants into the study centre (the Northern College of Acupuncture, in central York) and we’d carry out the assessments there; the participants would have been paid travel expenses. They’d be weighed, measured, and have their blood pressure taken; they’d complete the questionnaire; do the interview (if applicable); and have the CardioSTAT® heart monitor (if applicable) fitted by a trained professional. That’s a more usual way that trials conduct their assessments.
The best-laid plans…
But COVID put paid to that plan. To minimise the possibility of cross-infection, we had to take out as much face-to-face contact as possible. So we decided to move the study assessments online. Simple!
Except… it isn’t simple, of course. Switching to online assessments meant that the same set of equipment (for consistency of measurement) had to be delivered to each participant in advance of each assessment, then collected again afterwards. It had to be packaged to protect it (we ended up with a professional flight case with a tailor-made foam insert) and both the equipment and the case had to be cleaned, inside and out, between appointments, to minimise COVID risks. I researched and bought a set of weighing scales, a medical tape measure and a blood pressure monitor, all of which were medical grade (i.e., calibrated and reliable) but easy enough for non-medical professionals to use in their home environment. I gave up altogether on the idea of lending participants a stadiometer (a device used to measure height) because not only is it impossible to get an accurate reading unless somebody else is doing it for you (some trial participants were living alone), it’s also a pretty tricky thing to fit into a car without damaging either it… or the driver! So instead we settled for asking the participants their height, and where/when they’d last had that measured. Not ideal, but the best we could do.
We also had to think about how participants would do the assessments in their home environments. Because we were asking them to undress to take the measurements, they had to have a quiet, private space in their homes (or at least not mind undressing in front of their family). They also needed to have reliable broadband capable of supporting a video call. All those things had to go into the eligibility criteria – because if somebody didn’t have one of those things, they couldn’t take part.
I wrote all those changes into the protocol and sent it back to the ethics committee for approval. While the committee were pondering, I did a think-through of the assessment process and wrote myself a step-by-step script to make sure that all the right things happened in the right order. I built in emergency processes (what if their internet fails? what if they break the tape measure? what if they’re standing on carpet so the scales aren’t accurate?) to cope with the “known unknowns”. Then I tested the whole thing with my long-suffering partner, who obligingly went through all the motions so I could troubleshoot and adjust it.
iPad, check… Charger, check…
In the screening phonecall, some participants had told me that they didn’t have a device equipped with a camera and/or a microphone, so I temporarily donated my iPad to the study (the budget wouldn’t stretch to buying one), erased all my own stuff from it and bought a shockproof case, then spent a while testing it with an older friend who hadn’t used an iPad before; from that, I wrote a “quick set-up” guide with screenshots, showing participants how to access the meeting link and position the iPad so they could be seen and heard. I packed the iPad, along with a charger and a power extension lead (I didn’t want it running out of charge halfway through the interview!), in another professional flight case. And I offered help to participants who couldn’t connect the iPad to their home broadband themselves – a job I’ve often done standing on their doorstep to minimise COVID risk, sometimes in the rain or even, on one occasion, the snow! Fortunately that only needs doing once – the iPad remembers networks it’s connected to previously, which has been a blessing for the follow-up assessments.
All the participants live within an approximate 25-mile radius of the middle of York, with quite a few at the outer extremities of that radius. Because I’m the only researcher conducting the assessments, that means I drive the equipment out to the participant, drive back to the University to get online and do the study assessment, then afterwards drive out to collect the equipment, and back home again. Before I leave the participant, I check all the equipment is present and correct; when I get back I clean it all thoroughly with antibacterial and antivirus cleaner, then leave it in the fresh air to ventilate and dry. It’s quite a schlep – although I did sign up one participant who lives literally a hundred yards down the road from my house, so I’ve delivered to them on foot, which I enjoyed a lot!
In all seriousness, though, if we were doing this in a full-scale trial it would extend the trial’s timescale substantially (because in a trial centre I could have done three or possibly four assessments in a day, but doing it online I can mostly only do one, depending on the participant’s timings and their location). If we had to work online, the way to keep a trial on time would be to add more fieldworkers to the team – but that adds cost, especially when you tot it up for multiple trial centres across the UK.
The effect on the data
I’d have to say, though, that while this superquick change from in-person to online data collection was expedient from a COVID point of view, it’s really throwing up some problems.
For instance, quite a few participants are uncomfortable about undressing in front of a camera – entirely understandably. The right of patients to privacy and dignity is a well-enshrined principle of medical ethics and it’s really gone against the grain to insist on something that trespasses so badly on that. Early on, one participant said to me that they didn’t like undressing due to “body shame” and that it would be so much easier if I was face-to-face with them; as a result, I felt very unhappy about asking them to do something that brought up such difficult feelings. I’ve apologised a lot in the last few months, and I’m incredibly grateful to all the participants for continuing in the trial despite this.
But people could turn their camera off, couldn’t they? Well, unfortunately, I need to see them so I can make sure they’re taking measurements accurately. So the camera has to stay on. An example of this is when people are measuring their hips: a lot of people think the hips are higher than they are, and they put the tape measure around the iliac crest although it should be the ball of the hip socket. If I can’t see them positioning the tape measure I can’t be sure they’ve got it in the right place. Most people were obliging about staying in view while they positioned the tape measure, but a handful preferred to stand to one side, out of camera view – so I started putting up a picture of the hips’ proper location on the screen, to guide them (the one on the right, which is nice and clear).
The data can also be compromised by other issues that wouldn’t be happening if participants were coming to a study centre for their assessments. For instance, I need the participants to check that the tape measure isn’t twisted before they give me the reading, and some people can’t reach behind themselves to check by running a finger along it. And fitting the CardioSTAT® heart monitor accurately is tricky for some people, because their necks or shoulders or arms aren’t flexible enough to make sure they’re sticking the monitor onto the right place on their chest.
There have also been some issues with the qualitative data. For instance, some people have had their partners present in the room when they’re doing the interviews, and the partners can sometimes step in to give answers on the participant’s behalf, before the participant has chance to answer the question themselves. I’ve been careful to ask the participant if they agree with what their partner has said (it’s slightly worrying because I don’t have consent to use data provided by anyone except the participant), but it’s sometimes a bit questionable whether they actually agree or they’re just keeping the peace! And if the partner wasn’t present, maybe a different answer would have been given? Impossible to know. If they’d come to a study centre, we could have invited just the participant into the interview room while the partner (if they came) would have stayed in the waiting space; but I can hardly ask somebody to leave their own living-room.
We had to make these changes to the data collection to get ethical approval to carry out the trial during COVID. But on the whole, I think a strong recommendation should be made for a future trial NOT to use online assessments: it adds costs and time, and it doesn’t give us good quality data. Based on the data and the observations so far, this aspect of feasibility testing has revealed that it’s feasible to collect data in this way, but it’s not an optimal process and neither is the data it produces.