CHRONICLE. Henrik Sjövall takes us along to Cochrane’s most recent conference in Seoul, South Korea. Cochrane is a non-profit grassroots organization that assures the quality of the global knowledge flow. In his column, he draws a comparison to the HTA Center, which is the Region Västra Götaland ‘s local version of Cochrane.
“And then Pilate asked: What is truth?”
and the echo responded – the prophet remained silent.
With the riddle’s answer behind sealed lips,
to the underworld the Nazarenes walked.
But thank heavens for professors,
for it is quite clear what truth is!
They are legions, for they are many
who granted the doubtful Roman a response.
However, I find it strange that a single truth
can shift in shape and hue so wondrously.
That which is true in Berlin and Jena
is merely a bad joke in Heidelberg.
It is as I heard Prince Hamlet say, baffling
Polonius with the cloud’s illusion:
“Methinks it is like a weasel
– to me it appears to be a camel!”
(Gustaf Fröding)
Yes, what is truth? I had reason to consider this question in conjunction with a Cochrane conference in Seoul, South Korea last week. Many of you have surely heard the name Cochrane, but few know what it stands for. The name comes from Archie Cochrane, a doctor and visionary who recognized the lack of quality-assured healthcare information early on. The grassroots project that expanded to become today’s global Cochrane organization is named in his honor. He is said to have been a rather interesting man and once described Ernest Hemingway, whom he met during the Spanish Civil War, as an “alcoholic bore!” (www.lakartidningen.se/OldWebArticlePdf/9/9259/LKT0816s1215_1217.pdf)
Archie Cochrane wrote a book called Effectiveness and Efficiency in which he questions the scientific cornerstones of contemporary medical care; the book had an enormous impact. In it, he argues for randomized treatment to draw conclusions about the effect, i.e., what we now call randomization. This eventually resulted in the formation of a small group, led by Iain Chalmers, that began to critically review the basis for medical methods, starting with obstetrics and perinatal care. The little troop grew organically and became what is now known as the Cochrane Collaboration, a global network for the quality assurance of knowledge. I think most of you have heard of Cochrane reviews, but few people know how the network is constructed and how it works.
In it, he argues for randomized treatment to draw conclusions about the effect
The organization is relatively simple: people who take information issues seriously join together, often but not always according to field, and form groups that conduct knowledge reviews. First a relevant question is identified, such as: “Is method A better than method B for condition C?” The questions can come from within or outside of the group. Once they decide to tackle the question (which should be both relevant and possible to analyze), it is reconfigured into what is called PICO, where P stands for population, I for intervention, C for comparison and O for outcome. Many questions have several P’s, I’s, C’s and O’s. After that, professional information librarians convert the question into a search string, making an ambitious attempt to find essentially everything written on the particular issue. Different databases are searched depending on the nature of the issue. As a rule, this usually generates thousands of hits. The next step is an arduous filtering process which, in the best case scenario, boils the hits down to a “net literature” of a few dozen articles. These are reviewed for quality using rigorous checklists that involve searching for common errors that may result in the misinterpretation of data, called risk of bias.
Once this has been done for every single article, a global assessment of the current state of knowledge is conducted for each outcome, and then the degree of certainty of the conclusion is assessed with the GRADE system (http://gradeworkinggroup.org/). Everything is summarized in a document which, after an external review, is posted online, and this is called a Cochrane report. Cochrane publishes several hundred reports per year (343 so far this year, http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/index.html) in a broad range of subject areas, and these are subsequently cited by the media to varying degrees.
Cochrane is a decidedly grassroots movement and does not have its own long-term funds. Most of the work is done for free under the auspices of other employment. Far from everyone is a professional academic; many people work in administration or health care; others are in social work; some are journalists or work in PR. Most are relatively young with primarily non-profit motives. And best of all, those creaky old professors who seem almost obligatory at this kind of meeting (“dear Bob, good to see you again!”) are most notable for their absence. Women clearly outnumber men and there are lots of students and postdocs—most people are simply here to learn something!
Last year was my first Cochrane Colloquium in Vienna. I was astounded by the first plenary lecture, which called into question the entire work method: “Look here, we get 4,000 abstract hits, exclude 3,997 and then heavily criticize the remaining three. Is that really reasonable? There has to be some information in the remaining 3,997! Look at Google: they try to do the same thing, and often come up with the same conclusion. Time to reconsider?”
To begin by questioning your own work method is powerful, and I started to realize that this was unique. Unfortunately, that thread wasn’t carried any further in Vienna; most of the program was about traditional methodology. So I immediately signed up for the next Cochrane Colloquium in Seoul, North Korea, in hopes of hearing more.
You must surely be wondering what I have to do in this industry overall; I’m a gastroenterologist after all. The explanation is that I’m doing a large portion of the clinical duties for my combined research and medical position at the HTA Center, a local unit that is similar to Cochrane, but with a more local focus and funding from the Västra Götaland Region (VGR). The picture shows the HTA Center’s team, from left: Annika Strandell (gynecologist), Jenny Kindblom (clinical pharmacist), Lennart Jivegård (vascular surgeon), Therese Svanberg (information librarian), Christina Bergh (Head and gynecologist), Inger Thorin (business assistant) and Petteri Sjögren (dentist). The picture was taken in the garden pavilion at Botan, where we had our annual end-of-season celebration.
HTA stands for health technology assessment. We do approximately one HTA report per month using Cochrane’s work method, but with the important difference that as a rule, our questions come directly from health care’s practitioners. The actual analysis is then carried out by a project group composed primarily of members of the medical professionals with a knowledge of the topic at hand, and we add two people from the HTA Center, one with and one without specific expertise. The latter is also project manager, to avoid subject matter conflict of interest. The issues in our reports vary broadly: a selection from the current project basket includes hand transplantation, the value of green rehabilitation, metabolic surgery to treat diabetes in normal-weight individuals, and selection of medication for treating wet macular degeneration (“yellow spots”). For us, attending the Cochrane Colloquium was a form of continuing education.
The fact that the meeting was in South Korea was particularly significant for me: my first real trip abroad in 1975 started with a flight from Paris to Seoul, via Anchorage, Alaska. I was in Korea again with my wife in 1995 when we picked up our adopted son Joakim, so naturally, I was curious about what had been going on in the country since my last trip.
My first impression was very positive. As we walked around the city, it was teeming with young people in traditional garb who were happy to be photographed. After a semi-successful attempt to get some food in our bellies using chopsticks, we walked around to try to get a picture of Seoul. We visited a temple where the trees were covered with colorful lanterns (some type of festival), and at the entrance to the temple area, instead of traditional military statues, there were models of colorful stuffed comic book characters. People seemed relaxed; they were out for a stroll, and it wasn’t at all the noisy and dirty urban setting I remembered from 1995.
Once again, the opening lecture was about something controversial
The meeting was the next day. Once again, the opening lecture was about something controversial—overdiagnosis: “Problems of overuse, its magnitude in health care and its mechanisms.” Several lecturers were deeply critical of the pathologization of large parts of the population on the basis of genetic diagnostics, and asked the important question of whether it couldn’t be considered part of Cochrane’s task to keep this from happening. “If Cochrane doesn’t do it, who will?”
I primarily sought out two types of seminars: those about important but hard-to-analyze issues with political elements, and those about knowledge transfer. So far, Cochrane has focused on subjects that can be studied with randomized controlled trials (RTCs), but many important issues are impossible to evaluate with this method.
The next step down in the evidence hierarchy (picture) is controlled observational studies, and many people felt, like me, that Cochrane needs to have the courage to enter that gray zone and leave the safe RCT world behind. Indeed, I sought out that gray zone specifically and attended seminars about complimentary medicine and public health, and in these groups, you could sense fear from some project group members about leaving the safe world of RCT. Old man Hegel said: “Remember that the owl, the bird of knowledge, flies in the dark.” We must have the courage to venture into the twilight if we are going to be able to shine a light on the important questions.
The next subject to which I paid special attention, knowledge transfer, was about packaging Cochrane report conclusions in a way that will entice the media. For example, we discussed how to handle the press, differences between different types of journalists, the importance of establishing lasting relationships with mutual trust, and of course, the media’s special dramaturgy. If Cochrane is going to take on larger tasks like the quality assurance of general knowledge, then that link in the chain must also be solid. The article you are reading is part of that work.
The formal program was over on Thursday, but our flight home wasn’t until Saturday morning, so we had Friday free. As luck would have it, a “test excursion” was arranged to a temple area that I had visited in 1975 called Bulgoksa. My HTA colleague Petteri Sjögren and I signed up immediately. It was a lengthy bus trip, nearly 800 km total, and the bus driver was close to falling asleep on the way home. The biggest takeaway was an absolutely wonderful guide: his English may have been awful, but he compensated completely by being enormously engaging. For example, he tried to explain the Buddhist worldview by drawing with a coin in the sand.
What have I learned from that? Cochrane is a global grassroots organization with unique conditions to contribute to the quality assurance of the global knowledge flow. VGR’s HTA Center will have the ability to help with this work. The question is: where does Sahlgrenska Academy stand? Unlike most major universities, we currently have no unit that is dedicated to working with medical knowledge transfer. Maybe it’s time for our faculty, in cooperation with VGR, to establish such a unit? If we don’t do that, I think we risk being outperformed.
Let’s give Fröding the last word:
However, I find it strange that a single truth
can shift in shape and hue so wondrously.
That which is true in Berlin and Jena
is merely a bad joke in Heidelberg.
That’s not how we want things to be, is it?
