Category Archives: Critical Appraisal

Critical Appraisal of a Paper

Videolaryngoscopy Vs Direct Laryngoscopy

2nd November 2012: Videolaryngoscopy Versus Direct Laryngoscopy in Simulated Paediatric Intubation

Where can I find this paper?

What is this paper about (what is the research question)?

In the setting of simulated paediatric intubation, is use of a videolaryngoscope associated with higher “success” rates than traditional direct laryngoscopy?

Summary of the Paper

Design: Prospective case-control study (subjects acting as own controls)

Objective: To compare the effect of videolaryngoscopy performed by trained Emergency Physicians on outcomes related to intubation.

Control: Direct laryngoscopy performed using videolaryngoscope with video display off (equivalent to laryngoscopy with Miller 0, Miller 1 or Mac 3 blade for neonate, infant and adult respectively)

Exposure: Videolaryngoscopy performed using Storz videolaryngoscope, with participants directed to watch the screen rather than looking into the oropharynx.

Intervention: Attempted intubation of three mannequins (neonate, infant and adult) in randomised order with or without presence of “pharyngeal swelling ” (determined by coin toss).

Primary outcome: “Successful” intubation, defined as successful placement of an endotracheal tube in the trachea of the simulation mannequin before removal of the laryngoscopy blade. The outcome was determined by unblinded post-attempt review of 3-point video footage by study investigators.

Population: Fellows and faculty in Paediatric Emergency Medicine volunteering to participate at a single tertiary care paediatric hospital (Philadelphia, USA).

  • Inclusion: Physicians with “adequate” training and experience with intubation
  • Exclusion: Fellows in their first year who had not completed requisite airway management training in the operating room

Results: 26 subjects performed 156 intubations Data was available for 148 intubations.

  • First attempt success in newborn: DL 20/24 (83%), VL 22/24 (92%). Difference 9% (95% CI: -9 to 28%)
  • First attempt success in infant: DL 21/24 (88%), VL 17/24 (71%). Difference -17% (95% CI: -38% to 5%)
  • First attempt success in adult: DL 10/26 (39%), VL 21/26 (81%). Difference 43% (95% CI: 18 to 67%, p=0.002)

First attempt success was significantly worse with VL than DL when pharyngeal swelling was turned on.

Authors’ Conclusions:

Paediatric emergency medicine providers had improved first-attempt success at intubation using videolaryngoscopy with a curved blade in an adult simulator; success rates with straight blades in neonatal and infant simulators were not significantly different between direct laryngoscopy and videolaryngoscopy.

On the study design

This was a small scale prospective study generating paired data. The subjects acted as their own “controls”, comparing normal DL technique with VL. There is considerable potential for selection bias in this study; subjects volunteered to participate. We can all imagine those of our colleagues most likely to volunteer to participate in this sort of study (and those who wouldn’t), and how this might skew or influence outcomes.

The scenario order was randomised, which increases the stringency of the methodology, but the lack of blinding of the investigators assessing the primary outcome is unfortunate as this might introduce an observation bias.

The introduction of the “pharyngeal swelling” component is an interesting one; the rationale for randomly including this complicating factor is unclear. While this might better represent “real life” clinical situations it does introduce a potential confounding factor for which the authors do not completely account. This might have been better placed in a separate study.

What were the results and what does this mean?

We can see from table 1 that there was a difference in time since last adult intubation between the “fellows” and the “attending physicians” which seems more significant than the for the other age groups, although there is no p-value given for this.

The investigators looked at comparative proportions of success between groups. The confidence intervals were wide but crossed zero in both the neonatal and infant mannequins, implying no evidence of significant difference in success rates. A bigger study would be expected to tighten these confidence intervals.

For the adult mannequin, the 95% confidence interval was similarly wide but did not cross zero. This suggests that the probability of observing a demonstrable difference in success rate between the groups if in fact there is no difference (the null hypothesis is true) is 0.2% (or 1 in 500). However the width of the confidence interval suggests that the “true” magnitude of the difference between the two observed groups could be between 18% and 67%.

What can we take from this paper into clinical practice?

Not much yet! The paper suggests that the use of a videolaryngoscope in the simulation setting does not significantly reduce intubation success in neonate and infant mannequins, and may improve performance in adult mannequins when used by skilled PED clinicians.

However, the controlled environment of the simulation room and the anatomy of the mannequin may well impact the realism of the situation, and the study is a proxy for clinical intubation success at best. More research is needed as the true value of videolaryngoscopy is still unclear.

More questions to ask

  • How does the use of videolaryngoscopy translate into clinical ED practice, particularly in a pressured environment?
  • Are these results reflected in physicians with less intubation experience?
  • Would training/experience with the videolaryngoscope improve performance further?

Follow us on twitter: @PEMLit

26th October 2012: The Effect of Abdominal Pain Duration on the Accuracy of Diagnostic Imaging for Paediatric Appendicitis

Where can I find this paper?

What is this paper about (what is the research question)?

How do the test characteristics of US and CT change according to the duration of abdominal pain in the diagnosis of appendicitis in children?

Summary of the Paper

Design: secondary analysis of multicentre observational study

Outcome: presence or absence of appendicitis as determined by pathologist’s report of findings at surgery or composite telephone/medical record follow-up where surgery was not undertaken

Primary objective: to determine test performance characteristics of CT and ultrasonography according to the duration of abdominal pain in children being assessed for appendicitis

Population: ED patients aged 3-18 years presenting with acute abdominal pain of <96hrs duration

  • Inclusion: “possible appendicitis”, defined as patients who had blood tests, radiological studies (CT or USS or both), or surgical consultation for the purpose of diagnosing appendicitis
  • Exclusion: pregnancy, previous abdominal surgery, chronic GI conditions, severe developmental delay, CT or USS prior to ED assessment, pain >72h, no radiological examination performed.

Results: 2,349 patients in parent study, 1,810 in subgroup (1,216 had CT, 832 had USS).

38% had appendicitis (n=680).

With equivocal cases (radiology) removed:

  • OR of trend in CT: sensitivity 0.98, specificity 0.91, PPV 1.02, NPV 0.88
  • OR of trend in USS: sensitivity 1.40, specificity 1.15, PPV 1.26, NPV 1.26

With equivocal cases (radiology) included as positive:

  • OR of trend in CT: sensitivity 0.96, specificity 1.07, PPV 1.19, NPV 0.88
  • OR of trend in USS: sensitivity 1.39, specificity 1.10, PPV 1.19, NPV 1.26

Authors’ Conclusions:

The sensitivity and negative predictive value of ultrasonography increase with the duration of pain, and CT is less likely to be indeterminate with a longer duration of pain.

On the study design

This is a secondary analysis; this means the data used in the study was originally collected as part of another study, and is being analysed in new ways to answer different clinical questions. This approach is not uncommon; large studies generate a lot of data and it may be possible to identify related clinical patterns by subgroup and secondary analysis. Just remember that this is not the purpose for which these patients were recruited.

The ascertainment of duration of symptoms was completed on a standardised form before knowledge of CT or US results. This is a subjective outcome, so standardised forms help to increase objectivity. The kappa score (for inter-rater reliability) was 0.73 (95% CI 0.67-0.78) – not brilliant, particularly if we consider that the “true” K value could be as low as 67% agreement.

Abstraction rules were generated to help code US and CT findings to “normal”, “positive” or “equivocal”. The same is not true of the outcome measures: there is no mention of if and how uncertainty in the reports of the pathologist or surgeon was managed. Ambiguity in the reference standard would impact on the validity of the study. It is also unclear whether the pathologist and surgeon were blinded to the pre-operative radiological findings; this might also introduce an observation bias.

What were the results and what does this mean?

The authors give us various sensitivity, specificity, positive and negative predictive values for both CT and US at different time points, with odds ratios to express the relationship between increasing duration of pain and each test characteristic.

Table 2 shows us that as duration of symptom (pain) increases, whether equivocal cases are excluded or presumed positive, the sensitivity, specificity and negative predictive value of US increases. For CT, there is an improvement in positive predictive value when equivocal cases are included. But look at the confidence intervals. We can only be 95% sure that the true odds ratio is greater than 1 (i.e. the test statistic value increases) for sensitivity and NPV in US.

What can we take from this paper into clinical practice?

Serial US might be useful in equivocal cases, where clinical signs do not immediately necessitate surgery. The longer the history of symptoms (but <72h), the better a a negative ultrasound is at ruling appendicitis out. However, CT seems to offer more acceptable test characteristics regardless of duration of symptoms.

More questions to ask

  • What are the test characteristics when prospectively ascertained for clinically equivocal cases (since these are the patients we would not immediately take to theatre)?
  • How do these US findings compare to ED US?

Follow us on twitter: @PEMLit

19th October 2012: Can We Reduce Ketamine-Associated Vomiting in Young Children?

Where can I find this paper?

What is this paper about (what is the research question)?

Ketamine, commonly used for paediatric procedural sedation in the ED, is associated with vomiting. Can we reduce the incidence of vomiting among this population by using adjunctive atropine or metoclopramide?

Summary of the Paper

Design: prospective, randomised, open, controlled study of children receiving ketamine sedation in the ED.

Outcome: incidence of ketamine-associated vomiting (KAV), either in the ED or within the first 24h after discharge

Primary objective: to compare incidences of KAV between patients receiving atropine and metoclopramide as an adjunct to ketamine (as prophylaxis against KAV), or ketamine alone.

Interventions: patients undergoing procedural sedation in the ED were randomised to receive IM ketamine (4mg/kg) for sedation either alone or in addition to atropine (0.01mg/kg) or metoclopramide (0.4mg/kg), administered IM in the same syringe as ketamine.

Population: tertiary hospital (Korea), with emergency medical centre and regional paediatric referral centre.

  • Inclusion: patients 4 months to 5 years of age, ASA class I or II receiving ketamine for laceration repair
  • Exclusion: concurrent illness involving vomiting; previous reaction to ketamine, atropine or metoclopramide; non-consent; contra-indications to trial drugs; inadequate sedation after first dose of IM ketamine.

 Results: convenience sample of patients presenting between October 2010 and September 2011. 1883 patients met inclusion criteria, 368 enrolled and randomised, 25 subsequently excluded due to sedation failure, 343 analysed in dept, 338 analysed by phone follow-up.

Vomiting occured either in the ED or after discharge in:

  • 28.4% of children receiving ketamine alone
  • 27.9% of the group receiving atropine + ketamine
  • 31.2% of the group receiving metoclopramide + ketamine

p value = 0.86 (no significant difference)

Authors’ Conclusions:

“We were unable to reduce ketamine-associated vomiting using adjunctive atropine or metoclopramide.”

On the study design

This is a pragmatic paper: there is little deviation from “normal” sedation practice, no fancy bits of equipment were used, and it is easy to see how (if you really wanted to) you could set up a similar study in your own department.

The use of telephone follow-up strengthens the conclusions the authors are able to draw; a not insignificant proportion of patients experiencing vomiting did so after discharge; they have achieved 97.9-99% telephone follow-up within each subgroup, which is helpful to us. Did they use intention-to-treat, and include those lost to follow-up in a worst case scenario (that they had vomited)? No, but they missing data is a relatively small proportion and equal among subgroups, so it is unlikely that this would have vastly altered their findings.

There is little ambiguity in their outcome measure; it is easy to imagine that a a patient either vomited or they didn’t. What would be interesting would be to know how patients who retched but didn’t actually vomit were classified.

A problem I have with this article is their patient cohort; I can accept their rationale for using IM ketamine in children under 5 years (although I don’t necessarily agree with it), but sedation at 4 months of age feels a little risky to me (and raises child protection concerns; how do 4-month-olds sustain -lacerations necessitating sedation for repair?). I wonder whether this truly represents practice across the world.

They also seemed nonplussed about starvation times for sedation. While this has been a contentious issue in the past (and a subject in need of a blog post elsewhere), I wonder whether this might affect their rates of vomiting with <25% starved for 6h (solids) in each subgroup.

What were the results and what does this mean?

No significant difference in vomiting between subgroups; the study has been powered to detect a difference between groups with 101 patients in each arm. You might note that this was not achieved in the metoclopramide arm (95 patients analysed in dept, 93 of these later analysed by telephone).

Rates of vomiting in dept or at telephone follow-up were similar between groups with around half as many patients vomiting at home as in the department.

Vomiting also seemed to be more likely in patients who had not been “adequately” starved, although this data has not been statistically analysed (presumably because the paper was not powered for this outcome).

What can we take from this paper into clinical practice?

There is no apparent reduction in post-procedure vomiting by adding metoclopramide or atropine to your IM ketamine sedation.

More questions to ask

  • Is this also true of ketamine sedation when drugs are administered by the IV route?
  • Would the same pattern be observed if patients were “adequately” starved?

Feedback Please!

As this is the first PEMLit Critical Appraisal post, it would be really helpful to get your feedback. Please feel free to comment below. We are particularly interested in your thoughts on the type of paper chosen and the format of the appraisal. Thanks for reading!

Follow us on twitter: @PEMLit