I am ecstatic about the Miami Dolphins picking up Pat White. I haven't been this excited about a pick since .... probably ever.
One thing I enjoy about sports is trying to determine where there are market inefficiencies. By that, I mean where the general consensus undervalues something. Examples of market inefficiencies:
1. Living in Pittsburgh. Pittsburgh has many of the attributes that make for a great quality of life, but is still relatively affordable.
2. Wines from South Africa. Considering the quality of the wines, you can get great values in wines from South Africa compared to other regions.
3. Great defensive players in the NBA. Offensive statistics are well defined in basketball, so it is easy for teams to probably determine the market values for player's offensive abilities. However, teams still struggle to quantify a player's defensive abilities, and therefore an NBA team can pick up a great defensive player (especially if their defensive skills are in something other than blocks or steals, which are readily measured) relatively cheap.
4. Short MLB pitchers. Major league baseball teams over-value height when rating pitchers. Therefore, you can pick up short pitchers at a relatively low cost.
For the entirety of my football-watching lifetime, there is one type of player that has been disproportionately successful at the college level, but not used well in the NFL- the option quarterback. Nearly every year, there are several star college quarterbacks who are able to tear apart college defenses because they can both run and throw effectively. However, because they don't fit the NFL prototype of the dropback QB, they either fail when asked to play NFL-style QB (which is essentially a different position), or are converted to another position (for some reason, safety is a common choice, but also WR or running back).
For the most part, these players are then perceived as failures. Eric Crouch, a Heisman trophy winner from Nebraska, was converted to safety and never did much. This is seen on some level as proof that option QBs can't succeed on the NFL level. The one guy who has been successful, Donovan McNabb, is not really the same player he was in college. In my opinion, Donovan McNabb was a far better college player. The NFL has literally cut off McNabb's legs- he was (until Pat White broke his record this year) the most prolific running QB ever. The fact that McNabb has been a decent (but, in my opinion, not great) thrower obfuscates what a great player McNabb could have been if he had been allowed to play his game.
So, what we've had over the past 20-30 years is a bunch of incredibly successful college QBs who are readily available in the late rounds of the NFL draft- guys like Brad Smith from Missouri, Major Harris from West Virginia, Joe Hamilton from Georgia Tech, Michael Bishop from Kansas State, etc. Every year, there is this one single group of terrific college players who are completely dismissed by the NFL, and because they can't change positions, are essentially told they can't play football.
It occured to me about 20 years ago that if an NFL team committed to playing the option, they could pick up these players on the cheap and create a series of schemes that NFL defenses would not be ready for. The players come so cheap (many aren't even drafted) that could stockpile 5 of them on your roster. If one gets hurt, just put another one in. You could even play 4 or more of them at the same time- one at QB, 2 at running back, and 1 or more at wide receiver.
For years, the NFL dismissed this as crazy talk. The NFL would make claims that you couldn't do something like this- the NFL defensive players are too fast and too tall, and the QBs would get hurt too frequently.
However, one trend changed the perspectives of the NFL- the rise of the SEC.
A few years ago, Arkansas was in an unusual position- they weren't a particularly good team and were particularly weak at QB, but they had arguably the 2 best running backs in college football at the same time- Darren McFadden and Felix Jones. So they decided to play both of them at the same time in a variation of the single wing offense, the Wild Hog offense. It was incredibly effective, and made McFadden into an even bigger star.
The next progression was the emergence of Tim Tebow at Florida. He was essentially a fullback during his freshman year when they won the national championship. He then became a full time QB, and the best in the country, winning the Heisman as a sophomore. But he still didn't fit the NFL prototype. But then, as a junior, he played the prototype NFL QB, Heisman trophy winner Sam Bradford from Oklahoma, in the national championship game. Bradford is a great player and will likely be the #1 pick in the draft next year and an All-Pro in the NFL, but even as a Bradford fan, I couldn't help but notice that Tebow was a better player.
NFL teams noticed too. Even 3 years ago, NFL teams would have dismissed Tebow as a pro QB prospect, but the confluence of trends has made teams realize that anyone as dominant as Tebow has been as a college player can probably dominate in the pros, if any team was ever willing to commit to letting Tebow be Tebow.
The third event (also involving an SEC player) was that the Miami Dolphins brought the Wildcat to the NFL. Miami was coming off an awful 1-15 season and lost their first 2 games, and early in the year was playing the mighty New England Patriots (who had went unbeaten the previous year) in the 3rd week of the season, at New England.
Miami killed them. They won 38-13, in what I think was one of the most shocking regular season games of my lifetime. What was more startling than just the score was how Miami won- they installed their running back Ronnie Brown in the Wildcat formation, and he had an absolutely monster game.
It's not an accident that this happened in Miami. Their QB coach, David Lee, had been the offensive coordinator and QB coach for Arkansas during the Darren McFadden years. He realized that the using a mobile QB in a varied offensive set could throw off NFL defenses designed to attack the classic dropback passer.
And now they have drafted Pat White, a QB perfect for the Wildcat formation. As a Miami Dolphins fan, I couldn't be more ecstatic.
Sunday, April 26, 2009
Sunday, April 19, 2009
boggle, fossil digs, and patient care
I recently started playing a "Boggle" knock off on my IPhone. It's a frustratingly addictive game. When you first look at the 16 different letters on the 4x4 board, I usually only see a few words. Then, at some point I start getting a feel for the board, and start seeing a bunch of words that are within a letter of each other (sit, site, tie, ties, ....). Before I know it, 2 minutes are up, and I've still only seen perhaps 35 out of 130 possible words.
Playing it on a computer makes me realize how poor I am. When you play with friends, if I get 35 words, I feel like you've accomplished something, because I didn't realize that there were 130 words available for play.
It reminds me about what has been said about Michelangelo (or it may have been Leanardo da Vinci)- that when he looked at a block of marble, he could see the statue he was going to make, and all he had to do was chip away the marble that wasn't part of the statue. The point is that patterns are out there, and the limitation is in the human mind's ability to process what is right in front of them.
I most directly confronted this limitation in human observation 12 years ago, when I was in graduate school on a fossil dig. It was much less fun than it sounds, in no small part because I may be the world's worst fossil digger. We were looking for very small mammal fossils, with each bone being smaller than my thumbnail.
I remember one hot day, when I staring at a patch of dirt for over an hour. The patch wasn't particularly big- perhaps 3 feet by 3 feet, and I didn't see anything besides dirt and pebbles. Eventually, the expidition leader came by, and within 1 minute of looking at the same dirt I has spent an hour looking at, she was able to pull out three fossils. Through her more experienced eye, she was able to see things that my mind wasn't able to see. It was a striking (and humbling) example of how it takes practice to see patterns amongst the chaos.
This phenomenon comes up frequently as a physician. When I was first learning how to read MRI images, at first the images looked like a lot of noise to me. As I gained more iterations through practice, I started to filter out the noise, so at this point when I look at an MRI, I see lots of signal and very little noise.
This pattern repeated itself as I learned different techniques, whether it be performing nerve studies (electromyography and nerve conduction studies), spine injections, or using musculoskeletal ultrasound. It just takes practice.
It's a very similar way of developing the skills of playing Boggle. When you play Boggle, you eventually learn that it the word SITE is on the board, that you may also be able to spell TIE, and if you can spell TIE, you can probably also spell TIES, and SET, and SIT, etc. You learn which words commonly go together. So instead of seeing one word, you see 6 words.
Similarly, when you are looking at fossils, you learn that fossils will have rougher contours than pebbles, so you start to filter out the smooth contours.
Same thing with reading MRIs. You learn what a normal spine should look like on an MRI, and start filtering through if there is anything that looks abnormal.
One of my favorite cliches is that you need to listen for the dog that doesn't bark. I've actually never read any Sherlock Holmes novels, but my understanding is that he cracked a case because during a robbery, no one reported hearing the dog bark when someone broke into the house. This absence of a noise that should have been there indicated that the dog probably new the thief. It's a great example of knowing what should be present, and therefore being suspicious when the bark is absent.
I bring all of these tangential topics- Boggle, Fossils, Barking, etc- up in the context of patient care. I had a very interesting day of clinic on Friday. My patients had the normal mix of presenting symptoms- low back pain, knee pain, shoulder pain- but in fact my patients had an unusual mix of underlying problems that were far from straightforward.
I had a very good resident with me. After he had seen the patients, I asked him what I always ask the residents- "what do you think is going on, and what do you want to do?" We had one straightforward patient, but for all of the others, he wasn't sure what to do.
A common medical cliche is that we can all see the common diagnoses, but while we unusual diagnoses may see us, we may not see them. For example, I see hundreds of patients with herniated disks every year, but some of the patients who seem like they may have herniated disks may actually have cancer. One of the reasons patients see a fellowship trained specialist is for our ability to sniff out those odd cases.
It was interesting to watch the expression on my resident, as I could tell that he knew something was different with these patients, but he had not yet developed the experience to work through the differential diagnosis and determine what was actually going on. After I worked through the examination and discussed things with the patient, it became obvious. I am guessing, thought, that the experience was similar to playing Boggle when a competitor sees a word that was right there under our noses, or my experience when my expidition pulled out a fossil right from where I had already been looking.
And I guess that is what experience is all about- developing enough iterations to become a better observer.
Tuesday, April 14, 2009
Heterogeneity
One of the most common questions I get from patients is "will this procedure work?"
My answer is always some variation of "I don't know for sure, but ..."
Some procedures work very well, examples including:
1. Transforaminal epidural steroid injections for radicular pain ("sciatica") associated with a herniated disk.
2. Carpal tunnel release for carpal tunnel syndrome
3. Knee replacements for knee osteoarthritis
For many other procedures, however, there is far greater ambiguity. One of the main reasons for this ambiguity is heterogeneity.
Heterogeneity refers to the patient population not having much in common with one another, as opposed to homogeneity, which refers to all of the patients being similar. So, one of the reason's carpal tunnel surgery works well is that, for the most part, almost all of the patients who get the surgery truly have carpal tunnel syndrome.
That is not always the case, however. In fact, it is rarely the case. Patients are human beings, and human beings are unique, and this makes it difficult to assume that just because one treatment works for one patient, that it will also mean that the same treatment approach will work the same on another, different patient.
For example, in one of my favorite studies by Lurie and Weinstein et al from the journal Spine in 2006, they investigated the diagnoses associated with different surgical procedures. What they found was that for hip replacements, there was a high degree of homogeneity in terms of why the diagnoses for which the surgery was performed (i.e., most patients had hip replacements because of osteoarthritis), but that there was a far, far higher level of heterogeneity for when a lumbar fusion was performed.
The consequence of this heterogeneity is that the outcomes for hip replacements are far better than they are for lumbar fusions. For hip replacements, if the patient has a hip replacement, there is a very high likelihood it was done for a legitimate reason. For lumbar fusions there is a mix of some patients who were good candidates for the fusion and other patients who probably were not good candidates. As result, the outcome data for fusions is not particularly good. If patients were more narrowly selected, perhaps the outcomes would be better.
This same phenomenon- heterogeneity leading to less than optimal outcomes, comes up quite frequently. Other examples:
1. Spine injections. One of the main scopes of my practice is spine injections, including epidural injections, zgyapophysial joint injection, sacro-iliac joint injections, etc. The outcome data is not particularly good overall, although there are some subsets of patients for which the data is good (e.g., the aforementioned use of transforaminal epidurals for radicular pain, or SI joint injections).
The main issue is, again, heterogeneity. Spine injectionists, as a group, do a horrible in terms of applying standard of care. There is a great paper by Janna Friedly in 2008 that showed that only 42% of spine injections are performed with use of fluoroscopy. Only 42%! That is a bare minimum standard for an appropriate injection, and that is not factoring in whether the appropriate pain generator was selected, whether the proper level was selected, and whether the appropriate approach was used. This means that the absolute upper limit of how many spinal injections are performed appropriately is 42%, but when you factor in these other factors, it's likely that only 20-30% of spine injections are being performed according to standard of care. It's not shocking, then, that we are not seeing great outcome data.
2. Fibromyalgia. Fibromyalgia is largely a waste-basket term that encompasses a huge number of different diagnoses. In other words, there is a high level of heterogeneity. Just this morning, I was discussiong a fibromyalgia patient I see along with one of my rheumatology colleagues, and she is far from the typical fibromyalgia patient. A large number of fibromyalgia patients may have other diagnoses that have not been fully explored, including rhuematologic disease, depression, bipolar II disorder, myopathy, cervical myelopathy, Lyme disease, Epstein Barr Virus, etc.
Which brings me back to my conversation with patients. This is why, given the real world heterogeneity of patients, so often my answer to there questions is "I don't know for sure, but ...."
My answer is always some variation of "I don't know for sure, but ..."
Some procedures work very well, examples including:
1. Transforaminal epidural steroid injections for radicular pain ("sciatica") associated with a herniated disk.
2. Carpal tunnel release for carpal tunnel syndrome
3. Knee replacements for knee osteoarthritis
For many other procedures, however, there is far greater ambiguity. One of the main reasons for this ambiguity is heterogeneity.
Heterogeneity refers to the patient population not having much in common with one another, as opposed to homogeneity, which refers to all of the patients being similar. So, one of the reason's carpal tunnel surgery works well is that, for the most part, almost all of the patients who get the surgery truly have carpal tunnel syndrome.
That is not always the case, however. In fact, it is rarely the case. Patients are human beings, and human beings are unique, and this makes it difficult to assume that just because one treatment works for one patient, that it will also mean that the same treatment approach will work the same on another, different patient.
For example, in one of my favorite studies by Lurie and Weinstein et al from the journal Spine in 2006, they investigated the diagnoses associated with different surgical procedures. What they found was that for hip replacements, there was a high degree of homogeneity in terms of why the diagnoses for which the surgery was performed (i.e., most patients had hip replacements because of osteoarthritis), but that there was a far, far higher level of heterogeneity for when a lumbar fusion was performed.
The consequence of this heterogeneity is that the outcomes for hip replacements are far better than they are for lumbar fusions. For hip replacements, if the patient has a hip replacement, there is a very high likelihood it was done for a legitimate reason. For lumbar fusions there is a mix of some patients who were good candidates for the fusion and other patients who probably were not good candidates. As result, the outcome data for fusions is not particularly good. If patients were more narrowly selected, perhaps the outcomes would be better.
This same phenomenon- heterogeneity leading to less than optimal outcomes, comes up quite frequently. Other examples:
1. Spine injections. One of the main scopes of my practice is spine injections, including epidural injections, zgyapophysial joint injection, sacro-iliac joint injections, etc. The outcome data is not particularly good overall, although there are some subsets of patients for which the data is good (e.g., the aforementioned use of transforaminal epidurals for radicular pain, or SI joint injections).
The main issue is, again, heterogeneity. Spine injectionists, as a group, do a horrible in terms of applying standard of care. There is a great paper by Janna Friedly in 2008 that showed that only 42% of spine injections are performed with use of fluoroscopy. Only 42%! That is a bare minimum standard for an appropriate injection, and that is not factoring in whether the appropriate pain generator was selected, whether the proper level was selected, and whether the appropriate approach was used. This means that the absolute upper limit of how many spinal injections are performed appropriately is 42%, but when you factor in these other factors, it's likely that only 20-30% of spine injections are being performed according to standard of care. It's not shocking, then, that we are not seeing great outcome data.
2. Fibromyalgia. Fibromyalgia is largely a waste-basket term that encompasses a huge number of different diagnoses. In other words, there is a high level of heterogeneity. Just this morning, I was discussiong a fibromyalgia patient I see along with one of my rheumatology colleagues, and she is far from the typical fibromyalgia patient. A large number of fibromyalgia patients may have other diagnoses that have not been fully explored, including rhuematologic disease, depression, bipolar II disorder, myopathy, cervical myelopathy, Lyme disease, Epstein Barr Virus, etc.
Which brings me back to my conversation with patients. This is why, given the real world heterogeneity of patients, so often my answer to there questions is "I don't know for sure, but ...."
Sunday, February 1, 2009
quick thoughts on the editorial process
I'm editing some journal articles this weekend, which has me thinking about writing in general, and how information is disseminated.
One thing that is striking to me when editing someone else's work is the constant battle to make sure that I am editing the article that is in front of me, rather than some ideal, perfect paper that will never exist. Of course I wish every study had 10 times as many participants, had measured variables that are near and dear to my heart, etc. But I think in the big picture, that is not my job. My job is not to tell the author the flaws that kept them from perfection- my job is to give honest feedback about the strengths and weaknesses of their paper, and help make it better.
I've heard this discussed in the context of movie reviews, and how a good critic will review the movie in front of them, and not the movie that they wish had been made. One example that sticks out for me is the "X-men" series. I think that series made a fatal flaw in casting Halle Berry as Storm, which was a role that Angela Bassett was born to play. For a long time, I couldn't get past that casting error (and it's a big one). But it's still a very good movie when viewed on its own merits, and that is the basis by which I should judge it.
I have to keep this in mind when I am reading the work of others. Even the worst articles I review represent probably hundreds of hours of work, and I have to keep that in mind. Even for papers that are fatally flawed, I am still appreciative that the authors cared enough to try and share their knowledge to the medical and scientific community.
Another important point that comes from reviewing is just how important good writing is. It is such a pleasure to read a well-written scientific paper. Unfortunately, they are pretty rare. Science writers seem to have an unyielding affection for the passive voice, amongst other things, and are too adherent to jargon. It is a shame when good ideas get lost in bad grammar.
The final thought I have is that one reason I like editing is that hopefully it makes me a better writer myself. In Stephen King's "On Writing" (which I consider the best book ever written about writing), he talks about how if you want to be a great writer, the importance of being a reader. I think that also applies to scientific writing. My helping others improve their scientific writing, hopefully I will be a better writer, and and better medical scientist.
One thing that is striking to me when editing someone else's work is the constant battle to make sure that I am editing the article that is in front of me, rather than some ideal, perfect paper that will never exist. Of course I wish every study had 10 times as many participants, had measured variables that are near and dear to my heart, etc. But I think in the big picture, that is not my job. My job is not to tell the author the flaws that kept them from perfection- my job is to give honest feedback about the strengths and weaknesses of their paper, and help make it better.
I've heard this discussed in the context of movie reviews, and how a good critic will review the movie in front of them, and not the movie that they wish had been made. One example that sticks out for me is the "X-men" series. I think that series made a fatal flaw in casting Halle Berry as Storm, which was a role that Angela Bassett was born to play. For a long time, I couldn't get past that casting error (and it's a big one). But it's still a very good movie when viewed on its own merits, and that is the basis by which I should judge it.
I have to keep this in mind when I am reading the work of others. Even the worst articles I review represent probably hundreds of hours of work, and I have to keep that in mind. Even for papers that are fatally flawed, I am still appreciative that the authors cared enough to try and share their knowledge to the medical and scientific community.
Another important point that comes from reviewing is just how important good writing is. It is such a pleasure to read a well-written scientific paper. Unfortunately, they are pretty rare. Science writers seem to have an unyielding affection for the passive voice, amongst other things, and are too adherent to jargon. It is a shame when good ideas get lost in bad grammar.
The final thought I have is that one reason I like editing is that hopefully it makes me a better writer myself. In Stephen King's "On Writing" (which I consider the best book ever written about writing), he talks about how if you want to be a great writer, the importance of being a reader. I think that also applies to scientific writing. My helping others improve their scientific writing, hopefully I will be a better writer, and and better medical scientist.
Sunday, January 25, 2009
Evidence-based medicine and false precision
In Nassim Taleb's outstanding book, The Black Swan, he discusses how mathematical models are used in predicting the stock market. His general contention is that while there is a lot of thought and brainpower used in designing these models, they are still at best approximations of real financial markets, and can be catastrophically wrong. He discussed the 1987 stock market crash as an example. The book was published before this year's market collapse, but the 2008 crash helped illustrate just how important his point was.
I would term the phenomenon he is discussing "false precision." Because the statistical models work 70%, 80%, or 90% of the time, there is a false assumption that they will work 100% of the time. That is not true, but the precision that we see under "ordinary" circumstances can fool us into thinking that we will see that precision all of the time.
Many people have seen a similar phenomenon when they play fantasy sports. I have played both fantasy football and baseball in the past, but I'll use football as an example, since it is the more popular game (I don't play either anymore, since while fun, they are major time sinks). Fantasy football is what some will call a "mirror game"- it is based on football, uses many of the same principles of football, but it is not the same game. Back in era when I played, Daunte Cullpepper was probably the best QB in fantasy football, since he would throw a lot of touchdowns (thanks to Randy Moss and Cris Carter) and run for a decent amount of yards.
After playing for a few years, it was easy to start thinking that Daunte Cullpepper was the best QB in the NFL. He wasn't- Peyton Manning and Tom Brady were, even that wasn't reflected in their fantasy football numbers. What happened was that it became very easy to confuse that while fantasy football is a reasonable approximation of football, it is NOT the same thing as football. It is a mirror, a reflection. But the output data we received from playing fantasy football made us think that we knew more about the game than we really did. This false precision.
The data for the stock market and the data for sports are much, much cleaner than they are for medicine. It is not even close. The data for educational outcomes (used for things like No Child Left Behind) is much cleaner than it is for medicine, and No Child Left Behind, for all of it's strengths, is still dealing with a messy data set that only mirrors reality. Medicine is much messier than that.
Don't get me wrong- I think medicine needs to more toward using evidence to inform our judgments- just like you need to prep for your fantasy draft by reading and evaluating, just like you should pick you mutual funds by reading their prospecti, just like you want your kids to be educated using the most evidence-based techniques, when it comes to treating patients, I think we need to look to the evidence. We shouldn't be just making stuff crap up- we should make our best effort to actually use data.
We just can't be too arrogant about it. We can't assume that just because we are using the best data available that we are unfailable. We can't assume that because we are using evidence-based practices that our precision is any better than 50-70%.
I will be absolutely shocked if, within my lifetime, the predictive models we use for health care even approach the precision of the predictive models we currently use for the stock market, baseball, and the weather. And as we've all experienced, those models aren't particularly precise, and when they are wrong, they can be wrong with disastrous consequences.
The main reason medical models will never be as good is that the populations we are looking at are too heterogeneous. For example, the largest medical study ever conducted was the Women's Health Initiative. I am having trouble locating a specific cost for the WHI, but my memory was that it was a $12 billion study.
The study was about as well designed as a medical study could be. For all that said, it was largely inconclusive, and there have been literally hundreds of papers that have tried to parse the data to figure out what it means. The underlying problem is that the group of patients they studied- older women- was too heterogenous. It turns out that older women is too broad a category, and so the conslusions that would be true with one subgroup don't really apply to another subgroup.
Within my own field of specialty, the largest study ever dedicated to outcomes for patients with low back pain was the SPORT trial. Same problem- the group studied was too heterogeneous, and the study could not account for human behavior sufficiently, so we don't really have much more insight on how well back surgery works for low back pain then we did before the study was conducted.
Just to beat this dead horse some more, I am going to pick the first study in this month's New England Journal of Medicine. It is about using oral steroids for kids with wheezing, trying to determine whether steroids make a difference or not. The answer is "we can't tell," although I suspect it will be reported as "steroids don't make a difference." I just checked- the first story in Google News under the search terms (steroids, wheezing) is "Oral Steroids Ineffective in Treatment of Preschool Virus-Induced Wheezing." Why is it reported this way? I suspect that headlines of "we just spent several million dollars studying something, and we know only marginally more than we did before we did the study" doesn't sell a lot of copy.
If we look at the actual data, the kids who got steroids did spend less time in the hospital than kids who didn't (medians were 13.9 hours v 11 hours), but that this difference did not achieve statistical significance. Reporting this as "steroids don't make a difference" is a simplification, because the real picture is more complicated. This is not a criticism of the study authors- they acknowledge the study's limitations in their discussion. But most people don't have time for shades of gray.
The way I would interpret this study is that there probably is a sub-group of kids who would benefit from steroids, and we haven't figured out what that subgroup is yet. We just don't know enough yet. Even if we studied this with the same resources we use or analyzing the stock market, we are going to be wrong at least 5-10% of the time. Sometimes catastrophically wrong.
Going back to something closer to my scope of practice- low back pain. Low back pain is the most common chief complaint I see, and I think that I am very good at managing it. I am not perfect, and nobody else is either. If you have a failure rate of 10% in treating low back pain, you are outstanding.
One of the most frequent questions I am asked is whether a patient is best managed by surgery, spine injections, physical therapy, accupuncture, massage, chiropractic manipulation, nutritional supplementation, etc. The short answer is "I have no idea, really."
That is just being appropriately humble- in truth, I am easily in the top 1% in my competence to answer the question, but I am aware of the limitations of my and all of medicine's knowledge on the topic. Low back pain is a very complex topic, far more complex than the stock market, and there is an upper limit to how precisely we can answer thesse questions. I am just aware of our limitations.
What I believe is that all these different approaches, ranging for traditional approaches like physical therapy, surgery, and injections to more non-traditional approaches like accupuncture and nutritional supplements, they all work, but we have not yet perfected how to stratify which approach will work for which patients.
Does this mean that "well, since we can't be perfect, we should throw away this information?" Absolutely not! But it does mean that we need to be humble enough to realize that we are going to be wrong 15-20% of the time, at least.
Giving specific examples, with patients with low back pain radiating leg pain, the evidence for use of a neuropathic agent like Neurontin is significantly stornger than a muscle relaxant like Flexeril (which, in truth, is more of an anxietiolic than it is a muscle relaxant), so therefore I will use Neurontin more often.
As another example, the data is much stronger in support of using a transforaminal approach than an interlaminar approach for epidural steroid injections in the low back, so that is the approach I use the vast majority of the time.
As yet another example, if a patient has a directional preference (e.g., pain worse with bending forward), then a physical therapy program that takes that directional preference into account is far more likely to be succesful.
So, of course, I should adhere to these evidence-based guidelines. I just can't be so arrogant as to assume that just because I am adhering to evidence-based guidelines that I will always be right.
Coming back now to health policy .... there is a strong movement underway to make physicians more accountable to evidence-based guidelines. I think this is a good thing. Patients who have diabetes should have their hemoglobin A1c checked regularly, patients should have their blood pressure checked regularly and placed on appropriate agents, etc.
We just can't get arrogant about it. These standards we are moving toward will better approximate good health care- they do not equal good health care.
Just my 2 cents.
I would term the phenomenon he is discussing "false precision." Because the statistical models work 70%, 80%, or 90% of the time, there is a false assumption that they will work 100% of the time. That is not true, but the precision that we see under "ordinary" circumstances can fool us into thinking that we will see that precision all of the time.
Many people have seen a similar phenomenon when they play fantasy sports. I have played both fantasy football and baseball in the past, but I'll use football as an example, since it is the more popular game (I don't play either anymore, since while fun, they are major time sinks). Fantasy football is what some will call a "mirror game"- it is based on football, uses many of the same principles of football, but it is not the same game. Back in era when I played, Daunte Cullpepper was probably the best QB in fantasy football, since he would throw a lot of touchdowns (thanks to Randy Moss and Cris Carter) and run for a decent amount of yards.
After playing for a few years, it was easy to start thinking that Daunte Cullpepper was the best QB in the NFL. He wasn't- Peyton Manning and Tom Brady were, even that wasn't reflected in their fantasy football numbers. What happened was that it became very easy to confuse that while fantasy football is a reasonable approximation of football, it is NOT the same thing as football. It is a mirror, a reflection. But the output data we received from playing fantasy football made us think that we knew more about the game than we really did. This false precision.
The data for the stock market and the data for sports are much, much cleaner than they are for medicine. It is not even close. The data for educational outcomes (used for things like No Child Left Behind) is much cleaner than it is for medicine, and No Child Left Behind, for all of it's strengths, is still dealing with a messy data set that only mirrors reality. Medicine is much messier than that.
Don't get me wrong- I think medicine needs to more toward using evidence to inform our judgments- just like you need to prep for your fantasy draft by reading and evaluating, just like you should pick you mutual funds by reading their prospecti, just like you want your kids to be educated using the most evidence-based techniques, when it comes to treating patients, I think we need to look to the evidence. We shouldn't be just making stuff crap up- we should make our best effort to actually use data.
We just can't be too arrogant about it. We can't assume that just because we are using the best data available that we are unfailable. We can't assume that because we are using evidence-based practices that our precision is any better than 50-70%.
I will be absolutely shocked if, within my lifetime, the predictive models we use for health care even approach the precision of the predictive models we currently use for the stock market, baseball, and the weather. And as we've all experienced, those models aren't particularly precise, and when they are wrong, they can be wrong with disastrous consequences.
The main reason medical models will never be as good is that the populations we are looking at are too heterogeneous. For example, the largest medical study ever conducted was the Women's Health Initiative. I am having trouble locating a specific cost for the WHI, but my memory was that it was a $12 billion study.
The study was about as well designed as a medical study could be. For all that said, it was largely inconclusive, and there have been literally hundreds of papers that have tried to parse the data to figure out what it means. The underlying problem is that the group of patients they studied- older women- was too heterogenous. It turns out that older women is too broad a category, and so the conslusions that would be true with one subgroup don't really apply to another subgroup.
Within my own field of specialty, the largest study ever dedicated to outcomes for patients with low back pain was the SPORT trial. Same problem- the group studied was too heterogeneous, and the study could not account for human behavior sufficiently, so we don't really have much more insight on how well back surgery works for low back pain then we did before the study was conducted.
Just to beat this dead horse some more, I am going to pick the first study in this month's New England Journal of Medicine. It is about using oral steroids for kids with wheezing, trying to determine whether steroids make a difference or not. The answer is "we can't tell," although I suspect it will be reported as "steroids don't make a difference." I just checked- the first story in Google News under the search terms (steroids, wheezing) is "Oral Steroids Ineffective in Treatment of Preschool Virus-Induced Wheezing." Why is it reported this way? I suspect that headlines of "we just spent several million dollars studying something, and we know only marginally more than we did before we did the study" doesn't sell a lot of copy.
If we look at the actual data, the kids who got steroids did spend less time in the hospital than kids who didn't (medians were 13.9 hours v 11 hours), but that this difference did not achieve statistical significance. Reporting this as "steroids don't make a difference" is a simplification, because the real picture is more complicated. This is not a criticism of the study authors- they acknowledge the study's limitations in their discussion. But most people don't have time for shades of gray.
The way I would interpret this study is that there probably is a sub-group of kids who would benefit from steroids, and we haven't figured out what that subgroup is yet. We just don't know enough yet. Even if we studied this with the same resources we use or analyzing the stock market, we are going to be wrong at least 5-10% of the time. Sometimes catastrophically wrong.
Going back to something closer to my scope of practice- low back pain. Low back pain is the most common chief complaint I see, and I think that I am very good at managing it. I am not perfect, and nobody else is either. If you have a failure rate of 10% in treating low back pain, you are outstanding.
One of the most frequent questions I am asked is whether a patient is best managed by surgery, spine injections, physical therapy, accupuncture, massage, chiropractic manipulation, nutritional supplementation, etc. The short answer is "I have no idea, really."
That is just being appropriately humble- in truth, I am easily in the top 1% in my competence to answer the question, but I am aware of the limitations of my and all of medicine's knowledge on the topic. Low back pain is a very complex topic, far more complex than the stock market, and there is an upper limit to how precisely we can answer thesse questions. I am just aware of our limitations.
What I believe is that all these different approaches, ranging for traditional approaches like physical therapy, surgery, and injections to more non-traditional approaches like accupuncture and nutritional supplements, they all work, but we have not yet perfected how to stratify which approach will work for which patients.
Does this mean that "well, since we can't be perfect, we should throw away this information?" Absolutely not! But it does mean that we need to be humble enough to realize that we are going to be wrong 15-20% of the time, at least.
Giving specific examples, with patients with low back pain radiating leg pain, the evidence for use of a neuropathic agent like Neurontin is significantly stornger than a muscle relaxant like Flexeril (which, in truth, is more of an anxietiolic than it is a muscle relaxant), so therefore I will use Neurontin more often.
As another example, the data is much stronger in support of using a transforaminal approach than an interlaminar approach for epidural steroid injections in the low back, so that is the approach I use the vast majority of the time.
As yet another example, if a patient has a directional preference (e.g., pain worse with bending forward), then a physical therapy program that takes that directional preference into account is far more likely to be succesful.
So, of course, I should adhere to these evidence-based guidelines. I just can't be so arrogant as to assume that just because I am adhering to evidence-based guidelines that I will always be right.
Coming back now to health policy .... there is a strong movement underway to make physicians more accountable to evidence-based guidelines. I think this is a good thing. Patients who have diabetes should have their hemoglobin A1c checked regularly, patients should have their blood pressure checked regularly and placed on appropriate agents, etc.
We just can't get arrogant about it. These standards we are moving toward will better approximate good health care- they do not equal good health care.
Just my 2 cents.
Sunday, January 4, 2009
The Last Lecture
I just finished reading Randy Pausch's "The Last Lecture." Wow- what an inspiring book. For those who haven't read it (or seen the actual lecture on Youtube, which is worth looking up), Randy Pausch was a computer science professor at Carnegie Mellon with pancreatic cancer. He was given the opportunity to deliver one last lecture to Carnegie Mellon, and more importantly to his children. His lecture is about living out your childhood dreams, and just being a better person.
One thing that struck me in the book (it's not mentioned in the lecture itself) is an anecdote about when he was a professor at the University of Virginia. He was teaching a class on user interface, and would start the semester by bringing in a functional VCR to the front of the room, and then smashing it with a sledgehammer. His point was designers cannot lose site of the end user- the people who actually use the product. VCRs can do a lot of cool things, but they are frustrating to use.
This applies to a lot of consumer products. As an example, this is the year I finally made the switch over to using a Mac. So far, I've been very happy using the Mac. It is striking how much more work and play I have now that I don't waste 30-60 minutes daily dealing with anti-virus and anti-spam software. It's just an easier, cleaner, and more fun interface.
Anyway, reading Randy Pausch's book, I was struck by how often in medicine we lose site of the end user- the patient. Just like computer and VCR designers, we can do so many amazing things in medicine. But none of that really makes a difference unless it affects the end product, and improves the patient's experience.
And I really don't want to be smashing patients with sledgehammers.
One thing that struck me in the book (it's not mentioned in the lecture itself) is an anecdote about when he was a professor at the University of Virginia. He was teaching a class on user interface, and would start the semester by bringing in a functional VCR to the front of the room, and then smashing it with a sledgehammer. His point was designers cannot lose site of the end user- the people who actually use the product. VCRs can do a lot of cool things, but they are frustrating to use.
This applies to a lot of consumer products. As an example, this is the year I finally made the switch over to using a Mac. So far, I've been very happy using the Mac. It is striking how much more work and play I have now that I don't waste 30-60 minutes daily dealing with anti-virus and anti-spam software. It's just an easier, cleaner, and more fun interface.
Anyway, reading Randy Pausch's book, I was struck by how often in medicine we lose site of the end user- the patient. Just like computer and VCR designers, we can do so many amazing things in medicine. But none of that really makes a difference unless it affects the end product, and improves the patient's experience.
And I really don't want to be smashing patients with sledgehammers.
Bluefish
My favorite fish, growing up as a child in New Jersey, was bluefish. When I went away for college in Wisconsin, it didn't occur to me that they didn't have bluefish there, and I didn't rediscover bluefish until medical school.
In medical school, my love affair with bluefish really began. I loved the same things that I loved as a kid- how it was a big, powerfully flavored fish, but in medical school there was another, more important virtue. It was cheap. Very cheap. Another benefit was that most other people weren't particularly familiar with it, so when it was a great fish to prepare for dinner to impress a date (they didn't have to know it was $1.99/lb).
Now that I am older and have more perspective, I realize that many people don't like bluefish. Because it is so oily, it does not travel well, so it is very hard to get outside of the east coast. For example, I never ate bluefish at all in the years I've lived in Wisconsin, Illinois, or Arkansas- you just couldn't find it out there.
Another thing I realize about bluefish is that it is really a perfect metaphor for my homestate of New Jersey- unlike other fish that try to pretend that they aren't a fish ("try the flounder, it's not particularly fishy"), there is no mistaking bluefish- it very much tastes like a fish. Like NJ, when bluefish is at its worst, it's oily, rancid, and particularly smelly. But when it is at it's best, there is nothing better- it's powerful, colorful, flavorful, and completely in your face with how fishy it is. I just read on Wikipedia that the bluefish is cannabilistic. That fits too.
It's going to be a real stretch for me to try and extend this discussion of bluefish into something remotely relevant for a Kinemedics blog. In all honesty, I am just excited about two beautiful fish I picked up at Wholey's fish market in the Strip District, which is to me the most Pittsburghian location in Pittsburgh. But overly extended metaphors is what I do, so here goes ....
In a few weeks, I will be attending the Association of Academic Physiatrists (AAP) meeting. This is the first academic organization with which I became involved, and is still the meeting for which I have the closest ties.
If I have gained one thing from academic meetings, it is the value of meeting other people at other locations. I went to a very good residency program in New Jersey, but as good as the program was, the things I learned there were still just a subset of a much larger picture.
My favorite experience from the AAP was one of my earliest meeting's there. Somehow or another, I ended up sitting at the table for the AAP's Education Committee. In retrospect, I don't think I was supposed to be there, and I was the only resident sitting at a table with Residency Program Directors. I was too naive (and probably too brash) to keep quiet, so as points were discussed, I offered my opinions. I think the committee appreciated that I cared enough to want to help, and before I knew it, I was an actual member of the committee. I think this is how a lot of opportunities in life present themselves- they are accidents initially, but the world always needs people who give a crap.
It has been tremendously beneficial to my career to meet people from other organizations, and learn from them. While the fundamentals of who I am, both as a person and a clinician, are deeply rooted in my upbringing and training in New Jersey, I am glad that I have branched out and explored other parts of the world and expanded my experiences.
All that said, I am just really looking forward to that bluefish tonight.
Saturday, January 3, 2009
"It's a very odd experience, how this is starting to feel normal"
Over the past few days, I spent some time with a friend's family in the ICU waiting room.
It was an interesting experience for me- it certainly useful every once in a while to remember how things feel from the perspective of patients and there family.
One of the things I gained from the experience is the perspective that overall, modern medicine is a wonderful thing. There are a lot of complaints with the flaws in medicine, and we have all experienced our shares of frustrations. But overall, modern medicine can do some amazing things. This is particularly true at the University of Pittsburgh- I have been thoroughly impressed since I have come here with the overall quality of care, both technically, and just the general sense of humanity. This was over a holiday week (New Years), and the number of people working at UPMC who tirelessly extended themselves to take care of the patients and their families impressed me. I was proud to be part of the organization.
I was also impressed by the love family can bring to a patient. In the waiting room of the ICU, a few families were camped out. Some had not left the side of their loved ones for weeks at a time. As a physician, I was vaguely aware that families stay at the hospital to be with their loved ones, but I had never actually seen it first hand.
One thing was striking is how disorienting the experience is in terms of sense of time. Staying bunkered in the waiting rooms completely removes you from those external cues that separate morning from night. One day just blends into the next.
Another thing that was evident to me was the toll the human body takes from the lack of movement. This is true of the patients, but you also see it in the family members who are staying with the patients. Between the lack of exercise and the ample amounts of caffeine most family members consume, plus the intense emotional toll of their loved ones being sick, nearly everyone in the ICU waiting room was shaking their legs vigorously while they sat.
One of the family members said to me "it's a very odd experience, how this is starting to feel normal." I know what he meant, and I agree it is odd.
I was fortunate enough to make it through medical school without any patients dying on me. I remember the first time a patient of mine died- I was an intern in Cooperstown, NY. I wasn't expecting it at all- of the hundreds of patients I had seen that year, the one who died was by no means the sickest. But all of a sudden, there she was- just moments earlier she had been breathing and had a pulse, and now she lay motionless.
I had worked closely with a medical student that month, and the two of us just went into a back room, and I cried. For a long time- probably an hour. I had seen dead people before, but I had never seen someone die- actually go through the process of dying to the point where they were dead. And having lived through it, the experience was definitely not normal.
I can't say that I am used to the experience now- my line of work does not lend itself to patients dying very often. But I don't know that I would cry for an hour anymore. I don't think I am particularly jaded, but the process of death is just, well, more normal for me. And as my friend said to me the other day in the ICU, when I reflect on it, is very odd when the process starts feeling normal.
It was an interesting experience for me- it certainly useful every once in a while to remember how things feel from the perspective of patients and there family.
One of the things I gained from the experience is the perspective that overall, modern medicine is a wonderful thing. There are a lot of complaints with the flaws in medicine, and we have all experienced our shares of frustrations. But overall, modern medicine can do some amazing things. This is particularly true at the University of Pittsburgh- I have been thoroughly impressed since I have come here with the overall quality of care, both technically, and just the general sense of humanity. This was over a holiday week (New Years), and the number of people working at UPMC who tirelessly extended themselves to take care of the patients and their families impressed me. I was proud to be part of the organization.
I was also impressed by the love family can bring to a patient. In the waiting room of the ICU, a few families were camped out. Some had not left the side of their loved ones for weeks at a time. As a physician, I was vaguely aware that families stay at the hospital to be with their loved ones, but I had never actually seen it first hand.
One thing was striking is how disorienting the experience is in terms of sense of time. Staying bunkered in the waiting rooms completely removes you from those external cues that separate morning from night. One day just blends into the next.
Another thing that was evident to me was the toll the human body takes from the lack of movement. This is true of the patients, but you also see it in the family members who are staying with the patients. Between the lack of exercise and the ample amounts of caffeine most family members consume, plus the intense emotional toll of their loved ones being sick, nearly everyone in the ICU waiting room was shaking their legs vigorously while they sat.
One of the family members said to me "it's a very odd experience, how this is starting to feel normal." I know what he meant, and I agree it is odd.
I was fortunate enough to make it through medical school without any patients dying on me. I remember the first time a patient of mine died- I was an intern in Cooperstown, NY. I wasn't expecting it at all- of the hundreds of patients I had seen that year, the one who died was by no means the sickest. But all of a sudden, there she was- just moments earlier she had been breathing and had a pulse, and now she lay motionless.
I had worked closely with a medical student that month, and the two of us just went into a back room, and I cried. For a long time- probably an hour. I had seen dead people before, but I had never seen someone die- actually go through the process of dying to the point where they were dead. And having lived through it, the experience was definitely not normal.
I can't say that I am used to the experience now- my line of work does not lend itself to patients dying very often. But I don't know that I would cry for an hour anymore. I don't think I am particularly jaded, but the process of death is just, well, more normal for me. And as my friend said to me the other day in the ICU, when I reflect on it, is very odd when the process starts feeling normal.
Brazilian Jiu-Jitsu, Hammer and Nail, Moo Ridge, and the need to keep learning
I am in the process of learning Brazilian Jiu-Jitsu. I love it so far. I think BJJ is a great sport in of itself, but I also love the process of learning something where you are a complete novice.
Some parts of the sport are easier for me to pick up than others. The moves make a lot of sense to me- partly because I have some background in wrestling, partly because I have a very strong academic background in biomechanics, and partly because I've watched enough mixed martial arts (MMA) over the years to have some familiarity with basic terminology.
By far the hardest part for me is learning how to not rely on my size advantage. I am probably the biggest guy in the dojo, and in most cases outweigh the people I am practicing with by at least 50 pounds. Because of this, the temptation is to simply overpower people or muscle through. When I have sparred before, even with very skilled combatants, I have been able to succesfully use my size to overpower opponents. What I am trying to learn in BJJ class is that you can't simply do that with a skilled opponent. Today, every time I lunged forward, the people I was rolling with were able to grab the sleeve of my gi and start working a wrist lock or arm bar. At first, I was able to fight them off, but eventually I just gassed out.
This reminds me of one of my favorite medicine cliches- to a hammer, everything looks like a nail. What this means is that physicians tend to know that the things they know, and oftentimes we don't think outside of the box enough to consider options outside of our base skill set.
One of my favorite anecdotes related to this point dates back to my grad school days. Even though my PhD research was related to sports biomechanics in female athletes, many other people in my department did research on the anatomy of fossils. So one summer I was part of an expedition in western Colorado, looking for fossils that were part of the initial mammalian radiation.
When I tell people I was on a fossil dig, they find that incredibly interesting. It wasn't for me- it is probably the singular task that I have the least talent in the world. Also, these weren't big fossils- these were very small mammals the size of a mouse, so the bones we were looking at were smaller than my thumbnail.
I remember one day in particular that highlighted how bad I was at fossil collecting. I was looking at a patch of pebbles that was about 1 yard by 1 yard for what must have been several hours. I kept staring at the pebbles, and I couldn't see anything- they just looked like pebbles to me.
Then our expedition leader, Maureen, came by and looked at the same 1 yard patch I had stared at for the past several hours, and within 30 seconds picked out 3 or 4 fossils. It was humiliating, but it was also one of the greatest lessons I have ever had in my life- you can't find something if you don't know what you are looking for.
My lone contribution to the 6 week expedition is that I came up with a clever name for the fossil site. The whole time we were looking for fossils, hundreds of cows would walk over and stare at our team, so I called the fossil site Moo Ridge. I think Moo Ridge may even be referenced in some Paleontology journal somewhere.
The reason I bring this up- when relate this story to medical students and residents, I make sure they understand that everyone has their Moo Ridge- we all have weakspots where we can't see things that are right in front of us, because we don't know what we are looking for. Sometimes we learn to compensate for our blind spots by relying on some other strengths to compensate. But just like my experience in Brazilian Jiu-Jitsu class taught me today, eventually your strengths aren't enough, and you need to learn new skills.
I am reminded of this every week when I attend the Sports Medicine faculty's weekly case series. The great Freddie Fu, the chairman of Pitt's Sports Medicine team, likes to hammer home the point that being good is not good enough- that we always have to ask the question- "what can I do better?"
It's a great question, and it's one that I, and every physician, needs to be asking themself every day. And not just when they are getting choked out on the BJJ mat.
Some parts of the sport are easier for me to pick up than others. The moves make a lot of sense to me- partly because I have some background in wrestling, partly because I have a very strong academic background in biomechanics, and partly because I've watched enough mixed martial arts (MMA) over the years to have some familiarity with basic terminology.
By far the hardest part for me is learning how to not rely on my size advantage. I am probably the biggest guy in the dojo, and in most cases outweigh the people I am practicing with by at least 50 pounds. Because of this, the temptation is to simply overpower people or muscle through. When I have sparred before, even with very skilled combatants, I have been able to succesfully use my size to overpower opponents. What I am trying to learn in BJJ class is that you can't simply do that with a skilled opponent. Today, every time I lunged forward, the people I was rolling with were able to grab the sleeve of my gi and start working a wrist lock or arm bar. At first, I was able to fight them off, but eventually I just gassed out.
This reminds me of one of my favorite medicine cliches- to a hammer, everything looks like a nail. What this means is that physicians tend to know that the things they know, and oftentimes we don't think outside of the box enough to consider options outside of our base skill set.
One of my favorite anecdotes related to this point dates back to my grad school days. Even though my PhD research was related to sports biomechanics in female athletes, many other people in my department did research on the anatomy of fossils. So one summer I was part of an expedition in western Colorado, looking for fossils that were part of the initial mammalian radiation.
When I tell people I was on a fossil dig, they find that incredibly interesting. It wasn't for me- it is probably the singular task that I have the least talent in the world. Also, these weren't big fossils- these were very small mammals the size of a mouse, so the bones we were looking at were smaller than my thumbnail.
I remember one day in particular that highlighted how bad I was at fossil collecting. I was looking at a patch of pebbles that was about 1 yard by 1 yard for what must have been several hours. I kept staring at the pebbles, and I couldn't see anything- they just looked like pebbles to me.
Then our expedition leader, Maureen, came by and looked at the same 1 yard patch I had stared at for the past several hours, and within 30 seconds picked out 3 or 4 fossils. It was humiliating, but it was also one of the greatest lessons I have ever had in my life- you can't find something if you don't know what you are looking for.
My lone contribution to the 6 week expedition is that I came up with a clever name for the fossil site. The whole time we were looking for fossils, hundreds of cows would walk over and stare at our team, so I called the fossil site Moo Ridge. I think Moo Ridge may even be referenced in some Paleontology journal somewhere.
The reason I bring this up- when relate this story to medical students and residents, I make sure they understand that everyone has their Moo Ridge- we all have weakspots where we can't see things that are right in front of us, because we don't know what we are looking for. Sometimes we learn to compensate for our blind spots by relying on some other strengths to compensate. But just like my experience in Brazilian Jiu-Jitsu class taught me today, eventually your strengths aren't enough, and you need to learn new skills.
I am reminded of this every week when I attend the Sports Medicine faculty's weekly case series. The great Freddie Fu, the chairman of Pitt's Sports Medicine team, likes to hammer home the point that being good is not good enough- that we always have to ask the question- "what can I do better?"
It's a great question, and it's one that I, and every physician, needs to be asking themself every day. And not just when they are getting choked out on the BJJ mat.
Subscribe to:
Posts (Atom)