Specalists w/o Measures - BLIND lead the BLIND!
First and foremost, I refuse to blame the doctors for what I believe is one of the biggest problems with our medical system, but I will say that I think the medical administrations are clueless. They can tell you how many beds are full and what their costs are. However, they can not tell you if they are better or worse than anyone else for any given medical issue, or know which procedures or protocols have the best results (unless they send someone to dig through the files and summarize the data), or that they are required to provide the data by the rules established for clinical trials by the FDA. I am starting to feel that if your doctor cannot tell you what his survival rates are, you would be better served by finding a myeloma institution that understands the importance of measuring the success of their program. In business and in life "YOU CAN NOT MANAGE WHAT YOU DO NOT MEASURE!"
Would you buy a car that had no speedometer? Or how about buying a new car that has no window sticker that tells the miles per gallon you can expect. Measures are important in our everyday life. Without a ruler, how long is anything that you want to measure? Without measures how repeatable will your favorite recipe be?. Business uses profits, quality measures, customer service statistics and much more to measure success. Because they know if they do not deliver a quality product that will last, and at a reasonable cost, the customer will find out and they will soon no longer be in business. So where am I going with this?
When I was first diagnosed with multiple myeloma, and because I had a business and engineering background I had assumed that I would be able to find who had the best myeloma program by obtaining some basic measurement statistics from all of the myeloma treatment centers. Well I was wrong, and the only two I found were UAMS (University of Arkansas for Medical Sciences) and Mayo Clinic. I was shocked, so I decided to see if I could help the myeloma patient community by providing more survival information. I have been doing this now for over 2 years and have added 10 more to the list. But even those that have wanted to provide information have trouble because they can not collect the data from their data processing departments. In other words, the care providers do not collect or measure and evaluate the basic data that would help them to determine where their program is currently and help them to determine ways to improve and move the programs forward. A very helpful reader, scientist, and myeloma advocate was trying to help me get his doctor and their program listed on my site and after months and months of followup by him and myself with his care provider he finally wrote me back with the following information, "I checked in with Dr. XXXXXXXX at XXXX. today. Their filing/data retrieval system is a mess. They don't have any retrievable data greater than 5 years old. They have the goal of setting themselves up as a Center of excellence in the Midwest. I told them how beneficial being in your system could be. They agree but complained that they don't have anyone available to enter the data. Not impressed." This type of response is one I have received more times than I had ever thought possible. This just seemed to be such a basic data need. The other disappointing response I have heard is that, "Our institutional rules do not allow us to provide this information". I am not the only one that has noticed this failure in our medical system, an article by Sharon Begley of Newsweek states that, "most cancer centers covet their survival statistics as if they were state secrets." I do have to congratulate the many selfless and humanitarian myeloma professionals who have been forthright with their performance numbers. All of the Doctors and facilities listed on my website have gone above and beyond the call of duty to provide me with their survival information and in all cases have survival statistics well above the averages provided by the National Cancer Institute. This helps to prove that "YOU CAN MANAGE THAT WHICH YOU MEASURE!" And this all brings me to the title of this blog post.
Nobody has dared to take on this huge task for the entire medical profession and establish what most in the medical profession have chosen not to provide, "MEASURES OF PERFORMANCE". And who has been so audacious as to do this? Who else but Consumer Reports. They were the ones who took on the auto industry and provided independent testing and reliability ratings that have revolutionized the auto industry and provided the best and most reliable cars ever built. Would this have ever been accomplished without the measurements from Consumer Reports which we all take for granted today when we look to buy a car? The auto industry has always used Mad Men (advertising) to provide us with their one sided view of their products. They made the Vega sound like a great car! I have purchased most of my cars based on the Consumer Reports recommendations, and many other products as well.
Without measurement the same holds true today for the medical industry, where reputation is usually the surrogate for a hospital's success. Reputation or marketing is used, not performance data when patients choose a hospital. UAMS, Dr. Berenson's IMBCR, and Dr. Hari of the Medical College of Wisconsin have the best survival data, but I would say that Dana Farber and M.D. Anderson have the reputation. Blessed with exceptional myeloma specialists like Dr. Richardson and Dr. Orlwoski, I would expect that Dana Farber and M.D. Anderson would have great survival statistics, but I have no data at all to confirm this for any of these highly regarded institutions. Why don't they publish their survival data? I do not know. Is it just not a hospital administration priority? What can be a greater PRIORITY than giving your clinical staff the tools to improve performance and SAVE LIFE? My assumption is that these two reputable myeloma care centers use the data generated from their extensive involvement in clinical trials as a substitute for performance measures. Few institutions, however, have the resources ($15000 per patient) and skills required to be heavily invested in clinical trials. And what I have found is that it is not the administration, but the doctor who ends up fighting to get the data compiled, and this is with his clinical load and all of their other duties. With their 80 hour clinical loads, and with this additional responsibility, these doctors are my HEROES.
So what is Consumer Reports up to? In 2008 they announced the formation of the Consumer Reports Health Ratings Center and the addition of Dr. John Santa, M.D., M.P.H., to direct the new center. Consumer Reports seeks to add ratings for hospitals and other health-care providers to its ratings of health related products, treatment options, natural medicines, and drugs. Most recently they have put together a listing of surgical ratings for 2463 hospitals. There were a number of surprises in the ratings. It is the first unbiased, independent measurement I know of which determines how successful hospitals were for many elective surgeries. Just one of hundreds of articles on the subject can be read if you CLICK HERE. If you would like to view the ratings you can CLICK HERE. What does this have to do with Cancer and specifically Multiple Myeloma?
I was honored to be able to talk with Dr. John Santa and found out that he is currently in the process of working through the details as to how to obtain the survival data for cancers, or other measures which may be applicable. The same process Consumer Reports went through when they developed the surgery ratings. I got the impression from Dr. Santa that he is convinced that it is a question of when, not if, these cancer performance measures will be published. A key element in his methodology is to contact medical professionals, registries, medical associations, and groups to obtain input to refine the process so that they provide measures that are unbiased and represent a level playing field for all participants. In my discussion with Dr. Santa he was kind enough to answer a few questions and the Q & A follows:
Question 1 - I have seen your excellent work in hospital performance for surgery, and would like to commend you for this excellent work. It seems that there has been some blow back from some of the low performers, which is to be expected, however what is your read on this? Sour grapes or some valid concerns that can lead to improvements.
Dr. Santa - We value all reactions or suggestions. It is common for industry organizations that do not do well to have critical comments though there are also many poor performers who focus on improving and appreciate the feedback. We also get lots of requests from organizations that do well to use the information in some way to promote their work. We remind those folks that, in contrast to other ratings, our ratings cannot be used in advertisements or promotions. The science of measuring and reporting health performance is in its infancy. For us it is a “journey, not a destination.” Lots of work to do.
Question 2 - Are you looking at providing health comparisons for cancers?
Dr. Santa - We published ratings for cancer screening tests in early 2013 and received a very positive reaction from our readers and from the media. In the Choosing Wisely campaign that we are involved in as the lead consumer partner, we have been thrilled to work with the American Society of Clinical Oncology and collaborate on cancer content with them. So we are looking for more opportunities to write about and also rate services or products that relate to screening, diagnosis or treatment. We have had some conversations with organizations that have data on performance around cancer care but so far we have not ended up pursuing those opportunities.
Question 3 - For stage 4 cancers, and incurable cancers like multiple myeloma, I have found in my research that a skilled professional can mean the difference of life or death. For example a multiple myeloma specialist can have a life expectancy of 2 to 4 times that of the average hematologist/oncologist. Will you be putting together a rating for these cancers?
Dr. Santa - We are constantly looking for good information that compares anything in health. The information needs to come from a credible source that is hopefully independent of industry---not influenced by drug companies or hospitals for example. The information needs to be available to the public or such that we can make it available to the public. It needs to be current. And it is important that the information be supported by credible authorities in the field it covers. This can be much more difficult than it might seem. In the case of multiple myeloma for example, how would we figure out who are multiple myeloma specialists? And where would the data come from to evaluate their work? And how would it be independently validated? And if we did this would the information be updated regularly into the future?
Question 4 - What method are your looking at to create this ranking?
Dr. Santa - There are multiple sources of information about cancer and we are trying to learn about them. While some of the information is public, much of it is not. And much of the information that is public is not at the level of a hospital or a doctor so it does not work for comparisons of the sort we do. And invariably the methods used to collect the information may use different definitions and methods than other information sources used. So it can be a bit like the Tower of Babel---lots of folks talking but not much understanding.
Question 5 -From a Google search I have found that one of the reasons that there is question about your surgical ratings is that some 60% of complications happen after a patient has been released. From a layman’s view and understanding statistics, a sample that represents 40% of the population should be more than adequate to represent the mean with exceptional accuracy. Do you see it that way as well or is this another possible measure of the competence of the hospital? The higher the readmission rate would equate to a show of very poor performance.
Dr. Santa - Performance in a hospital during a single stay is usually the best and easiest first step in terms of valid reporting of data about what happened. Once a patient leaves a hospital many more factors come into play and recording them accurately can be difficult. For example the patient may be discharged to a different health system whose information is not easily available or may use different definitions than another. Our sense is that if we can get reasonably accurate information about 40-50% of what has happened that can be of help to consumers. When it is possible to add more information we do so. You have to start somewhere, otherwise you never start. Consumers are smarter than they are given credit for. Our experience is that when we describe the limits of our work consumers note those limitations. They use our information as one of many sources in their decision. That’s the way it should be.
Question 6 - I have heard that some people feel that patients do not want to know this information, that it would confuse them, or that this would reflect poorly on the low performers and that would limit cooperation. It is my belief that the first point is not valid, and may be because the patient is uninformed of the difference in performance, or they have come to trust what their doctor says as “The Truth”. What are your thoughts on this subject? The second one just has no validity for doctors who believe in the “Hippocratic Oath”. How do you read this potential issue?
Dr. Santa - We are fortunate to be living in an era when diversity and differences are honored. People are different and those differences should be honored. For example surveys show that there are some people who want to know as much as they can before they make a health decision while others are prepared to do whatever their doctors tell them to do. I respect both of those approaches. Many of the first types of folks are our readers. And our intention is to give them the best information we can. I must admit some impatience with folks who think that the solution to preventing confusion is to not provide any information at all. Think about any learning process you have moved through in your life. I remember being confused about reading, writing and arithmetic……….but I remember more fondly when I figured each out and the world that was now available to me as a result. All learning involves confusion---it is a good sign, it means people are wading in, taking risks and that’s good not bad.
I am especially concerned about those that urge we suppress information in order to avoid “scaring” doctors/patients or penalizing poorly performing health systems. This thinking has unfortunately been prevalent in many circles for the last decade. I think it has slowed down improvement, creates distrust and harms patients. I think good health care needs to start with transparency. How can a patient trust a doctor or hospital that keeps something important a secret from them? We need to stop treating patients and consumers like children. These are the most important decisions in our lives and both doctor and patient deserve to know everything the other knows and everything that could possibly be important. If a patient does not want to know the details they just need to say so.
You cannot improve what you don’t measure. You cannot compare unless you have common definitions and approaches. And you lose trust from consumers and patients if you are not transparent and share this information with them.
Dr. Santa, thank you so much for your work and continued efforts of Consumer Reports to provide patient outcome data that will help patients and the medical profession make INFORMED care and treatment decisions. I hope and pray you find a way to provide comparisons for hospitals and doctors who now treat cancer patients. I believe that together we will SAVE LIFE, and maybe my own or yours!
For more information on multiple myeloma go to the site www.myelomasurvival.com or you can follow me on twitter at: https://twitter.com/grpetersen1
Would you buy a car that had no speedometer? Or how about buying a new car that has no window sticker that tells the miles per gallon you can expect. Measures are important in our everyday life. Without a ruler, how long is anything that you want to measure? Without measures how repeatable will your favorite recipe be?. Business uses profits, quality measures, customer service statistics and much more to measure success. Because they know if they do not deliver a quality product that will last, and at a reasonable cost, the customer will find out and they will soon no longer be in business. So where am I going with this?
When I was first diagnosed with multiple myeloma, and because I had a business and engineering background I had assumed that I would be able to find who had the best myeloma program by obtaining some basic measurement statistics from all of the myeloma treatment centers. Well I was wrong, and the only two I found were UAMS (University of Arkansas for Medical Sciences) and Mayo Clinic. I was shocked, so I decided to see if I could help the myeloma patient community by providing more survival information. I have been doing this now for over 2 years and have added 10 more to the list. But even those that have wanted to provide information have trouble because they can not collect the data from their data processing departments. In other words, the care providers do not collect or measure and evaluate the basic data that would help them to determine where their program is currently and help them to determine ways to improve and move the programs forward. A very helpful reader, scientist, and myeloma advocate was trying to help me get his doctor and their program listed on my site and after months and months of followup by him and myself with his care provider he finally wrote me back with the following information, "I checked in with Dr. XXXXXXXX at XXXX. today. Their filing/data retrieval system is a mess. They don't have any retrievable data greater than 5 years old. They have the goal of setting themselves up as a Center of excellence in the Midwest. I told them how beneficial being in your system could be. They agree but complained that they don't have anyone available to enter the data. Not impressed." This type of response is one I have received more times than I had ever thought possible. This just seemed to be such a basic data need. The other disappointing response I have heard is that, "Our institutional rules do not allow us to provide this information". I am not the only one that has noticed this failure in our medical system, an article by Sharon Begley of Newsweek states that, "most cancer centers covet their survival statistics as if they were state secrets." I do have to congratulate the many selfless and humanitarian myeloma professionals who have been forthright with their performance numbers. All of the Doctors and facilities listed on my website have gone above and beyond the call of duty to provide me with their survival information and in all cases have survival statistics well above the averages provided by the National Cancer Institute. This helps to prove that "YOU CAN MANAGE THAT WHICH YOU MEASURE!" And this all brings me to the title of this blog post.
Nobody has dared to take on this huge task for the entire medical profession and establish what most in the medical profession have chosen not to provide, "MEASURES OF PERFORMANCE". And who has been so audacious as to do this? Who else but Consumer Reports. They were the ones who took on the auto industry and provided independent testing and reliability ratings that have revolutionized the auto industry and provided the best and most reliable cars ever built. Would this have ever been accomplished without the measurements from Consumer Reports which we all take for granted today when we look to buy a car? The auto industry has always used Mad Men (advertising) to provide us with their one sided view of their products. They made the Vega sound like a great car! I have purchased most of my cars based on the Consumer Reports recommendations, and many other products as well.
Without measurement the same holds true today for the medical industry, where reputation is usually the surrogate for a hospital's success. Reputation or marketing is used, not performance data when patients choose a hospital. UAMS, Dr. Berenson's IMBCR, and Dr. Hari of the Medical College of Wisconsin have the best survival data, but I would say that Dana Farber and M.D. Anderson have the reputation. Blessed with exceptional myeloma specialists like Dr. Richardson and Dr. Orlwoski, I would expect that Dana Farber and M.D. Anderson would have great survival statistics, but I have no data at all to confirm this for any of these highly regarded institutions. Why don't they publish their survival data? I do not know. Is it just not a hospital administration priority? What can be a greater PRIORITY than giving your clinical staff the tools to improve performance and SAVE LIFE? My assumption is that these two reputable myeloma care centers use the data generated from their extensive involvement in clinical trials as a substitute for performance measures. Few institutions, however, have the resources ($15000 per patient) and skills required to be heavily invested in clinical trials. And what I have found is that it is not the administration, but the doctor who ends up fighting to get the data compiled, and this is with his clinical load and all of their other duties. With their 80 hour clinical loads, and with this additional responsibility, these doctors are my HEROES.
So what is Consumer Reports up to? In 2008 they announced the formation of the Consumer Reports Health Ratings Center and the addition of Dr. John Santa, M.D., M.P.H., to direct the new center. Consumer Reports seeks to add ratings for hospitals and other health-care providers to its ratings of health related products, treatment options, natural medicines, and drugs. Most recently they have put together a listing of surgical ratings for 2463 hospitals. There were a number of surprises in the ratings. It is the first unbiased, independent measurement I know of which determines how successful hospitals were for many elective surgeries. Just one of hundreds of articles on the subject can be read if you CLICK HERE. If you would like to view the ratings you can CLICK HERE. What does this have to do with Cancer and specifically Multiple Myeloma?
I was honored to be able to talk with Dr. John Santa and found out that he is currently in the process of working through the details as to how to obtain the survival data for cancers, or other measures which may be applicable. The same process Consumer Reports went through when they developed the surgery ratings. I got the impression from Dr. Santa that he is convinced that it is a question of when, not if, these cancer performance measures will be published. A key element in his methodology is to contact medical professionals, registries, medical associations, and groups to obtain input to refine the process so that they provide measures that are unbiased and represent a level playing field for all participants. In my discussion with Dr. Santa he was kind enough to answer a few questions and the Q & A follows:
Question 1 - I have seen your excellent work in hospital performance for surgery, and would like to commend you for this excellent work. It seems that there has been some blow back from some of the low performers, which is to be expected, however what is your read on this? Sour grapes or some valid concerns that can lead to improvements.
Dr. Santa - We value all reactions or suggestions. It is common for industry organizations that do not do well to have critical comments though there are also many poor performers who focus on improving and appreciate the feedback. We also get lots of requests from organizations that do well to use the information in some way to promote their work. We remind those folks that, in contrast to other ratings, our ratings cannot be used in advertisements or promotions. The science of measuring and reporting health performance is in its infancy. For us it is a “journey, not a destination.” Lots of work to do.
Question 2 - Are you looking at providing health comparisons for cancers?
Dr. Santa - We published ratings for cancer screening tests in early 2013 and received a very positive reaction from our readers and from the media. In the Choosing Wisely campaign that we are involved in as the lead consumer partner, we have been thrilled to work with the American Society of Clinical Oncology and collaborate on cancer content with them. So we are looking for more opportunities to write about and also rate services or products that relate to screening, diagnosis or treatment. We have had some conversations with organizations that have data on performance around cancer care but so far we have not ended up pursuing those opportunities.
Question 3 - For stage 4 cancers, and incurable cancers like multiple myeloma, I have found in my research that a skilled professional can mean the difference of life or death. For example a multiple myeloma specialist can have a life expectancy of 2 to 4 times that of the average hematologist/oncologist. Will you be putting together a rating for these cancers?
Dr. Santa - We are constantly looking for good information that compares anything in health. The information needs to come from a credible source that is hopefully independent of industry---not influenced by drug companies or hospitals for example. The information needs to be available to the public or such that we can make it available to the public. It needs to be current. And it is important that the information be supported by credible authorities in the field it covers. This can be much more difficult than it might seem. In the case of multiple myeloma for example, how would we figure out who are multiple myeloma specialists? And where would the data come from to evaluate their work? And how would it be independently validated? And if we did this would the information be updated regularly into the future?
Question 4 - What method are your looking at to create this ranking?
Dr. Santa - There are multiple sources of information about cancer and we are trying to learn about them. While some of the information is public, much of it is not. And much of the information that is public is not at the level of a hospital or a doctor so it does not work for comparisons of the sort we do. And invariably the methods used to collect the information may use different definitions and methods than other information sources used. So it can be a bit like the Tower of Babel---lots of folks talking but not much understanding.
Question 5 -From a Google search I have found that one of the reasons that there is question about your surgical ratings is that some 60% of complications happen after a patient has been released. From a layman’s view and understanding statistics, a sample that represents 40% of the population should be more than adequate to represent the mean with exceptional accuracy. Do you see it that way as well or is this another possible measure of the competence of the hospital? The higher the readmission rate would equate to a show of very poor performance.
Dr. Santa - Performance in a hospital during a single stay is usually the best and easiest first step in terms of valid reporting of data about what happened. Once a patient leaves a hospital many more factors come into play and recording them accurately can be difficult. For example the patient may be discharged to a different health system whose information is not easily available or may use different definitions than another. Our sense is that if we can get reasonably accurate information about 40-50% of what has happened that can be of help to consumers. When it is possible to add more information we do so. You have to start somewhere, otherwise you never start. Consumers are smarter than they are given credit for. Our experience is that when we describe the limits of our work consumers note those limitations. They use our information as one of many sources in their decision. That’s the way it should be.
Question 6 - I have heard that some people feel that patients do not want to know this information, that it would confuse them, or that this would reflect poorly on the low performers and that would limit cooperation. It is my belief that the first point is not valid, and may be because the patient is uninformed of the difference in performance, or they have come to trust what their doctor says as “The Truth”. What are your thoughts on this subject? The second one just has no validity for doctors who believe in the “Hippocratic Oath”. How do you read this potential issue?
Dr. Santa - We are fortunate to be living in an era when diversity and differences are honored. People are different and those differences should be honored. For example surveys show that there are some people who want to know as much as they can before they make a health decision while others are prepared to do whatever their doctors tell them to do. I respect both of those approaches. Many of the first types of folks are our readers. And our intention is to give them the best information we can. I must admit some impatience with folks who think that the solution to preventing confusion is to not provide any information at all. Think about any learning process you have moved through in your life. I remember being confused about reading, writing and arithmetic……….but I remember more fondly when I figured each out and the world that was now available to me as a result. All learning involves confusion---it is a good sign, it means people are wading in, taking risks and that’s good not bad.
I am especially concerned about those that urge we suppress information in order to avoid “scaring” doctors/patients or penalizing poorly performing health systems. This thinking has unfortunately been prevalent in many circles for the last decade. I think it has slowed down improvement, creates distrust and harms patients. I think good health care needs to start with transparency. How can a patient trust a doctor or hospital that keeps something important a secret from them? We need to stop treating patients and consumers like children. These are the most important decisions in our lives and both doctor and patient deserve to know everything the other knows and everything that could possibly be important. If a patient does not want to know the details they just need to say so.
You cannot improve what you don’t measure. You cannot compare unless you have common definitions and approaches. And you lose trust from consumers and patients if you are not transparent and share this information with them.
Dr. Santa, thank you so much for your work and continued efforts of Consumer Reports to provide patient outcome data that will help patients and the medical profession make INFORMED care and treatment decisions. I hope and pray you find a way to provide comparisons for hospitals and doctors who now treat cancer patients. I believe that together we will SAVE LIFE, and maybe my own or yours!
For more information on multiple myeloma go to the site www.myelomasurvival.com or you can follow me on twitter at: https://twitter.com/grpetersen1
RSS Feed