The Importance of Intersocietal Commission for the Accreditation of Vascular Laboratories (ICAVL) Certification for Noninvasive Peripheral Vascular Tests: The Tennessee Experience
Scientific Articles: "The Journal for Vascular Ultrasound 28(2):65-69, 2004 by David G. Stanley, MD, FACS

ABSTRACT
Purpose
Accurate, safe and cost-effective non-invasive vascular tests (NIVTs) upon which vascular specialists can confidently base recommendations for treatment decisions are critically important for optimal patient care.

Vascular surgeons are referred patients from many sources. These patients often present with NIVTs performed at a variety of facilities. A vascular specialist must be aware of the accuracy of NIVTs from his vascular lab as well as those sent from other facilities in order to make correct treatment recommendations. Without this knowledge, the consultant must either repeat the NIVT in a vascular lab with known quality assurance (QA) statistics or order arteriograms or other tests.

Methods
In order to determine the accuracy of studies from other facilities, the Vascular Diagnostic Center of Oak Ridge (VDC) and the vascular surgeons that practice at the Methodist Medical Center of Oak Ridge (MMC) developed an ongoing quality assurance program to address the quality and accuracy of NIVTs from other facilities. Since 1985, duplex vascular studies from other vascular laboratories, hospitals, and physician’s offices have been carefully compared with NIVTs performed at the VDC. We also compared all studies from VDC and other facilities to arteriograms and to pathology demonstrated in the operating room at MMC in patients who underwent surgery.

Conclusion
In our experience, studies from ICAVL accredited laboratories were accurate in 83% of reviewed cases, whereas studies from facilities without ICAVL accreditation generated much lower correlation to quality and accuracy indices. This information is clearly valuable to physicians, patients, and third party payers.

Introduction
In order to diagnose and treat peripheral vascular disease effectively, inexpensive, accurate, reproducible studies with no patient morbidity or mortality are desired. Duplex color flow imaging and waveform analysis meets all these criteria if properly done.

Oak Ridge Surgeons, P.C. (ORS), a group of five vascular surgeons on the medical staff of Methodist Medical Center of Oak Ridge (MMC), interpret the NIVTs performed by the VDC, an ICAVL certified independent diagnostic testing facility (IDTF) performing over 8,000 NIVTs per year. There are approximately two hundred and fifty thousand patients in our referral area. We rely on non-invasive studies performed by the VDC for our non-invasive vascular tests. ORS vascular surgeons perform approximately 1,000 open or endovascular procedures each year including approximately 200 carotid endarterectomies per year. Since our surgeons rely on the VDC for accurate information and schedule surgery based on NIVTs, we have maintained a very strict quality assurance program since 1985.

Arteriograms were performed on every patient considered for possible carotid endarterectomy from 1985 to 1992. The NIVTs were compared to arteriograms at quarterly QA meetings with VDC surgeons and vascular ultrasound technologists. During that time, payment for carotid endarterectomies was denied by Medicare unless arteriograms also demonstrated significant pathology. This Medicare policy allowed VDC to compile valuable QA statistics for carotid duplex scans. In February 1992, we reported our statistics in the Journal of the Tennessee Medical Association (Table I). This report demonstrated that VDC duplex scans were as accurate as MMC arteriograms for carotid stenosis.

In 1992, the Medicare local medical review policy (LMRP) was amended to approve payment for carotid endarterectomy based on either duplex scan or arteriograms. Many vascular surgeons, including our group, began doing carotid surgery based on NIVTs.

We continue to have QA meetings on our NIVT accuracy and report them to MMC and our medical staff on an annual basis.

During regularly scheduled QA meetings, the medical and technical staff review the monthly results and carefully evaluate false positive and false negative studies to determine the cause of the error and, if possible, to formulate strategies to avoid similar problems in the future. The non-invasive study reports, non-invasive videotapes, arteriography reports, arteriography films, and operative reports are available for review. Discussion focuses on whether the correct clinical decision was made as a result of the carotid ultrasound study.

Following the QA meetings, the data are compiled for statistical analysis as outlined in a later section. The methodology offers precise and statistically valid measurement. The QA reports are reviewed to collect data for the number of true positive, false positive, false negative, and true negative results using two separate gold standards. The first gold standard is arteriography, to which the duplex on the operative and non-operative sides are separately compared. The second gold standard (on the operative side) is the operating room measurement of the plaque specimen by graded dilators and calipers at the time of removal, to which the duplex and arteriography are separately compared.

To be considered a positive finding, the test has to reveal clinically significant stenosis of >60%. To be considered a negative finding, the test has to reveal stenosis of <60%. To be considered a false positive result, the test has to be read as >60% stenosis and be >+10% off the gold standard measurement. To be considered a false negative result, the test has to be read as <60% stenosis and be <-10% off the gold standard measurement. For valid comparison, the non-invasive and arteriography studies must be performed within three months of each other (except for 99% stenosis versus occlusion, which must be performed within one month).

The data are then displayed in 2 X 2 dichotomous diagnostic decision matrices for ease of manipulation. This yields four matrices. Then the sensitivity, specificity, positive predictive value, negative predictive value, and overall accuracy of the different methodologies are calculated. Sensitivity indicates the percentage of tests that correctly diagnose that disease is present. Specificity indicates the percentage of test that correctly diagnose that disease is not present. The positive predictive value indicates how likely it is that disease is present with a positive test result. The negative predictive value indicates how likely it is that disease is present with a negative test result. Table II shows sample 2 X 2 decision matrices and sensitivity, specificity, and positive and negative predictive values over a ten-year period. Table III shows an example of an annual QA report.

Patients are referred to our vascular surgeons from many sources. These patients often arrive with carotid duplex ultrasound studies from community hospitals, IDTFs, physician’s offices, and mobile vascular testing sites. We found that many of these studies were not complete or accurate. Patients referred with NIVTs for consideration of carotid endarterectomy received duplex studies based on reports of severe vascular disease.

Errors in NIVTs included: Less severe disease than indicated, external carotid artery stenosis misdiagnosed as internal carotid artery stenosis, omission of hard data such as peak systolic and diastolic velocities, no mention of plaque morphology or waveform changes, and no vertebral artery or subclavian artery evaluation. Examples of insufficient data included reports consisting of only one or two sentences stating that the patient was noted to have “carotid artery disease” and arteriograms were always recommended. We documented facilities that had acceptable protocols and accurate reports as well as those with limited or inaccurate studies. Including documented accuracy of vascular tests from other facilities as part of our QA has been very helpful in evaluating and treating patients referred to our vascular surgeons. We were able to consider which NIVTs to repeat and thus avoid inappropriately operating on false positive tests that had indicated severe or critical carotid stenosis. Of course, this QA process cannot detect false negative studies as these patients were not referred to ORS. Therefore, patients with undetected severe carotid disease continue to be at risk of strokes.

Since we had reliable QA data for NIVTs done at VDC, we compiled four hundred thirty-seven (437) studies from twenty-four other diagnostic facilities comparing the NIVTs done at VDC with the other facilities. We found that 62% of the studies from other facilities were in the same diagnostic category as the gold standard of the repeat VDC study or arteriogram. Thirty-eight percent (38%) correlated poorly with the VDC study, which changed the treatment recommendations (Table IV).

We compiled comparison studies from twenty-four other facilities. Accuracy, based on our QA standards, ranged from one hundred percent correlation to zero percent correlation compared to duplex ultrasounds performed at the VDC.

After compiling these statistics, we obtained information as to which facilities were accredited by the Intersocietal Commission for the Accreditation of Vascular Laboratories (ICAVL), which facilities were not accredited by ICAVL but had only Registered Vascular Technologists (RVTs) performing the tests, and which facilities were not accredited nor had only RVTs performing the tests at the twenty-four facilities included in this study.

Statistical analysis of this information revealed that in our referral area, ICAVL accredited facilities correlated 83% of the time with the NIVTs performed at VDC, where facilities without ICAVL accreditation had only 45% positive correlation with VDC studies. The overall positive correlation from all facilities was 62% correct and 38% incorrect (Table IV). Good correlation was considered if a study led to the same treatment decision as the VDC report. Poor correlation was a false positive or false negative study that would have resulted in the vascular surgery specialist making an inappropriate treatment decision based on that study.

We then compiled the cost benefit directly related to accurate NIVTs at a medical center similar to Methodist Medical Center in Oak Ridge where a significant number of carotid endarterectomies are performed annually. We retrospectively analyzed the cost to third party payers (usually Medicare) at MMC in 2001 based on two scenarios: 1) Arteriograms routinely done prior to recommending carotid surgery or medical treatment versus 2) Treatment decisions made based primarily on the carotid duplex evaluation with selective arteriograms in patients with unusual symptoms or inadequate NIVTs.

In reviewing our office records, we found that six hundred thirty (630) patients were referred to our practice in 2001 for consideration of carotid endarterectomy. Four hundred forty-two (442) patients were eliminated from surgical consideration based on duplex ultrasound alone. One hundred eighty-eight (188) patients were selected for surgery based on duplex ultrasound alone and twenty-seven patients required arteriograms in addition to duplex ultrasound for various reasons in order to make the proper decision. This analysis indicates that at our community hospital over $900,000 per year was saved in medical costs due to having accurate, reliable carotid duplex studies (Exhibit A)

Discussion
Vascular surgeons rely on accurate non-invasive studies for clinical decisions. Accurate studies allow vascular specialists to make cost-effective treatment decisions. Inaccurate non-invasive studies have both false negative and false positive findings. False negative NIVTs may result in strokes related to undiscovered vascular disease. False positive tests, if relied upon for treatment decisions, may result in unnecessary surgery or other inappropriate treatment. The routine of ordering arteriograms due to unreliable NIVTs on every patient that may be a candidate for carotid endarterectomy is extremely costly in health care dollars and mortality/morbidity from invasive arteriograms (Exhibit A).

Detailed, documented QA experience of VDC and the vascular surgeons at MMC suggests that ICAVL certification of non-invasive vascular diagnostic facilities in our referral area is associated with accurate NIVTs. Although these comprehensive vascular exams are more expensive to generate, they allow vascular specialists to make treatment decisions, in most cases, based on NIVTs alone. The decreased expense, with less morbidity and mortality, of accurate NIVTs is significant.

In January 2004, Tennessee joined other states, which require facility accreditation or technologist certification as a necessary component for CMS reimbursement for NIVTs.