“Testing” is a term that covers a vast range of activities. Not every test is a measurement; some are visual inspections or can be qualified with nominal descriptions such as “pass or fail” or “hot or cold”. However, for tests that either are measurements or include measurements, understanding the uncertainty of those measurements is important to interpreting resulting data. An important component of test method validation is an uncertainty analysis that systematically assesses the factors influencing the measurement.
Instruments and the process of calibration to reference standards are not perfect. Neither is the person performing the test, the environment, or the test method itself. Therefore, every measurement has some uncertainty associated with it. The use of recognized standard procedures reduces many potential sources of measurement uncertainty, however even when the method is standardized, the uncertainty of the method should be understood.
A measurement is a function of all the input quantities that affect the measurement. For example, for the measurement of volume there might be three measurements (length, width, height). The uncertainty in the volume measurement can be determined by calculating and combining each measurement uncertainty along with the uncertainty associated with other factors (e.g., temperature or hydration).
Measurement uncertainty is a statistical estimate of experimental error. Thus, when estimating the uncertainty of a measurement, uncertainty components which are of importance in the given situation should be taken into account using appropriate statistics.
When a test method validation includes an uncertainty analysis to systematically assess the factors influencing the measurement, use of the test method for inappropriate applications can be avoided. The test method may also be improved, controlled, or monitored in areas of high uncertainty, ultimately building more confidence into the associated results.
For more information, please visit our website at medinstitute.com.
Test method validation is a documented process that is used to confirm that the procedure to be employed for a specific test is suitable for its intended purpose. The validation of a test method by a laboratory should be a planned activity. Validation plans should be updated as development proceeds. The plan should include a description of the method and describe the various validation activities.
When planning a test method validation, consider the scope of the test method and the associated risk (link to previous blog) before the necessary validation activities are selected. The scope of the method is defined in terms of the method’s purpose and the breadth of medical devices that the method is intended for. The range and accuracy of the test method should be adequate for the test method’s intended purpose. The activities selected should be a balance between the rigor needed for a credible validation, breadth of applicability of the test method, and associated effort.
A validation should establish that the test method has:
- Accuracy within the needed range
A test method that has adequate accuracy will have instruments that are calibrated to reference standards and will have compared well to other methods or compared well in an inter-laboratory study.
A test method that is repeatable will yield successive measurements in close agreement under tightly controlled conditions within the same laboratory.
A test method that is reproducible will yield measurements in close agreement from different laboratories or within the same laboratory over a longer period of time.
A test method that is robust will demonstrate minimal sensitivity to external factors, such as operator skill or changes in ambient conditions.
Validated test methods, used appropriately, provide data that can be relied upon to make important decisions related to the verification and validation of medical devices with confidence.
To have a more thorough discussion or to answer any questions you may have regarding the test method validation process please contact MED Institute via our website at http://www.medinstitute.com
The medical device industry has long understood the requirements related to process and software validation, however, US FDA Title 21 Code of Federal Regulations Part 820 Quality System Regulations does not have explicit requirements related to test method validation. Despite the lack of requirements, the FDA does have expectations around test method validation and has issued 483s and warning letters for insufficient method validation activities. Test methods that have not been validated yield a risk of the method being inadequate for evaluating medical devices and can lead to failed clinical devices, market recall and patient harm. Thus, to satisfy FDA expectations and avoid the pitfalls of using an inadequate test method, test methods used to produce data in support of regulatory filings or the manufacture of medical devices for human use need to be validated.
21 CFR 820.3 states: Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled. Method validation is a documented process that is used to confirm that the procedure to be employed for a specific test is suitable for its intended purpose. Test method validation gives an overall understanding of uncertainty of the method. A validated method provides confidence that the method is appropriate and that the data generated are reliable and repeatable.
When establishing test method validation activities, first consider the purpose and risk associated with the test. ISO 17025 states: The validation shall be as extensive as is necessary to meet the needs of the given application or field of application. The extent of validation activities required for a given method is driven by its intended purpose and closely related to the risk of patient harm. For example, a test for conformity of appearance may not require the same level of validation activities as a durability test replicating the clinical usage of the device.
The level of validation should be commensurate with the risk associated with the test method being wrong.
Test method developers should determine the risk associated with the test method by evaluating the severity of potential harm and the probability that the harm occurs. With the purpose of the method and the associated risk assessed, the validation activities and level of rigor needed may be planned.
As an example, consider a test method to measure the length of medical device that is 100 cm +/- 1 cm long. The severity and probability associated with the device length being incorrect is low. Thus when planning the test method validation activities associated with a metal ruler, uncertainty determination including accuracy, resolution and repeatability is needed. However determination of the uncertainty related to the thermal expansion of a metal ruler isn’t likely warranted.
A deeper look at validation activities including uncertainty determination will be covered in future blog posts.
For more information, please visit our website at http://www.medinstitute.com
On September 20th Justin Renfrow, Director of Business Operations, participated in the 2016 MedTech Policy Forum. This was a joint effort between the Indiana Medical Device Manufacturers Council (IMDMC) and the Indiana Health Industry Forum (IHIF). Justin participated on a panel with Paul Moses, Purdue Research Foundation, and Kyle Hultgren, Imagine Medical Devices. The panel was chaired by Brooke Beier, Assistant Director for Business Development for Purdue’s Office of Technology Commercialization. The panel focused on the collaborative efforts needed to take a medical device to market. Participants discussed success stories and a few of the lessons learned along the way. In regards to the event, Justin stated, “IMDMC does an excellent job connecting thought leaders from around the state to share their experiences and to collaborate. I’m always proud to be involved.”
To learn more about how MED can help you, visit our website.
Last month, MED Institute participated on the ‘Too Many Cooks in the Kitchen’ Kickball team as part of the 4th annual Purdue Research Park Kickball Tournament. 3 other Cook companies participated on the team: Cook Advanced Technologies, Cook Research Incorporated and Cook Regentec. In total, the team played in 4 games throughout the evening scoring a total of 52 runs over the course of the evening! We held our opponents to only 6! This is the second year that the Cook team has won the kickball championship game. It’s really great that so many Cook companies can come together as one for events like this. We are excited and proud to take home the trophy for the second year in a row!
In part five of this series, we discussed forming an industry consortium as another means to develop a test method. In this part, we’ll discuss what you can do when the standards community disagrees with your proposed method.
One course of action available to you is to promote awareness of test methodology by publication in peer-reviewed journals. The development of a publication portfolio can help to validate the methods you have developed and will allow industry peers to confirm your practices. This can help everyone come to a consensus on the best way to move forward.
This concludes our series on what you should do when a testing standard doesn’t exist. To continue the discussion or to answer any questions you may have, please contact MED Institute via our website at http://www.medinstitute.com
We ended our 2016 tradeshow season in Minneapolis, MN at MD&M Minneapolis – September 21 and 22nd. It was a nice change of pace to have this show a little earlier in the fall- no snow or cold weather had hit Minnesota yet! We also had a great two-day show. Once again, we featured a TA ElectroForce 3100 fatigue tester as part of our booth to show off one of our numerous testing capabilities. Last year we completed over 3 billion cycles with our fatigue testers for a single client! We also had opportunities to discuss other capabilities including R&D, simulation, regulatory, feasibility clinical trials and many other services we offer.
Additionally, Justin Metcalf, our Director of Engineering, gave a Tech Talk while at the show on Design Verification Testing: What Should You Do When a Testing Standard Doesn’t Exist? Justin discussed the importance of being involved with standards committees to establish a new standard test method or modify an existing method. He also discussed developing a new non-standard test method and how to mitigate risk with test method validation. This particular topic was a very popular one- it was standing room only! Over 35 people were in attendance during his presentation! Overall, we were really happy with our booth, project discussions and tech talk success at the show. Feel free to follow up with us if you’re interested in receiving more information on the presentation given. See you next year in Minneapolis!
In part four of this series, we talked about employing a third party to build testing equipment to develop a new test method. In this part, we’ll discuss forming an industry consortium as another means to develop a test method.
Through the use of Cooperative Research and Development Agreements, an industry consortium can be formed. This can be a useful means of obtaining and sharing data that are expensive to collect.
Collaboration through an industry consortium can result in test method development, new instrumentation, and innovative approaches to shared problems. The knowledge and data developed in the industry consortium can be used to educate others and to support standard development.
For example, in 2006, SRI International, Stanford University, and a consortium of stent manufacturers teamed together to investigate loads on stents in the superficial femoral artery (SFA) and to improve the durability of peripheral vascular implants. An important outcome of this collaborative effort was the identification of bending, torsion, and axial deformations that would need to be simulated in the in vivo loading experienced by the stents. The results suggested possible fracture mechanisms and also provided important parameters for future stent design.
In the final part of this series, we’ll review the options you have when a testing standard doesn’t exist and discuss your recourse if the standards community disagrees with the testing method that you propose.
For more information, please visit our website at http://www.medinstitute.com
I’m Bill Voorhees. I have been employed by MED Institute for over 25 years, although I have been associated with the company from its inception, serving on its advisory board from 1983 until being hired on in 1990. Prior to coming to MED, I spent 14 years at the Biomedical Engineering Center at Purdue (4 years as a doctoral student and 10 years as Research Faculty).
It was a natural progression to come to MED. At the Biomed Center, we did a lot of practical, applied physiology research. At MED we take the next step and help bring the fruits of such research to the patient. Our role in the process comprises all aspects of product validation testing including non‑clinical in vitro and in vivo testing as well as clinical trials. I have to admit I may be most pleased with being involved in helping conduct the first clinical trial, analyzing the resulting data, and preparing the regulatory submissions that led to FDA’s approval of the first coronary stent available in the U.S., The Gianturco-Roubin Flex‑Stent Coronary Stent. Those were exciting times, and there are more to come as we offer our expertise in all aspects of new product approval to a new set of clients learning to navigate the medical product approval process.
To learn more about our services, please visit our website medinstitute.com.
I’m Colleen Tennyson and I’ve been with MED for over 31 years! I feel very blessed for all of the opportunities provided to me and am eternally grateful for the chance to serve in various roles over the years, including finance, quality, HR, and compliance.
Both MED and I have changed a lot since 1984. I was single, had no children and no degree. Now I’m married with children, grandchildren and great grandchildren and have a degree in business. MED did only electronics development and worked primarily for sister Cook companies. I watched us grow from 4 employees to over 240 and back to 24 as we have branched off in order to provide a full range of services from research to publication to clients other than Cook.
And some things never change. MED was dedicated then to serving patients first and that is still true. The message has remained consistent and clear – – quality first – – always. Few companies have that privilege.
It has been a wild ride, and a lot of fun! I’m looking forward to the future.
To learn about our services, please visit our website www.medinstitute.com