6 Methods for CAPA Effectiveness Verification

Verifying the effectiveness of corrective and preventive actions closes the loop between identifying a problem and completing the actions to solve a problem. It seems reasonable to expect that if a problem is worth solving, it is also worth verifying that the problem is actually solved. But, determining the best verification approach and deciding when to conduct the verification for the wide range of problems that could occur can be elusive.

Before we discuss CAPA effectiveness, we need to look at a few of the reasons why performing this check is often a challenge.

Why is it so difficult to determine an appropriate CAPA Effectiveness Verification method? Here are a few reasons:

• The problem is not well defined. Sometime breaking the mental logjam is as simple as asking "What problem were we trying to solve?" That sounds like an easy question, but when the answer is not well defined or stated simply, measuring success is not easy to grasp.

• The root cause is not determined. This is a natural consequence of the first reason. It is next to impossible to  determine the root cause for a fuzzy problem, or one that seems too complicated to explain. Those who try also get a fuzzy root cause.

• It's not really a CAPA. It is ever my experience that the CAPA system has become a quality work order system (a.k.a. dumping ground) because the common data managements systems utilized, such as Trackwise, provide project management structure and visibility. But without a stated problem or the determination of a root cause, it is not a CAPA. It's just a project.

• CAPA Effectiveness Verification is used for everything. CAPA Effectiveness Verification can be too much of a good thing when it is expected for every possible CAPA. This usually occurs from the cascading problem of a CAPA being required for every deviation, and a deviation being required for every conceivable blip. Soon you become a drowning victim of your own making.

• We overthink it. Rather than allowing reason to prevail, there are those who tend to complicate just about everything. Determining and applying the effectiveness method is no exception. Yes, we operate in a scientific environment, but not every method of verifying effectiveness has to be labor intensive. Major processes need not be applied to minor problems.

• It's considered not important. There are those who believe that living with an ongoing problem is the path of least resistance when compared to conducting the same boilerplate investigation (same problem different day) and getting on with production. Having a low tolerance for recurring problems is truly the root cause for many who are trending water in a deviation-swirling tide pool. 

Assuming that we have a real CAPA where an investigation was conducted on a well defined problem to determine the root cause and product impact, we can turn to the regulatory requirements and business obligation to evaluate how well we spent our resources to permanently eliminate the problem. This brings us to options for methods for verifying CAPA effectiveness.

What are some examples of CAPA Effectiveness Verification Methods? Here are 6 examples:

• Audit Method is used when the solution involves changes to a system where a determination is made whether changes are in-place procedurally and in-use behaviorally. An example is an audit of a new line clearance checklist to demonstrate effective implementation of the new line clearance checklist.

• Spot Check is used for random observations of performance or review of records provide immediate, but limited feedback. An example is a spot-check of batch records to ensure that the pH step was performed correctly after training on the new procedure.

• Sampling is used for observations of variables or attributes per defined sampling plan. An example of sampling is when a statistical sample is randomly drawn from lot XYZ123 post implementation of process improvement to confirm the absence of defect.

• Monitoring is used for real-time observations over a defined period. An example of monitoring is the real time observation of to verify that changes to operator owning practices were implemented.

• Trend Analysis is the retrospective review of data to verify that expected results were achieved. An example of trend analysis is the review of environmental monitoring (EM) data for the period covering the last 30 batches to show the downward trend in EM excursions due to process improvements.

• Periodic Product Review is a retrospective review at least annually of trends of multiple parameters to confirm the state of control. An example of periodic product review is the review of data after major changes were made to the facility and equipment as part of a process technology upgrade post recall.

Now that we have a real CAPA and selected a method to verify the effectiveness, we need to determine an appropriate timeframe to perform the verification. Timeframes are subjective, but there needs to be a basis for the decision. This brings us to points to consider when determining an appropriate timeframe for the CAPA Effectiveness Verification.

How do we select an appropriate CAPA Effectiveness Verification timeframe? Here are points to consider:

• Less Time. Allow relatively less time after implementing the solution when:

• Higher opportunity for occurrence / observation
• Higher probability of detection
• Engineered solution
• Fewer observations needed for high degree of confidence

• More Time. Allow relatively more time after implementing the solution when:
• Lower opportunity for occurrence/ observation
• Lower probability of detection
• Behavioral/ training solution
• More observations needed for high degree of confidence

The following are several fictitious examples of CAPAs that require an Effectiveness Verification. What CAPA Verification Effectiveness method would you recommend?
What timeframe do you recommend?

Example 1.

Problem:
There are widespread errors in selecting an appropriate effectiveness verification and timeframe in the Trackwise fields when compared to the requirement in the new procedure.
Root Cause:
There is a general lack of understanding of acceptable CAPA Effectiveness Review methods that would satisfy the procedural requirement.
CAPA:
Develop and deliver targeted training on CAPA Effectiveness Verification methods to CAPA system users who have the responsibility to make this determination.

Example 2.

Problem:
Transcription errors are being made when copying information from sample ID labels to laboratory notebooks.
Root Cause:
Labels made on the current label printer (make/ model) are frequently unreadable.
CAPA:
Replace the current label printer with one that produces legible labels.

Example 3.

Problem:
The incorrect number of microbiological plates as required by SOP XYZ123, were delivered to the lab of two separate occasions by a newly trained operator after routine sanitization of Room A.

Root Cause:
The instructions in SOP XYZ123 are more interpretive than intended, which can mislead inexperienced operators to place the correct number of plates in Room A.

CAPA:
Revise SOP XYZ123 to add the specificity required for the correct number and specific placement of micro plates in Room A.

Example 4.

Problem:
Increased bioburden levels were noted in the microfiltration process train A.

Root Cause:
The phosphate buffered saline (PBS) delivery piping system upstream of the microfilter exhibited high bioburden levels.

CAPA:
Revise the cleaning procedure to incorporate a water for injection fish to remove residual harvest material from the process piping and provide training on the flushing process.

Example 5.

Problem:
A statistically significant trend was observed in assay X results for 6 lots of the 25mg vial manufactured at site A, but not the 10mg vial manufactured at site B for the same period.

Root Cause:
There was a difference in sample preparation techniques between the two sites.

CAPA:
Revise the sample preparation of the test method for consistency between sites and provide training on revised test method.

Please share your experiences with CAPA Effectiveness Verification in the comment section below.


The QA Pharm

10 Points to Consider Before Becoming a Pharma Consultant

I am often asked for my advice on entering the consulting field, and I am happy to share my opinion from my own personal experience and what I have observed.

So here are 10 points to consider:

1. Family support is essential. You will be away from home most weeks. If you are someone with a young family; someone who likes to make all the decisions at home; someone who has a partner that has difficulty having you away from home---don't go into consulting.

2. Be financially stable. The consulting world is becoming more competitive and hourly rates are declining in many specialty areas. If the next on-time paycheck is critical for survival; if you cannot be without an income for six months--don't go into consulting.

3. Form your business entity. Don't mix personal and business money. Protect yourself by forming some type of a corporation. Talk to a accounting advisor who can listen to your unique situation to determine the best option for you. I am a "C Corporation," but make your decision on your needs, not what other consultants do.

4. Be realistic with your skills. Determine what your product is and refine it into tools, methods, approach, etc. Clients will readily detect when you have overstated you expertise. While it is acceptable to hone your skills and accrue more experience when consulting, it is not acceptable learn the subject on the clients' dime.

5. Charge by the hour. Consulting is typically billed by the hour, not the job. You have more control. Some clients may never agree when you are done.

6. Show something for your work. Agree upon deliverables with the client. What does "done" look like? Develop a common understanding of the client support required for the project to succeed. Know when you add value or not. When you no longer are adding value and just billing hours, discuss with the client. Warm-body-consultants give consultants a bad name.

7. Market through people who know you. Most projects come from people who know you directly and personally. Hopefully you have been building a network throughout your career from places you have worked and professional conferences. Being out in the consulting world is not a time to start thinking a network.

8. Be a consultant, not an employee. Study the IRS test rules for determining the difference between and employee and an independent consultant/ contractor. Don't allow either yourself or the client to cross the line emotionally or legally. If you feel you are "going to work" like you have all your life, you are not consulting.

9. Keep careful records. Not all clients require time and expense records. You should require it for yourself. Keep every cotton-picking receipts and a meticulous record of your hours: what you do and time. Never, ever pad the expense report or bill unreasonable hours.

10. Build-up, not tear down. You are there to help, not criticize. Avoid coming across as a "know-it-all". Lead the client to the point where they internalize and embrace your teaching. Find those among your client personnel who have that unexplainable spark, mentor them, and leave a legacy.


The QA Pharm

Part 2: Responding to FDA483 Observations and Fulfilling Commitments

Part 2: Organizing to Get the Work Done

Responding to FDA483 observations was my focus in Part 1 of this series. Responses lead to commitments, and commitments lead to changes that are intended to prevent recurrence of the underlying problem that led to the observation.

In Part 2, I share my views on getting the work done to fulfill the FDA commitments. The following have become benchmarks from those who are the best at getting the job done in a sustainable way and getting the problems behind them:

• Form teams. This might seem obvious, but forming cross functional teams representing stakeholders is often overlooked. The solution will very likely involve creating different ways of working that cross functional lines. It is a rookie mistake to think that the fix is something for the Quality Department to do on its own, or just an editorial change to a procedure. Depending on the problem, the team may need to be dedicated to allow focus, particularly if accelerated timelines are involved. Include subject-matter-experts that have the scientific, technical, and cGMP backgrounds to provide appropriate guidance.

• Use consultants judiciously. Consultants can provide valuable additional support. They should bring relevant industry experience that can help scope the project and give practical advice that keeps the project moving toward the solution. The best consultants are good listeners who can help you design the right solution that works for you. But, if the only team members that show up to meetings are the consultants, something is terribly wrong. Do not outsource your responsibility to consultants. Let them help you, but it's your company and you own the results.

• Determine the deliverables. Many teams go off the rails because they fail to determine the tangible deliverables that address the problem. You cannot grasps the degree of effort, determine the resource needs, or estimate the timeline until there is a clear vision of what "done" looks like. If you cannot describe "done," then you do not know the problem well enough.

•  Solve the problem, not just the Commitment. Make a careful assessment of what it will take to prevent recurrence of the observation, not just satisfy the specific commitment. The FDA always states in the FDA483 that the observations are just examples and are not all-inclusive. Companies are expected to determine all that is necessary to permanently resolve the problem that underlies the observation wherever it applies. Perform a root cause analysis and ensure that the solutions are operational and sustainable. That invariably involves more work that the narrow view of a specific example cited in an observation or the response given.

• Determine how you will know the problem is solved. In true fashion of a CAPA, determine how you will measure effectiveness. What will tell you that the solution worked? What will be your first sign that it did not work? Consider whether this is yet another attempt at solving the same old problem. Remember that recurring observations lead to increased levels of enforcement. Be sure you know the problem is truly solved.

• Avoid Information Technology (IT) Solutions. This may sound a bit strange, but in my personal experience, nearly every commitment made in a FDA483 response that involved an IT solution missed the commitment date by light-years. I know that it should not have to be that way. I'm just saying that in my 20 years of consulting experience, that has always been the case. Consider applying an IT solution as a later improvement, not within the commitment time line if at all possible.

• Use Project Management Principles and Tools. Quality System development work is not like a capital project where much of what is needed to estimate the time and cost is known upfront. Quality system and compliance projects are more often like exploratory surgery where a lot is learned once you open it up the patient. You may discover that multiple linking systems are affected after you start looking at the problem more deeply.  Recognize that plans will need to be adjusted along the way. Do not confuse progressive revelation with scope creep. Be adaptable. Establish plans and reports based on the work and target dates, not subjective feel-good dashboards with perpetual dates that slip. Anticipate problems with hitting the date and mitigate them before failure is a foregone conclusion. When you see that the timeline is slipping, ask for help and consider first what it will take to get back on track. Don't just keep moving the target date back as the default solution.

• Provide Infrastructure Support. Remediating compliance problems often require new ways of thinking, not just new ways of working. Compliance problems frequently go hand-in-hand with company culture problems. Progressive companies that recognize the business value of an effective Quality Management System also recognize the opportunity that regulatory inspections provide to examine its culture and values. Change management coaches can help a company adapt to the changes required to support a quality and regulatory compliance culture. Recurring compliance problems are frequently a culture problem that discounts the value of an effective Quality Management System and regulatory compliance to the business. 

• Provide Management Support. Ensure that each project team has a sponsor who is the best fit based on the functional owner of the process being remediated. Clearly identify the role of the sponsor and ensure that it includes removing obstacles such as lack of organization alignment, stalled decisions, team dysfunction, and inadequate budget. Sponsors are the champions who are willing to delve into the level of the working team to provide visible support, rather than sit on the management team and undermine their own folks at project update meetings. Being sponsors of quality system remediation projects provides a unique opportunity for function heads to understand dependent processes across functional lines.

• Establish Governance Oversight. The work associated with fulfilling commitments is tightly connected to managing risk. Thus, it is important to keep site governance councils frequently updated with progress and informed about unacceptable risks. These risks always include the risk of meeting a commitment date made to the FDA. Take commitment dates seriously. Other unacceptable conditions uncovered along the way should also be communicated to the governance body. Establish clear lines of communication and escalation between the team that actually does the work and governance. Learn how to escalate and resolve potential problems quickly. Nothing consumes the timeline as much as reluctant escalation and indecision.

• Verify that Deliverables Solve the Problem. Take an independent look to ensure the work is completed and that the tangible deliverables are in-place and in-use. This is a good role for an independent consultant. It is far better to find out for yourself that the mark was missed than for the FDA to call it to your attention as a repeated observation. Be a healthy skeptic and ask, "How do we know for sure the problem is permanently solved?"

• Revisit Commitments. Although you may have verified that the solution is in-place and in-use, revisit the problem at an appropriate time interval to ensure that the organization has not reverted to the old ways of working.

How have you managed regulatory commitments? What are your success stories? What have been epic failures, and why?

The QA Pharm

Part 1: Responding to FDA483 Observations and Fulfilling Commitments

Part 1: The FDA483 Response

In this two-part series will discuss responding to the FDA483 and getting the work done to fulfill the commitments in a sustainable way.

Unlike a Warning Letter, a response is not required--but highly advisable. It is standard industry practice to respond to FDA483 observations. Responsible companies that want to have a good relationship with the FDA respond to each observation by saying exactly how problems will be addressed.

There is plenty advice available, even from the FDA, on what constitutes a good response. For certain, the difference between a good and a poor response may differentiate how you are viewed by the FDA.

From my experience, the following are points to consider that have worked well when forming a response:

• Do you agree with the response? Stating up front that you either agree or disagree, in part or on the whole, stages the rest of the response. Think twice before you say you completely disagree especially with everything-- because the FDA will always have the regulations, industry good practice, and the facts on their side. But it is quite possible that the FDA investigator got the facts wrong or drew the wrong conclusion. This is why a close-out meeting with investigators at the conclusion of the inspection is important. It is better to straighten out the record at the close-out meeting than in the FDA483 response.

• Defend what you are doing right. The examples of violations cited in observation are just a snapshot in time. Although you may not be proud of what was found, the finding may not be representative of the current state of control. The example may pre-date significant improvements that you have made on your own initiative, although they may be in progress and not fully implemented. Be sure to give yourself credit for such improvements. The last thing you want to do is give the impression that the FDA was the first to call a problem to your attention.

• Take product lot numbers seriously. When the FDA mentions specific lot numbers associated with a problem, it immediately puts you in the position of having to defend why those lots are still on the market, and why you have not shut down manufacturing associated with that product, process, or facility. Put your best minds and writers to work to lay out the defense of your release decision and the state of control. Be objective and use data. If you come to the conclusion, albeit late in the game, that the product should be recalled, seriously consider the right thing to do. Also take into account causing a critical drug shortage, because the FDA will work with you to avoid a shortage while you resolve inspection-related problems.

• Defend State of Control. Some types of observations paint a picture of being out of control, particularly when it is associated with validation, sterility assurance, potency, content uniformity--you know--the SISPQ issues. Also, taken collectively, observations that run the gamut of the quality system can indicate a precarious state of control. Take your best, realistic, honest shot at how you know you are manufacturing and distributing a safe, quality product. And, for goodness sake, do not--I repeat--do not anchor your defense that there have been no complaints or dead bodies in the street. Meeting all end product specifications may be the best defense you have if statistically treated, but be aware that a "final product spec" defense will always peg you as a 1970s time traveler.

• Commit to Action. Address the specific issues that were called to your attention, but do not stop there. Carefully determine the root causes of the issues cited in the observations. There will not always be a one-to-one relationship of root cause to observation. Nonetheless, run the problem to ground and understand what is behind the problem. Determine the action required to prevent the reoccurrence of the problem cited. Undoubtedly, the solution will require changes in multiple areas at the system level. Be sure to convey that you are taking system-level action, not just addressing the superficial, specific examples. Remember, observations are just examples. Indicate the action you promise to take and target completion date. Be clear in advance what "done" looks like and the specific deliverables that will provide the evidence of successful completion.

• Commit to Developing Appropriate Action. If you do not know exactly what all is necessary to fix the problem, it is completely acceptable to indicate that the scope of activities and target dates will be established in a plan to be provided later. But state in the response when the plan will be completed and provided. Major milestones and target month are usually acceptable in a follow-up communication.

• Recognize the Minimalism. Very likely you have taken a minimum essential approach to the commitment. By the time all the internal reviewers have edited the response, you have pared down the work to something manageable for normal business. However, the work associated with FDA inspection observations is not normal business. The scope of the actual work required to prevent recurrence of the observation may be more than expected when you actually delve into the root cause and prevent recurrence through systemic changes. 

• Provide Realistic Target Dates. There is always a balance between showing responsiveness with aggressive target dates and over committing to unrealistic dates. When dealing with operational and quality system problems, there may be more work involved than meets the eye. Some problems may require engineering studies or process development in order to understand the right approach to take when solving a problem. Take time to really think through all that is involved in order to provide realistic target dates. You don't want to go back to the FDA--at least not too often--to revise target dates. Do inform the FDA of changes in completion dates. Don't let a broken promise be a surprise when they return to reinspect. It's far better to inform them in advance.

• Engage Your Internal Associates. I cannot tell you how often I discover the situation where commitments and target dates were given to the FDA in very well written responses, but those who carry out the work were not involved--even totally clueless. It's best to involve those who have to fulfill the commitments in the actual writing of the response. Time is of the essence when preparing a response, but cutting out those who actually do the work will always uncover significant execution issues at the worse possible time.

• Take FDA Suggestions. When the FDA makes a suggestion about retaining a GMP consultant to assess your overall quality system and help you to improve--take the suggestion. This recommendation usually comes in Warning Letters where the FDA has come to the conclusion from the breadth of problems and recurring problems that there is no confidence in your Quality Unit or Management Oversight. Consider such a recommendation as a direct hit and taking on water, not just a shot across the bow. So be responsive and act accordingly.

What is your experience with preparing regulatory observation responses? What has worked well? What has not worked well?

In Part 2 I will talk about organizing to get the work done to fulfill commitments.

The QA Pharm