They’re Better Together

Thanks so much for joining us as we look at how to more effectively Coordinate QC Lab and Manufacturing Schedules. As you know QC Labs are a critical part of manufacturing operations and also supply chains today, providing a large number of different tests of many different types are to support quality throughout the process. Two of the most significant test types you can see at the top are In-Process Control Tests or Tests performed during a manufacturing operation, and an example of this is viable cell density, as well as intermediate testing, things like testing buffers and media, as well as things like WIFI and other ingredients that go into the process in order to ensure quality.

Part of the problem of QC Labs is essentially the balancing act between all of the test needs of the facility and the priority of the ordering of those tests, and we’ll be looking at that in quite a lot of detail today. One of the other things, and I think this is less well known in the industry, is that In-Process Testing, so the first two types that we saw in intermediate testing, are some of the most significant hidden bottlenecks in pharmaceutical manufacturing. Today when we think about bottlenecks or things that would slow down the manufacturing process if they took longer, we think about large pieces of equipment, things like production fermenters, chromatography columns.

IPC testing and intermediate testing is one of the examples of where we have to pause the process to obtain an intermediate result, a quality result, and the process can’t continue until that result is obtained, and today most capacity analysis looks only at the equipment but not the capability of the lab to support these types of test needs. Now this problem is exacerbated by the fact that many pharma companies today are only focused on scheduling manufacturing or QC lab as two separate things and essentially provide coordination only on the activities that overlap, and this coordination is often manual, it’s often phone calls or emails, or it’s semi-automated, some type of a nightly data dump. Now the impact of this is pretty significant for both teams, and it does affect overall site operations, and you can see here a few categories we’ll be talking about today, poor visibility, different priorities, late breaking changes, and overall org structure.

So let’s go through these one by one. The first issue that we see with QC and Manufacturing labs running separate schedules is that there’s poor cross-team visibility about exactly what the other team is doing. You can see here results from interviews conducted in June, July, and August this year, where we looked at the interaction frequency, in this case on the y-axis. How often do people communicate around these issues, and communication content, what was the volume of the communication.

Most pharma manufacturing today is characterized by bare minimum communication content and interaction frequency, in our estimation less than daily communication frequency, and only about the tests that specifically are required in order for manufacturing operations to continue. Now this of course is a very significant issue, because it means that the amount of information that teams have to make decisions on is very low. The second issue is really that manufacturing and QC teams have very different needs from a schedule, and you can see here on the left-hand side we’ve shown different categories of difference, in this case objectives and priorities, equipment, process flow, and operators, and there’s a few critical differences as we see them.

The first is that QC lab schedules are driven by due dates, so when we’re trying to optimize performance we’re really trying to minimize how late we are versus that due date. This is a subtle but important difference from Manufacturing Scheduling, where we’re trying to maximize throughput and minimize rework. And one of the other, I think, real challenges with a QC lab schedule is that we’re trying to group together tests to minimize the total number of setups that we need to have, so there’s sort of some additional complexity associated with a lab schedule where we’re trying to sort of group samples and batches together to minimize the total number of steps that we have.

The second significant thing here is islands of automation around QC lab schedules. If you go into a QC lab you’ll see that there’s typically very sophisticated individual pieces of equipment, but fewer connected systems where there is automated monitoring of batches as they go through a kind of a series of tests. Now this of course is related to the next step, process flow, and part of this is because in a QC lab you can actually perform these tests for a single batch in any order based on priorities and availability.

This of course contrasts the manufacturing schedule where there’s a very specific sequence of steps that have to happen to a batch in order to process it and make a drug substance or drug product. The final sort of significant difference, and I think this is a very real one for many of you that have operated in either QC Labs or scheduling and manufacturing, is that there’s a lot more tasks that an individual operator will be asked to perform as part of their typical day, and there’s very specific training requirements for individual pieces of equipment. This contrasts manufacturing scheduling, typically shop floor operations for manufacturing.

You’ll have a team, a small team, that’s assigned to a process area, let’s say several unit operations, and they receive more generalist training that allows them to operate most of the equipment within that process area. So these very real differences mean that there’s often a kind of an issue in the intersection between these two processes, and traditionally it has been thought that you need two different scheduling systems in order to really meet the different priorities of these two parts of the manufacturing network. So the next issue we’d like to talk about is Sample Testing, and you can see here that often late breaking changes are a cause of very significant problems or headaches for manufacturers.

At the top we’re looking at a manufacturing schedule for a sample by a manufacturing operation with fermentation, harvest, sampling, and then the sending of a particular sample to QC to have it tested. In this case you’ll see that there’s a manufacturing notification that happens in the middle of the fermentation process and will trigger a sequence of preps for standards and systems. In this particular case you’ll see that the IPC test has started on time, but there’s a delay in getting the results of that, and because of course this is on the critical path, we need the results from this test in order to continue into downstream, the subsequent protein A, cation exchange, anion exchange purification steps will be delayed.

Now this can be just the delay period at least, but it can also be more significant than that if we see clean hold time expiries, product expiries, or even if this delay causes us to move the start of the protein A chromatography into a graveyard shift for example, where we have fewer operators available to process that particular batch. The final sort of challenge as we see it is that QC and manufacturing organizations often operate at arm’s length from one another. You can see sort of three different sort of versions of this.

The first on the left hand side is that operational reporting structures are quite separate between manufacturing and quality groups up to quite a high level, typically the head of the site, and so you’ll see that the reporting structure and therefore the incentive alignment right for those individuals is really focused on either quality or manufacturing with very little reporting systems in place to ensure overall kind of efficiencies. And what this means of course is that because we have these two different groups and they’re often collecting different metrics or KPIs, they’ll tend to be some more finger pointing, right, where we’re reporting advantages in one group at the expense of another. And you can see an example here where manufacturing’s like, hey we couldn’t get a test back in time so we had a delay, and QC’s like, well you didn’t even give us a heads up about the test and actually it arrived three hours late.

So there’s kind of a lot of back and forth today around these types of issues and no common reporting. The final point, and I think this is a really important one, is around cycle time and the amount of time that we allow, the window of time as many people call it, that we allow for these types of tests. Now today this is typically a fixed number, let’s say four hours, and in this histogram we’re showing along the x-axis the amount of time it really takes, right so this is hours, and the proportion of batches that take between that number of hours.

You can see here that there’s often a bimodal distribution for these types of tests where if things go well it’ll take a certain amount of time, that’s typically within the target cycle time window, but if it takes longer then we’ll see often very significant delays as we retest or we do some other kind of response because we’re out of spec. And so one of the obvious questions is, you know, given these challenges, what can we do as an organization to try and improve our coordination between these two very different organizations? Now we think that one of the first and most critical things that you can do is to augment or potentially replace this concept of a time window where you give a fixed amount of time for QC tests and move to what we call a real-time prediction methodology. What does that mean? So what it means from the manufacturing side is that we’re going to start making predictions of the when the sample is going to arrive a lot much larger number of days out, typically seven or more days out, so that QC can start prioritizing its own internal testing and its own internal priorities amongst the very large number of tests that it has to do in that seven-day period.

The second thing we’re going to be doing is refreshing predictions regularly. And this is not just a prediction based on someone’s opinion but based on a sophisticated machine learning algorithm or something like that, that’s looking at past data and learning the different types of patterns of behavior and giving minute-by-minute updates around when that batch or when that sample is going to arrive into the QC lab during the shift. And the final piece is QC push notifications.

So this is similar, think about this similar to being like a notification you might get for a delayed flight. You certainly want to know that your flight is going to be delayed if it’s a long period of time before you leave the house, right? And so the idea of receiving in advance push notifications when there are an issue or where we expect a delay longer than a certain amount is very critical to managing the QC lab and their effective prioritization and execution of these tests. On the quality side, real-time predictions mean more accurate start and stop tracking for batches and lots, and not just the lots themselves but also all of the individual steps that have to happen inside those lots, and a realistic base and downside estimate.

So capturing this bimodal distribution, what happens if the test is going to go normally? What time do we expect it to come back into manufacturing? And then if and when we see a downside estimate so that the manufacturing will know if we do kind of go down that road of having to do some retesting, what does that mean? And again, manufacturing push notifications are critical so that we have advance warning as soon as possible for manufacturing so that these critical handoff points can be effectively managed. The second major thing that we think can be done and provides very significant benefits for the overall operations of a facility is to build out and maintain a QC dispatch board as part of scheduling. Now what a QC dispatch board does is it allows individual operators, you can see here Audrey Hepburn and George Peppard, and they have a list of the set of things that they’re going to be doing in their shift, and this list of things includes at the top the very next thing that they’re going to do.

Notice that it includes both the testing activities, so the actual test itself, as well as system checks, reviews, and sign-offs, and what this means, this can update during the day, right, and what this means is that anytime as a technician I’m about to start a new step, right, I can go in and see what that new step should be, and I can get minute-by-minute accuracy of when samples are going to arrive, and also kind of provide updates myself to the system around, you know, when I expect to finish things. This is tremendously powerful, and this, when integrated with a scheduling platform like smart scheduling, allows us to very quickly respond to delays and see those delays come in, bubble the effect of those delays back into manufacturing. The third is around the increasing need to provide visibility into operations, and as we discussed earlier, I think most folks today are operating with the bare minimum of interaction frequency, right, mostly daily kind of communication, and the bare minimum of communication content, and we both mean by this not just sending a small amount of data, but sending it in a system that’s very difficult to read, right, like a spreadsheet or other type of system.

So really, in order to improve here, we need to move to real-time updates, to push-based alerting, and we need to have better types of content that we’re sharing around delays, operator availability, for example, indicating if an operator in an area is going to be off today, and then that in turn may mean that there’s more stress points on that particular manufacturing process. We very much need as an industry to move towards a joint reporting and KPI framework in this particular part of the manufacturing process, and today, as we mentioned earlier, this is very siloed and leads to a lot of finger pointing, and one of the things that I think is a necessary precursor to effective operational excellence in this area is to actually have that kind of realistic joint set of dashboards around what’s happening, what are the causes for delays, and this allows us to drive efficiencies and to prioritize improvement projects around these particular areas. And finally, one of the things that we think is most significant around the platform or the software that we’re using is to align scheduling platforms into a single more integrated scheduling platform.

Now, there’s many ways to do this. Many of you have an existing system that’s being used, let’s say, for Manufacturing or for QC, and it’s very difficult to rip that one out and put something new in, and so there’s many different approaches here that we think are really valuable. The first is a kind of a separate but tightly coupled system where you maintain the existing system of record, but there’s essentially full and real-time visibility for that particular schedule into the other system of record that’s doing the scheduling. Right.

This, in turn, allows us to respond optimally when we see a delay and in real time. The second flavor that we’ve often seen is that we have a single system, so the system itself, a Manufacturing Scheduling System, is a single system, but we have two different operators, we may have two different reporting systems, and we see completely different views of that schedule. So this really allows independent approaches, it allows us to do different things in different teams, but it does optimize across both teams to produce a best schedule that balances QC Labs and also Manufacturing needs. And the final, and I think best-in-class solution here, is a single integrated scheduling platform, and again, this doesn’t mean that everybody has to be seeing the same thing, it doesn’t mean that everybody has the same visibility into the manufacturing floor or into QC, but it provides the simplest possible approach to really optimize and report out our findings in an enterprise-wide fashion. Now, for teams that have implemented this style of approach where they’ve moved from two very distinct scheduling styles and different flavors to a single integrated schedule, we do see some very, very significant changes to manufacturing operations. The first is we see a relatively significant increase in throughput, and this of course is due to the fact that those In-Process Testing operations are actually bottleneck steps, right, so they’re rate limiting for our facility. If we can sort out those steps and reduce the frequency of overage in terms of the amount of time that they take, we can directly increase the throughput of that facility. There’s also very significant reductions in lead time that are potentially possible, and a very significant decrease in the number of frequency of batch delays that we see as a result of being able to more effectively coordinate between these two teams. So there’s very significant improvements that are possible by looking at a kind of a more integrated approach to QC Lab and Manufacturing Schedules.

We’d be happy to talk with you in more detail about this. Thanks so much for joining us.