Chapter 3: Process Development and Optimization

This issue contains a process development tool that is difficult
to grasp and use, but it is so important that I feel I must explain
it as simply as I can. The tool is so powerful that most industries
have embedded it in their business methods. A few companies - Motorola,
DuPont, General Electric, Boeing and Honeywell - have found this
tool to be a great success enabler. We live in high tech times where
some products have failure rates of less than one in a billion or
even a trillion. This vital technique is one any venturer can adopt-after
some practice.

Consider an invention assembled by a handyman from available modules
in a novel container. Battery driven, it must operate unattended
for long periods and meet government regulations. It's a real improvement.
The crunch comes with ensuring such performance in production. Factors
involved are materials supplied by others, design of the process
and product to be manufactured and execution needed to make it all
happen. Hunches and anecdotes cannot suffice. I will illustrate this
tool using a consistency problem commonly encountered in all three
factors above.

Our work begins with supplier selection and creating guarantees.
Seven manufacturers make the modules at a range of prices that may
or may not reflect their performance levels and reliabilities. Two
of these make sensors; three make smart batteries while two make
the mechanical item. The smart batteries differ in size, shape and
mode of operation. There are 12 configurations, but since the method
is general, I can illustrate how the tool works by selecting a battery
supplier and guarantee.

To take the market, we must certify that our product will meet new
highs in performance. The fundamental question is "What can
we certify, at what level of performance, and at what cost?" The
following trip is not for the faint of heart. But I will keep it
simple.

Our first step is to obtain quotes from each battery supplier along
with their performance guarantees. The quotes indicate that improved
performance comes at a price. After comparing this information with
what we know about the market, we tentatively conclude that a battery
life of at least 12 months is needed to take the market at a price
premium of 10%. All three venders assert they can meet the 12-month
guarantee.

Now assume we assemble 36 devices for testing under field conditions.
We will use these to test 12 batteries from each supplier, A, B,
and C. We should be able to detect differences in battery performance
(average life) and reliability (consistency). Battery-life results
are given in tables 1 and 2.

Table
1 Battery test results and summary

Type
| S/N | Mos. | Type | S/N | Mos. | Type
| S/N | Mos. |

A | 1 | 13 | A | 5 | 13 | A | 9 | 13 |

A | 2 | 13 | A | 6 | 14 | A | 10 | 11 |

A | 3 | 14 | A | 7 | 13 | A | 11 | 13 |

A | 4 | 12 | A | 8 | 12 | A | 12 | 12 |

B | 1 | 14 | B | 5 | 3 | B | 9 | 14 |

B | 2 | 12 | B | 6 | 15 | B | 10 | 12 |

B | 3 | 15 | B | 7 | 11 | B | 11 | 17 |

B | 4 | 10 | B | 8 | 12 | B | 12 | 18 |

C | 1 | 14 | C | 5 | 15 | C | 9 | 15 |

C | 2 | 15 | C | 6 | 18 | C | 10 | 13 |

C | 3 | 17 | C | 7 | 14 | C | 11 | 15 |

C | 4 | 14 | C | 8 | 13 | C | 12 | 13 |

Table 2 Performance summary

Supplier | A | B | C |

Avg. Months | 12.8 | 12.8 | 14.7 |

Range* | 11-14 | 3-18 | 13-18 |

*Min-max (Consistency)

All three battery types averaged above our target life. However,
batteries from suppliers A & B failed to exceed our goal essentially
half of the time while those from C always exceeded 12 months. Supplier
B produced a grossly defective part, lasting only three months. Supplier
C is preferred. Our hunch is that failures will be rare and that
we can guarantee our product for 12 months. But how rare is "rare"?
Do we proceed with our hunch or do we determine how safe we are?
Let's revisit at what we know.

From our research on the product, we know that current battery consistency
(or range = max - min) is (9 - 7) = 2. This compares with ranges
of 3, 15 and 5 for suppliers A, B & C. Back when battery life
was four months, the range was only 3.5 to 4.5-pretty narrow and
consistent. Maybe scatter increases with performance. Is 12 months
as safe as we thought, given the small number of batteries tested?
How can we know?

This is where the tool alluded to enters. When throwing darts, our
darts land on or near target most of the time. Our skill can be characterized
by two quantities: our average hit distance from bull's eye and the
size of our "pattern". Mathematicians have determined that
most measurable events have averages and "patterns" similar
to what we find throwing darts. And they have figured out how to
calculate a quantity known as the Standard Deviation, SD, descriptive
of our pattern. SD is universally true although it may take different
forms. To keep things simple, I will use the dart-throwing pattern
and apply that to the question of battery life.

SD is illustrated by the bell curve shown in Figure 1. In the central
region, its curvature is downward. On each wing, the bell curve levels
out such that the curvature turns upward. The point at which the
curvature changes from down to up is the SD. Pause here and estimate
SD by the rule above. A second method: draw a line parallel to the
base 40% down from the peak-the line will intersect the bell curve
at the SD points. Figure 1 was scanned in from a well-known text
on quality control. Did the author really understand SD? Even experts
can err-demand proof, always. The original was schematic, right?

Figure 1 showing % of events occurring per SD away from the average
(central peak).

The SD gives us a shortcut way to estimate failure rates from test
data. For example, 16% of the time, individual batteries test more
than one standard deviation below the lot average. 2% of the time,
they test more than two Standard Deviations below the average. These
percentages are shown in Figure 1.

By drawing bell curves for our data, the points of curvature change
can be estimated. To illustrate, I did this for our data and drew
arrows in Figure 2 for each supplier that show the SD locations.
My estimate of the SD for supplier A is 0.9, that for B is about
3.9, and for C it is 1.6. Your estimates might vary slightly from
mine, but that would not change our conclusions.

Figure 2 SD points for battery suppliers

To determine if a guarantee of 12 is realistic we first calculate
where our 16% failure level is for supplier C. Those batteries averaged
14.7 months with a SD of 1.6 months. The 16% failure level (about
one in six) is thus 14.7 - 1.6 = 13.1 months. At two SD below the
average, 14.7 - 3.2 = 11.5 months, we can expect a 2% failure rate
(one in 50). So our guarantee of 12 will have a failure rate of maybe
5%. A guarantee of 10 months seems very safe and is still 25% better
than the current average. If that is enough to take the market at
prices reflecting the added cost, we should proceed with a 10-month
guarantee. But if a highly consistent (say not more than one failure
in 500) guarantee for 12 months is emphatically needed to move the
market our way, we can ask supplier C to reduce variability below
SD=0.9 or improve average performance or both. We can also ask the
sensor suppliers to reduce battery drain.

I have only considered battery life. Analyzing the other sources
of product variation may impact our guarantee in like manner. This
is one reason why moderately complex innovations are so hard to sell.
Qualifying market-worthy product can be expensive.

Without considering SD, we might well have set our guarantee at
12 and wondered why one in 20 of our prized inventions suffered battery
failure to the detriment of our market and bank account. Most developments
are less complex. But if you can grasp how to use your new tool,
you can avoid innumerable commercial problems. Understanding data
is the single most critical issue in process and product development.

If you can look at a bell curve and "know" by that inspection
where to set your guarantee, you can do away with hunches and join
the elite. Modern competition demands nothing less.

Congratulations, if you are still with me, this one is tough.

Next issue I will explore the important discipline of forecasting.

For further reading I suggest Statistical Methods for Quality Improvement,
Thomas Ryan. Wiley, ISBN 0-471-84337-7

© Copyright 2000 by Harry Rosenberg. All rights reserved.

> back to top