Design of Experiments Overview (the D in R&D)

Your Research team, the "R" in "R&D," determines what can be done to provide a great product to your customers. Now your Development ("D") teams must figure out how actually to make your great product. Design of Experiments (DOE) techniques allow your development teams to resolve these questions faster, with better solutions, and thereby increase your company profits.

Research tells Development, "To make our product, these factors are likely to be important and these responses should be measured". Development designs and runs experiments to know how to set the factors to get the best responses. ALL factors have to be accounted for- even factors nobody has thought of! DOE tells you the best experiments to run for the factors you want to vary, it has you hold all other known factors constant, and it has you randomize your run order to account for factors nobody has thought of yet. Your work will really produce a model that you can count on. You use this model to determine the best factor settings from the outset, ensuring quality throughout the life of the product.

Exactly what is Design of Experiments?

Design of Experiments tells you which data to collect to get the most information about your product. Sometimes called multivariate analysis, meaning that you are studying many factors, DOE gives you a list of experiments to run. This list varies all your factors at once, which allows you to understand widely how your factors behave and interact with each other. Interactions are all over nature and are not revealed by old One-Factor-At-A-Time experiments.

Technical details: DOE is the first piece of a 2-piece system for regression analysis or curve fitting. The design step specifies the best set of data to fit a particular curve or model. It is best in 2 senses: 1. it is possible to separate the effects of the various causes, and 2. it is efficient. Without a design it may not be possible to determine the effects because they will contaminate each other -- they will be correlated.

Suppose you want to fit data to a line. You can simply start running experiments and start fitting the results to a line. But suppose you aren't very clever and you never run any experiments near the ends of the line. You won't be able to predict anything near the ends of the line unless you are lucky. So your design says "collect data near the ends of the line." This is smart and efficient.

Now suppose you want to fit data to a parabola. You not only need to collect data at the ends, but also in the middle so you can know the extent of the curvature. If you are not so smart, you will just start collecting data wherever you feel like it and, in so doing, waste a lot of time and possibly fit a parabola that can't predict well. A design says "collect data at the ends and in the middle." This is smart and efficient.

With multivariate analysis -- many factors -- you fit a surface in several dimensions. Even a smart person has trouble determining where to collect data for the best fit in this situation. A design tells you where to collect data to be sure to get a good fit and to be efficient about the data collection.

Response Surface Methodology (RSM)-what does this have to do with DOE?

RSM is the system for analyzing the results of your designed experiments. Each response (a measured result or goal) is fitted onto a surface like a map, where you pinpoint the best solution to your project. If you have more than one response goal, RSM will tell you the best combination of your factors to meet all your goals simultaneously. RSM also shows you, like on a topographical map, what will happen to your response(s) if you change the factor settings. This helps you understand how robust your solution is. Will you have wide production tolerances or not when your project goes to manufacturing?

RSM is done with software. RSM gives you your "Sweet Spot," your answer, the best settings for each of your factors to produce your high quality, reproducible product.

Technical details: RSM is the second piece in the system. It fits the data you have collected to a polynomial. This is a surface in many dimensions, one for each factor and one for the response. This is the model. It lets you predict the outcomes of thousands of experiments you have never run.

RSM is not dependent on DOE. You can fit a response surface to any data. However, like the line or parabola above, you will only get a good model if you were lucky. If you collected your data using a design you will know that you have a smart, efficient set of data that will produce the best fit possible.

The Relationship of DOE and RSM to Product Development

DOE plus RSM give you the tools to develop high quality products as quickly as possible. Instead of having to run experiments until something works, you run an efficient set of wisely chosen experiments that will fit to an RSM model. This model will predict where you need to work to develop the best product.

Previously, technical staff would run experiments for every guess they had as to how things should work. Although no-one would admit it, this "hunt and peck" method was very common. When would they know if they could develop this product or the best way to develop it? Who knows?

One-Factor-at-a-Time (OFAT) experimentation replaced "hunt and peck." Staff, using what they learned in college, ran experiments like this: Hold every factor constant except one, vary that factor until you find where it works best, then hold that factor constant and vary another one. Continue until all of the best factor levels are found.

OFAT seems so reasonable that most people will agree that it must work. But it usually doesn't! Why not? It turns out that Nature often works through interactions. It isn't the temperature or the time that is important when baking a cake -- it is setting the right temperature for the baking time that matters (or baking for the right amount of time for the chosen temperature). The OFAT method taught in college prevents you from seeing these interactions. An experiment design LOOKS for them.

Each time you add more factors and/or more responses, your task becomes even more complex. After a certain level, only a designed experiment will produce a genuine "Sweet Spot" in a reasonable time period.

When your development teams use DOE plus RSM:

They will provide a realistic estimate of the amount of work required before any experimentation is started

They will be able to determine the best way to make your product.

They will take all the important interactions into account.

They will even be able to show which of all the possible interactions are important and which are not.

They will be able to do all this work efficiently.