Experimentation is an essential design activity providing information needed to define requirements, evaluate alternatives, refine the look and feel, establish detailed parameters, tune the operation, and validate the design.  Experimental design is a discipline that supports good design by offering efficient plans, computerized tools for tailoring plans, and strategies for analyzing data emerging from the experiments.  More recently, several areas have offered theory and techniques for extending effective experimentation beyond the traditional approach of pre-defined orthogonal vector spaces of inputs for study.

Recent advances include managing experimentation within adaptive schema – continuously using data from experiments to update plans as evidence is accumulated.   There have also been advances in cognitive experimentation along several diverse directions, whether for system designs that require human decision making, particularly in complex and stressful environments; to experimentation methods that are evolving in attaining meaningful design information on emotional affective responses using Kansei attributes.  There are also advances in concept exploration experimentation, to form experimental sequence strategies of building prototypes, whether developing several concepts in parallel, scaled builds, or focusing only on a subset of requirements.

Recognizing these emerging frontiers of design experimentation, this Design Research Thrust seeks to refine the science of experimentation. We will combine statistics with other disciplines, including psychology, sociology, optimization and controls, to improve experimentation in the context of engineering design. We also recognize exciting potential for more participative modes of design that are enabled by wider availability of information technology and increased capability analyze large data sets.   This will be a practical foundation enabling engineers to decide which approaches are preferred at each phase of the design process.

Specific Research Goals include:

  • Investigate automated methods of natural experimentation and large data gathering from sources such as online sites, social networks, open source communities, massively deployed sensor networks, built infrastructure, and other ICT systems;
  • Investigate improved system characterization and verification though hybrid methods of designed experiments with search based optimization techniques;
  • Investigate designed experimentation with complex spaces beyond orthogonal experimental domains including networks, hierarchies and dynamically evolving spaces;
  • Investigate hybrid investigations of computer data and model based experimentation with laboratory testing and large user base field testing;
  • Investigate the evolution of the experimental test sensors, equipment and instrumentation with the evolution of the experimental plans, for long-term design characterization studies in rapidly evolving ICT infrastructure;
  • Investigate experimental prototyping strategies and planning methods for systems with human interactions and decisions;
  • Investigate early, non-parametric experimental design strategies amongst use of serial and parallel experimental plans of different design concepts, scaled hardware, and other prototyping methods on which to conduct experimentation or designed experiments; and
  • Demonstrate examples of experimental design across a range of fields.