Experiments in knowledge-intensive system architecting: Interactive architecture optimization

Large aerospace organizations typically spend significant resources in optimizing system design by means of specialized software (e.g., computer-aided design, simulation). Conversely, architectural decisions are made much faster, with far less resources, and typically, in a much less structured way. This is despite clear indications that architectural decisions fix many design decisions, and have a larger impact on lifecycle cost and performance than design decisions. The reason for this is that the architecting process is a much harder problem than the design process, due to large and varied sources of uncertainty, and humans usually perform much better than machines in this kind of ill-posed problems. In fact, many attempts to automate the architecture trade-space exploration process have failed due to these difficulties. Thus, current architecting practices are in the two extremes of the human-machine task allocation continuum. They either put too much weight on the human, who is subject to bias and computational limitations, or too much weight on the machine, which lacks the ability to make complex judgments based on common sense and prior knowledge. A more optimized task allocation that synergistically exploits the advantages of humans and computers is needed for system architecting. We call this new paradigm “knowledge-intensive system architecting”. A recent effort in this direction was the demonstration of the use of rule-based systems to improve the architecture evaluation process in the context of automatic trade-space exploration. This is an example of using expert knowledge to improve the performance of an essentially automated process (with the goal of supporting a currently unstructured 100% human process). In this paper, opportunities for synergistic human-machine collaboration in architecture optimization, as opposed to architecture evaluation, are discussed. We compare the outcomes of a multi-objective architecture optimization proc- ss between a 100% automatic process represented by a genetic algorithm with generic operators (crossover and mutation) and a hybrid process where human input is used to guide the search. In a first experiment, the expert knowledge is introduced in the form of heuristics (e.g., add synergistic instrument, remove interfering instrument). We compare the performance of these knowledge-intensive heuristics with that of simple crossover, and random search (as a control). In a more interactive second experiment, the human is presented with a few architectures during the search process and is asked to propose an improved version of each architecture. We compare the performance of the search with and without the human input. The examples used for these experiments concern the architecting process of Earth observing satellite systems, for which the authors have developed a system architecture toolkit throughout the last five years.