Optimization is the process of making a design as efficient as possible, whether guided by human intuition or a computer algorithm. But to see how it shapes engineering, we first need to break design down to its two core questions:

1. Problem Definition — What am I trying to solve?

2. Problem Optimization — What is the best possible solution?

Problem Definition

The first question defines the objectives and constraints a valid solution must satisfy. These are rarely obvious from the start. In practice, they emerge and sharpen as our understanding of the problem deepens. Design is therefore an inherently dynamic process: the problem and its optimal solution evolve together, continuously shaping each other.

Even with a clear, well-defined design brief, there is usually an endless pool of potential solutions — and some will naturally outperform others. Consequently, a design’s success hinges primarily on how accurately the problem is framed. Optimizing a design that answers the wrong question serves little purpose; it is far better to start with an appropriate formulation and a suboptimal design, as the solution can always be refined in a subsequent design phase, but it gets increasingly harder to switch concepts as the project progresses.

The primary limiting factor in any design endeavour is time. The more time is available, the more alternatives can be tested — allowing engineers to either explore a broader range of concepts or further refine existing solutions. In reality, deadlines always get the final say. At some point, the designer must embrace a classic engineering concept — the “good enough” — and halt further tuning. This highlights one of the true arts of this trade: balancing performance against finite resources. Therefore, in order to explore more alternatives within a fixed schedule, the only path forward is reducing iteration time. While faster numerical solvers help, the real key lies in building smoother and more efficient workflows.

Beyond time, the quality of a design also heavily depends on the expressiveness and accuracy of the tools used. Flexible tools allow to explore a much broader design space, unlocking solutions that may otherwise remain hidden. Simultaneously, the accuracy of designs hinge on simulation fidelity and their underlying assumptions — meaning different models will yield different “optimal” results. Ultimately, it falls on the designer to recognize the limits of their workflow and keep expectations grounded in reality.

Problem Optimization

At its core, an optimizer is simply a sophisticated iterator — an algorithm that takes informed steps toward a best-fit solution. For well-posed problems, a designer can delegate that second core question (“What is the best solution?”) directly to the optimizer. The designer then shifts into the role of a guide, steering the search using prior experience, domain knowledge, and a growing understanding of the problem.

It is important to understand that the optimizer are tools of execution, not judgment and that their purpose is to boost designer capabilities but is never intended to replace them. If you ask the wrong questions, even the most advanced optimizer will happily give you the wrong answers. The true value of an optimization-based design approach does not thus reside in automating the search for new solutions but in forcing designers to rigorously formalize their intent, ideas and assumptions.

Physics Exploitation and The Curse of Dimensionality

Despite their power, optimizers are far from infallible. When relying on them for non-trivial design tasks, two major challenges frequently arise:

1. Physics Exploitation — The algorithm cheats the simulation model.

2. The Curse of Dimensionality — The design space grows faster than our ability to search it.

Physics exploitation stems from the fact that an optimizer navigates an n-dimensional design space where model accuracy is rarely uniform. Without proper constraints, the algorithm will ruthlessly exploit any loophole in the physical or numerical model — producing a design that is mathematically “optimal” yet physically absurd or impossible to manufacture.

This creates a delicate balancing act; the design space must be constrained enough to enforce physical reality but remain open enough for the algorithm to discover truly novel concepts.

The curse of dimensionality describes the exponential growth in computational effort — both time and memory — required to explore a design space as its degrees of freedom (DoFs) increase. Each added variable expands the space, forcing the optimizer to perform significantly more function evaluations just to decide its next step. For gradient-based optimizers, this means calculating gradients for every objective and constraint function at every single iteration — a process typically achieved by constructing the Jacobian matrix. As the number of variables grows, evaluating this matrix quickly becomes the primary computational bottleneck.

This creates another fundamental trade-off; a richer parameterization offers greater design freedom and unlocks potentially superior solutions, but it also makes the search space exponentially harder and more expensive (sometimes impossible) to navigate. A good parameterization must strike a balance. It should be flexible enough to capture the full design space while keeping the number of independent variables as lean as possible.

Constrained Optimization

At first glance, one might think that the objective is the most important trait of an optimization setup. In reality, constraints are often far more critical. The objective provides the algorithm with a general descent direction, while the constraints define what is physically and practically possible.

In real-world engineering, constraints are rarely set in stone — they are assumptions about stress limits, operating conditions and flow features among many others. By observing the sensibility of the design to these constraints, a great deal of insights on the key performance drivers can be extracted. This realization loops us straight back to question one (“What am I trying to solve?”). It proves that optimization isn’t just about finding answers, but rather about increasing problem understanding.

Automatic Differentiation

To overcome the prohibitive computational cost of gradient evaluations, modern workflows increasingly rely on automatic differentiation (AD). AD computes derivatives of a computer program by systematically applying the chain rule to the sequence of elementary operations that constitute its computational graph. Unlike finite-difference methods, which approximate gradients by perturbing design variables and repeatedly re-evaluating the model, AD propagates derivative information through the program to obtain derivatives that are accurate to machine precision.

This enables large-scale design optimization to remain computationally tractable. In particular, AD can exploit the structure of the computational graph to evaluate gradients at a cost that is largely independent of the number of design variables, making feasible optimizations with hundreds or even thousands of variables.

Conclusion

Optimization in engineering design is a constant balancing act between expressive freedom and computational feasibility. Securing a good enough design means weighing model accuracy against simulation speed, setting reasonable boundary condition and structuring efficient parametric models.

Hopefully, we can now all conclude that there is no universally optimal design. An optimum exists only within the context of a particular objective, set of constraints, parameterization, model, and operating conditions and more importantly… the designer itself. Change any of these, and the optimal solution will change with them. The success of a design ultimately depends on the rigor and precision with which the engineer formulates that problem.