Think Like Richard Bellman

Richard Bellman was an American applied mathematician known for his work on decision making under uncertainty and the invention of dynamic programming in 1953. Here’s what we can learn from his approach to problem-solving and thinking:

  1. Breaking Down Complex Problems: The key idea behind dynamic programming is to break a problem down into simpler sub-problems, solving each of those sub-problems just once, and storing their solutions - ideally, using a memory-based data structure. The next time the same sub-problem occurs, instead of recomputing its solution, one merely looks up the previously computed solution. This principle is widely used in computer science today and is a powerful approach to solving complex problems.

  2. Practical Focus: Bellman’s work was often driven by real-world problems. For example, his work on dynamic programming was motivated by practical problems of resource allocation and decision-making under uncertainty. His approach emphasizes the importance of problem-solving that is firmly rooted in addressing practical needs.

  3. Interdisciplinary Thinking: Bellman worked on a wide range of topics and often crossed disciplinary boundaries. He made contributions to mathematics, computer science, economics, engineering, and even biology. This broad perspective often led to innovative problem-solving approaches.

  4. Simplification: Bellman was a proponent of simplification in mathematical models. He is famously quoted for saying:

A model should be as simple as possible, but no simpler.

This indicates his belief in the balance between simplicity for understanding and complexity for accuracy.

  1. Persistence: Bellman’s success in developing the concept of dynamic programming came out of a determined and systematic exploration of the problem at hand, illustrating the importance of persistence in problem-solving.

By taking inspiration from Bellman, we can learn to break down complex problems, to focus on practical needs, to think broadly across disciplines, to simplify where possible, and to persist in the face of challenging problems.

Simple Model

This phrase often attributed to Richard Bellman, is a derivative of a similar statement made by Albert Einstein:

Everything should be made as simple as possible, but not simpler.

In the context of models - whether they’re mathematical, scientific, computational, or even conceptual - it has profound implications for our approach to problem-solving and understanding the world.

Let’s break down the phrase:

  1. “A model should be as simple as possible” emphasizes the value of simplicity and parsimony. In science and engineering, this principle is often referred to as Occam’s Razor: given two explanations or models that adequately explain the data or phenomena, we should prefer the simpler one.

    The benefits of simpler models include:

    • Easier to understand: Simpler models are generally easier to comprehend and communicate. This can be particularly important when models are used to inform decisions or communicate findings to non-expert audiences.

    • Easier to test: Simpler models typically have fewer parameters or assumptions, making them more straightforward to test empirically.

    • Less likely to overfit: In the context of statistical or machine learning models, simpler models (with fewer parameters) are less likely to overfit the data - that is, to model the random noise in the data rather than the underlying trend or pattern.

  2. “But not simpler” is a crucial caveat. It recognizes that there’s a trade-off between simplicity and accuracy. If we oversimplify a model, we might leave out critical features or interactions, leading to inaccurate predictions or poor decisions.

    The drawbacks of oversimplification include:

    • Loss of essential detail: Models are abstractions of reality, and in the process of simplification, some details are inevitably lost. However, if a model is too simple, it may fail to capture the essence of the problem, leading to misleading or incorrect conclusions.

    • Misrepresentation: If the simplifications imposed on a model do not accurately represent the core dynamics of the system being modeled, the model can lead to poor decisions and incorrect understanding of the system.

Taking the two extremes:

  • An overly complex model could be so intricate that it becomes nearly impossible to understand or interpret, is prone to overfitting, and may not generalize well to new scenarios or data.

  • An overly simplistic model could miss crucial details or dynamics, leading to incorrect or misleading results and potentially poor decisions.

In conclusion, finding the right balance between simplicity and complexity in a model is an essential aspect of effective problem-solving. As Richard Bellman suggested, the model should capture the essential features of the problem or system being studied while avoiding unnecessary complexity. It’s as much an art as it is a science, requiring both theoretical understanding and practical wisdom.