The journey of an AI Fundamentalist is to understand the first principles of how to build performant, safe, resilient, and fair systems. One of the core structuring methodologies in a complex view of the world is Systems Engineering. Systems engineering is a cross-disciplinary approach to engineering that highlights aspects of design, development, implementation, integration, and ultimately decommissioning of complex systems through the lens of systems principles. As a field, its roots trace back to Bell Labs in the 1940s as a result of their management of large-scale mission critical projects for the US Department of Defense. Systems engineering was then later refined by NASA in the 1960s, exemplified by the Apollo 11 (and I would personally argue, Apollo 13) mission.
With that history lesson, what exactly is systems engineering, and how does it apply to my life as an AI practitioner? It begins with thinking about the world around you, as complex systems vs constituent objects. Most people, myself included, go through life looking at the world as a collection of independent objects, one **after another. For example, we bemoan the rising prices of gas, without remembering to consider the complex interrelationships of the greater dynamic system we live in, which extends all the way to our global economy. When observed through a macroeconomic lens, the causes of rising gas prices have many contributing and interlocking factors, factors with nonlinear relationship that are greater than the sums of their parts. Without going down too deep of a macroeconomic and geopolitical rabbit hole, the system governing gasoline prices is influenced by objects on both the demand and supply sides of the equation:
These interacting systems all contributed to the supply/demand equilibrium of oil prices. Any one of the above factors, and some not mentioned, are capable of shocking the overall system up or down dramatically. Learning how to take a holistic view of the world and understanding how one action may have large ripple effects due to unforeseen interactions is a guiding principle of systems thinking, and by extension, systems engineering.
To dig deeper into an exploration of systems thinking, read Donella Meadow’s excellent book: Thinking in Systems: A Primer
Now that we have established the world view of complexity vs constituent parts, we’ve unmasked a huge new world of complexity and interdependencies. How do we attempt to build and manage mission critical, high performance and high complexity systems admits this chaos? This is where systems engineering shines. NASA has been a leader in pushing the field further, literally writing one of the main handbooks on the topic (share link). One of NASA’s key takeaways from systems engineering, was for everyone involved to know the top level goal - getting a human to and from the moon - and understanding how their work contributed to that goal and focusing on their specific function. A well engineered systems project is incredibly complex, requiring intensive time and personnel to fully execute. When done properly they allow for successful projects on the scale of the Apollo program, that are absolutely incredible. The harmony of thought and detailed requirements analysis, failure point testing, contingency planning is too much for any one person to wrap their mind around, which is where systems engineering excels.
For our garden variety applications, even for missions critical business applications, we are not realistically encountering the same degree of complexity nor do we have eyes of the world on us. However this does not exempt us from engaging with the key principles and practices of systems driven engineering. The world we live in all to often feels as straightforward as: download data, pip install scikit-learn, and rock & roll. It may be time to take a step back and understand the inner complexities of what we are trying to model
Just because we are using machine learning and the value proposition is learning from the data instead of creating a model to represent a known process, does not mean that it is okay to be ignorant on what we are modeling. After the model is created, evaluate if it makes sense for the business problem.
Another takeaway from systems thinking is the mindset of testing, testing, testing. Understand where the failure points are in your model, if it can handle a stress test, and what it looks like when it breaks. Only when you know where a model breaks do you know where it is safe.
Finally, remember that while objects are simple, systems are complex. When creating mission critical modeling systems, parsimony is key with documentation, plentiful validations, and guard rails. The fanciest algorithm can not make up for a lack of business knowledge or system validation.
As AI-related systems become a more integral part of modern life, let’s all strive to build our systems with inspiration from the core principles of systems engineering. By standing on the shoulders of giants, we can make our AI systems more accurate, fair, and resilient than the fallible human actors they are likely augmenting.
We are saying systems as a model is an algorithm trained on data but isn’t deployed in a vacuum, other effects are present.
Check out our podcast on this topic for a conversational live discussion, and let us know your thoughts, questions, and future episode ideas at theaifundamentalists @ monitaur -dot- ai.
