Fundamentals of systems engineering

Episode 6. What does systems engineering have to do with AI fundamentals? In this episode, the team discusses what data and computer science as professions can learn from systems engineering, and how the methods and mindset of the latter can boost the quality of AI-based innovations.

Show notes

News and episode commentary 0:03

ChatGPT usage is down for the second straight month.
The importance of understanding the data and how it affects the quality of synthetic data for non-tabular use cases like text. (Episode 5, Synthetic data)
Business decisions. The 2012 case of Target using algorithms in their advertising. (CIO, June 2023)

Systems engineering thinking. 3:45

The difference between building algorithms and building models, and building systems.
The term systems engineering came from Bell Labs in the 1940s, and came into its own with the NASA Apollo program.
A system is a way of looking at the world. There's emergent behavior, complex interactions and relationships between data.
AI systems and ML systems are often distant from the expertise of people who do systems engineering.

Learning the hard way. 9:25

Systems engineering is about doing things the hard way, learning the physical sciences, math and how things work.
What else can be learned from the Apollo program.
Developing a system, and how important it is to align the importance of criticality and safety of the project.
Systems engineering is often associated incorrectly with waterfall in software engineering,

What is a safer model to build? 14:26

What is a safer model, and how is systems engineering going to fit in with this world?
The data science hacker culture can be counterintuitive to this approach
For example, actuaries have a professional code of ethics and a set way that they learn.

Step back and review your model. 18:26

Peer review your model and see if they can break it and stress-test it. Build monitoring around knowing where the fault points are and also talk to business leaders.
Be careful about the other impacts that can have on the business or externally on the people who start using it.
Marketing this type of engineering as robustness of the model, identifying what it is good at and what it's bad at, and that in itself can be a piece of selling.
Systems thinking gives a chance to create lasting models and lasting systems, not just models.

How can you think of modeling as a system? 23:23

Andrew shares his thoughts on the importance of thinking holistically about the problem and creating a consistent, consistent, and reliable model.
Traceability and understanding of the system is the secret weapon. Understanding what tools from the box were used at which time, and the impact that it will have on either your customers or the decisions that your business makes on behalf of your customers.

‍