Machine Learning is defined as a computer recognizing patterns without having to be explicitly programmed. With the rise of big data and computing power, more and more companies are creating algorithmic models to make decisions that affect consumers daily. Medical imaging models to help radiologists find cancer, to self-driving cars to prison sentencing to fraud detection, models are changing the world around us. However, these models are not always transparent or intuitive about how they work, which significantly affects trust in the system.
There is a lot of innovation and progress waiting to be released in the form of AI/ML applications in many industries. For example, in Radiology, a classification model can be used to help doctors determine if an X-Ray of a tumor is benign or metastatic. Or in Finance, a classification model could help determine if a customer will default on their loan payments. However, with the lack of built out Machine Learning Assurance capabilities, many of these groundbreaking innovations stay in a sandboxed state. In this blog post, we will introduce the field of Machine Learning Assurance and how it’s principles, combined with the assistance of Monitaur, ML innovation can be unleashed.
Machine Learning Assurance is the process of recording, understanding, verifying, and auditing machine learning models and their transactions. Machine Learning Assurance has Five Tenets, which build upon each other, we will explore:
- End-to-end process understanding
- Objective third-party
In Financial Auditing and SOX auditing, there is an oft-repeated phrase: if it isn’t documented, it didn’t exist. One of the problems that currently plague many AI implementations is the lack of a consistent, comprehensive, and easy to understand log of all inputs and decisions of a model. Without detailed, reliable logs Machine Learning Assurance cannot be obtained.
Logging and Verifiability go hand in hand. Without Logging, Verifiability is not possible. Verifiability is the ability to go back and verify or audit that a given transaction and decision were made. For example, if we see that Sally was assigned a credit limit of $10,000, to have assurance around the models decisioning, we need to be able to back, 6 months later, for instance, and examine the decision that resulted in the credit limit of $10,000. Verify that the inputs and decision make sense are were appropriately performed
A critical part of any audit is the ability to reperform the transaction or decision that was given. For example, in a Financial Audit, it is common for auditors to recalculate the cash inflows and outflows to a critical account. The ability to reperform a model’s decision is complicated because of the feature transformations, model versioning, and application environment and not a feature normally available to auditors. To have full confidence and trust around a decision, being able to rerun the decision and get the same result, to reperform, provides that level of assurance that risk managers, auditors, and regulators need to feel confident that a model is acting as expected.
End-to-end process understanding
To fully understand a process, stakeholders need to understand the full process of where the data comes from, how it is transformed, which model was used, how testing was conducted, how it was deployed, how deployment is being monitored, etc (see CRISP-DM Audit Framework for more details). Documentation from each step of the process, along with accompanying flowcharts, in conjunction with Logging, Verifiability, and Reperformance, breed confidence in the process, that is well-thought-out and controlled.
However much thought and well-intent have gone into a process, mistakes happen and individual or group biases, implicit or explicit, leave their imprint. The mere size of the consulting and auditing spheres attest to the value of an independent, trusted third-party’s fresh eyes to a problem. For assurance, completely independent and objective eyes must be cast upon a high risk / high reward process to ensure that it is performing as intended and does not run afoul of any legal or ethical guidelines. AI/ML will never be fully deployed and trusted until objective third-parties are assured high-risk implementations, such as self-driving cars.
Monitaur supports each of the Five Tenets in enabling AI/ML innovation by providing a comprehensive assurance model management platform that enables recording, understanding, verification, and auditing of your machine learning models. For companies in regulated industries using models to make decisions, Monitaur delivers transparency and auditability that’s necessary to manage compliance and unlock innovation.