Machine Learning Assurance (MLA) is a controls-based process for ML systems that establishes confidence and verifiability through software and human oversight.
The objective of MLA is to assure interested stakeholders that an ML system is functioning as expected, but in particular, to assure an ML system's transparency, compliance, fairness, safety, and optimal operation.
ML systems present a new paradigm of technology and business decisioning.
Individuals and organizations responsible for managing risk and compliance – including regulators – have not previously contended with such dynamic and variable models and systems.
Because machine learning is making key decisions that affect people's lives, livelihoods, and opportunities, trust and confidence in ML systems is paramount. There should be a reasonable ability to evaluate and verify their safety, fairness, and compliance.
Companies, regulators, and consumers all benefit from an objective method of assuring ML systems.
Three core pillars empower an effective MLA function and responsible use of ML.
MLA requires clear understanding and documentation of the considerations, goals, and risks evaluated during the lifecycle of an ML application.
MLA requires each business and technical decision and step have the ability to be verified and interrogated.
MLA requires any ML application can be reasonably evaluated and understood by an objective individual or party not involved in the model development.
Evaluate key business drivers and questions at project inception and revisit at every following step.
Understand data lineage, quality, and usage rights.
Pre-process standardized data sets for training, test, and production.
Create the simplest, best fit, and performant models.
Determine accuracy and precision of models before launch and continuously in production.
Deliver business sign-off and develop capacity for continuous inspection and review.