Principal Quality Engineer (AI/ML)
As an AI Quality Engineer, you will design, implement, and scale the testing strategy for machine learning systems across the entire MLOps lifecycle. You will drive the quality of our AI-powered SaaS products by collaborating with Data Scientists, ML Engineers, and DevOps to ensure models are accurate, fair, scalable, and production-ready.
Key Responsibilities
1. Test Strategy and Planning
• Define comprehensive test strategies and quality gates for AI/ML models, data pipelines, and AI-integrated applications. • Design and implement test cases that validate model accuracy, robustness, and scalability under real-world scenarios. • Work with cross-functional teams to define acceptance criteria and AI-specific quality metrics (e.g., F1-score, precision/recall, latency SLAs).
2. Model Validation and System Testing
• Conduct functional, regression, and adversarial testing for AI/ML systems. • Validate model outputs for correctness, bias, fairness, and ethical alignment. • Test and monitor data pipelines for quality, consistency, and lineage. • Execute A/B testing and scenario simulations to assess model behavior under varying conditions.
3. Performance Monitoring and Optimization
• Continuously monitor model performance in production (accuracy, drift, latency, throughput). • Define and track KPIs such as precision, recall, F1-score, mAP, AUC, etc. • Work with engineering and data science teams to identify root causes of degradation and recommend retraining or optimization strategies.
4. MLOps Quality and Automation
• Integrate testing into MLOps pipelines (CI/CD/CT) to ensure automated validation at every stage of model lifecycle. • Implement data validation, drift detection, and model versioning frameworks. • Enforce reproducibility and traceability across model experiments, training datasets, and deployed artifacts.
5. Governance and Collaboration
• Promote responsible AI practices including explainability, transparency, and risk mitigation. • Contribute to documentation and audits for AI quality compliance (e.g., ISO 42001, ISO/IEC 24028). • Educate teams on AI testing techniques and foster a culture of quality.
Required Qualifications
• Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field.
• 3–5 years of experience in QA, SRE, or testing roles, with at least 2 years focused on ML/AI systems.
• Proficiency in Python and QA/test frameworks (e.g., pytest, Great Expectations).
• Strong understanding of model evaluation metrics and testing methodologies.
• Experience with MLOps tools such as MLflow, Kubeflow, Azure ML, or Vertex AI.
Preferred Qualifications
• Experience with bias/fairness toolkits (SHAP, LIME, AIF360).
• Familiarity with adversarial testing and differential privacy.
• Knowledge of cloud infrastructure, container orchestration (Kubernetes), and CI/CD.
• Domain knowledge in manufacturing, mobility, or ESG-related AI applications is a plus.
What We Offer
• An impactful role within a major corporation’s new business initiative.
• A collaborative and high-performing team environment.
• Competitive compensation and growth opportunities based on performance.