
AI Quality Assurance Analyst
Job Description
AI Quality Assurance Analyst
Department: Systems Management
Employment Type: Full Time
Location: Bloomington, MN
Reporting To: Head of Testing & Support
Compensation: $90,000 - $100,000 / year
Description
The ideal candidate has experience in mortgage operations (e.g., underwriting, processing, or closing) combined with strong QA/testing skills and an interest in AI technologies.
Responsibilities and Duties:
- Validate AI-driven outputs related to loan processing, underwriting, document recognition, and decisioning.
- Design and execute test cases for AI models, automation workflows, and system enhancements.
- Identify, document, track defects, inconsistencies, and performance issues.
- Perform regression, functional, and user acceptance testing (UAT) for mortgage systems.
- Collaborate with product, engineering, and data science teams to improve AI accuracy and reliability.
- Analyze AI model behavior and flag potential risks, biases, or compliance concerns.
- Ensure alignment with mortgage industry regulations and investor guidelines (e.g., Non-QM with the future agency).
- Support data validation efforts, including OCR (optical character recognition) and document classification tools.
- Monitor production performance and provide feedback for ongoing AI model training and optimization.
- Assist in developing QA standards and best practices for AI implementations.
Education and Experience:
- 3+ years of experience in QA, software testing, or mortgage operations
- Strong understanding of the mortgage lifecycle, including origination, processing, underwriting, and closing
- Experience with Loan Origination Systems (LOS), such as Vesta, LauraMac, or similar platforms
- 2+ years of ServiceNow ticketing experience required
- Familiarity with QA methodologies, including test planning, execution, and defect tracking
- Strong analytical and problem-solving skills with a high level of attention to detail
- Ability to analyze system behavior and validate expected versus actual results
Preferred Qualifications:
- Experience with AI/ML tools, automation platforms, or intelligent document processing
- Understanding of OCR, NLP, or decisioning engines
- Knowledge of mortgage compliance and regulatory requirements
- Experience with SQL, data validation, or basic scripting is a plus
- Exposure to Agile or Scrum development environments
- Ability to work in a hybrid office setting in Bloomington, MN
Key Competencies:
- AI & QA Testing Expertise: Designs and executes test cases to validate AI models, automation workflows, and system functionality.
- Analytical & Critical Thinking: Evaluates AI outputs, identifies discrepancies, and assesses model behavior for accuracy and risk.
- Mortgage Domain Knowledge: Applies understanding of loan lifecycle, documentation, and regulatory requirements to testing scenarios.
- Attention to Detail & Data Validation: Ensures precision in testing, defect tracking, and validation of OCR and document classification outputs.
- Cross-Functional Collaboration: Works effectively with product, engineering, and data science teams to improve AI performance.
How This Role Demonstrates Our Values:
- Integrity: Ensures AI-driven decisions and outputs are accurate, compliant, and aligned with regulatory and investor requirements.
- Collaboration: Partners with cross-functional teams to enhance AI solutions and support seamless system performance.
- Excellence: Maintains high standards in testing, validation, and continuous improvement of AI-driven processes.
- Critical Curiosity: Investigates AI behavior, identifies potential risks or biases, and seeks opportunities to improve model accuracy and efficiency.
Benefits
- Competitive compensation package, including base salary and performance-based bonus opportunities
- 401(k) plan with 100% company match up to 4%
- Comprehensive health coverage: medical, dental, vision, HSA, and FSA options
- Generous paid time off: 20 days PTO, company holidays, and sick time
- Paid parental leave
- Company-paid life insurance and disability coverage
- Employee Assistance Program (EAP): mental health, financial, and wellness support
- Professional development: tuition reimbursement and growth opportunities
- Commuter and transit benefits
Successful applicants will exemplify strong ethics, integrity, respect for others, accountability for decisions and actions, and good citizenship.
Maintaining a reliable, uninterrupted high speed internet connection is a requirement of hybrid or remote positions.