Position Summary

We are seeking an experienced Salesforce Application Engineer to design and build a scalable quality assurance and evaluation framework for AI-enabled employee support experiences delivered through Salesforce.

As AI handles a growing volume of employee interactions, the organization requires a systematic mechanism to evaluate response quality, detect regressions, identify hallucinations, validate policy compliance, and monitor overall AI performance at scale.

This role will develop the technical infrastructure, workflows, integrations, dashboards, and automation required to continuously assess AI-generated and human-assisted responses across Salesforce-based employee service channels. The engineer will work closely with AI, Product, Employee Experience, Compliance, Data Science, and Quality Engineering teams to establish measurable quality standards and operational controls.

Key Responsibilities

1. AI Evaluation Framework Development

• Design and implement a comprehensive evaluation framework for AI-generated employee support responses.
• Establish automated and human-in-the-loop evaluation workflows.
• Develop mechanisms to assess:
  • Response accuracy
  • Completeness
  • Relevance
  • Clarity
  • Tone
  • Groundedness
  • Policy compliance
  • Employee experience quality
• Build configurable evaluation scorecards and quality thresholds.
• Enable evaluation across multiple use cases, channels, employee groups, and support domains.
• Maintain versioned evaluation datasets, benchmarks, and test scenarios.

1. Salesforce Application Engineering

• Design, configure, and develop Salesforce solutions supporting AI quality assurance and evaluation.
• Build custom objects, Apex services, Lightning Web Components, flows, validation rules, and automation.
• Develop reusable services for capturing AI prompts, responses, source references, confidence scores, and evaluation outcomes.
• Extend Salesforce Service Cloud and related employee support capabilities.
• Implement secure, scalable, and maintainable solutions aligned with Salesforce engineering standards.
• Support sandbox, development, testing, staging, and production environments.

1. Response Quality Measurement

• Create automated evaluation pipelines for AI-generated and human-assisted responses.
• Compare responses against approved knowledge sources, expected answers, and business policies.
• Develop scoring logic for factual accuracy, relevance, completeness, and actionability.
• Capture evaluator feedback and convert it into structured quality metrics.
• Build workflows for sampling and reviewing high-risk or low-confidence interactions.
• Enable trend analysis by use case, model version, region, policy area, and interaction type.

1. Hallucination Detection and Grounding Validation

• Develop mechanisms to identify unsupported, fabricated, or inconsistent AI responses.
• Validate whether responses are grounded in approved enterprise knowledge sources.
• Capture and assess citations, source references, retrieval results, and confidence indicators.
• Flag responses that contain unverifiable claims or contradict enterprise policy.
• Route suspected hallucinations for human review and remediation.
• Partner with AI and data science teams to improve retrieval, prompting, and model behavior.

1. Regression Testing Infrastructure

• Build automated regression suites for AI-enabled Salesforce capabilities.
• Maintain benchmark prompts, expected responses, edge cases, and negative test scenarios.
• Compare response quality across model, prompt, knowledge base, workflow, and application releases.
• Detect quality degradation before production deployment.
• Integrate AI evaluation tests into CI/CD and release-management pipelines.
• Establish release gates based on defined quality and compliance thresholds.

1. Policy and Compliance Validation

• Translate employee support policies, procedures, and regulatory requirements into executable evaluation rules.
• Build automated checks for prohibited content, sensitive data handling, required disclosures, and escalation requirements.
• Ensure AI responses comply with applicable HR, privacy, security, legal, and corporate policies.
• Maintain audit trails for evaluations, overrides, approvals, and corrective actions.
• Support compliance reviews, audits, and evidence collection.
• Implement access co