Job Description
You will work across the AI layer of the platform — contributing to retrieval pipelines, agent workflows, and model evaluation. The work is hands-on and empirical: you run experiments, measure results, and iterate. You will work closely with the Senior AI Engineer and broader engineering team, taking increasing ownership as you develop depth across the platform's AI systems.
Responsibilities
• Contribute to the hybrid retrieval pipeline — implementing and tuning retrieval components, running experiments to improve quality, and validating results on real evaluations.
• Build and maintain components of the agent orchestration layer — tool integrations, prompt management, and supporting human-in-the-loop workflows.
• Instrument AI and agent systems for observability — tracing model and tool calls, capturing token and latency telemetry, and supporting failure analysis.
• Build and maintain evaluation datasets and test suites across retrieval and agent workflows, contributing to CI-level quality gates.
• Support document AI capabilities — working with parsing, extraction, and OCR pipelines as the platform serves new client engagements.
• Implement per-tenant isolation checks across retrieval and agent layers, helping ensure no cross-tenant data leakage occurs.
• Contribute to model behaviour evaluation — running prompt injection and adversarial tests as part of ongoing eval work.
Qualifications
Required qualifications
• 3+ years of software engineering experience, with at least 1 year building LLM-powered systems — RAG pipelines, agent workflows, or fine-tuning — in a production or nearproduction setting.
• Practical experience in at least one of: retrieval-augmented generation (RAG) and reranking; agent orchestration with LangGraph or comparable; or LLM fine-tuning.
• Proficient in Python and comfortable working with async code, data pipelines, and REST APIs.
• Exposure to evaluation methodology for LLM systems — has contributed to or authored an eval dataset or test suite.
• Familiarity with agent or LLM observability tooling — tracing, logging, or monitoring of model and tool calls.
• Working knowledge of modern LLM and information-retrieval concepts; can discuss trade-offs between approaches with evidence.
Preferred qualifications
• Experience with knowledge graphs or property-graph query languages.
• Exposure to document AI — PDF parsing, table extraction, or OCR pipelines.
• Has contributed to or built an evaluation dataset and labeling workflow.
• Familiarity with prompt-injection risks and mitigation strategies in agent and retrieval pipelines.
• Open-source contributions to LangGraph, sentence-transformers, or comparable projects.
Required qualifications
• 3+ years of software engineering experience, with at least 1 year building LLM-powered systems — RAG pipelines, agent workflows, or fine-tuning — in a production or nearproduction setting.
• Practical experience in at least one of: retrieval-augmented generation (RAG) and reranking; agent orchestration with LangGraph or comparable; or LLM fine-tuning.
• Proficient in Python and comfortable working with async code, data pipelines, and REST APIs.
• Exposure to evaluation methodology for LLM systems — has contributed to or authored an eval dataset or test suite.
• Familiarity with agent or LLM observability tooling — tracing, logging, or monitoring of model and tool calls.
• Working knowledge of modern LLM and information-retrieval concepts; can discuss trade-offs between approaches with evidence.
Preferred qualifications
• Experience with knowledge graphs or property-graph query languages.
• Exposure to document AI — PDF parsing, table extraction, or OCR pipelines.
• Has contributed to or built an evaluation dataset and labeling workflow.
• Familiarity with prompt-injection risks and mitigation strategies in agent and retrieval pipelines.
• Open-source contributions to LangGraph, sentence-transformers, or comparable projects.
• Contribute to the hybrid retrieval pipeline — implementing and tuning retrieval components, running experiments to improve quality, and validating results on real evaluations.
• Build and maintain components of the agent orchestration layer — tool integrations, prompt management, and supporting human-in-the-loop workflows.
• Instrument AI and agent systems for observability — tracing model and tool calls, capturing token and latency telemetry, and supporting failure analysis.
• Build and maintain evaluation datasets and test suites across retrieval and agent workflows, contributing to CI-level quality gates.
• Support document AI capabilities — working with parsing, extraction, and OCR pipelines as the platform serves new client engagements.
• Implement per-tenant isolation checks across retrieval and agent layers, helping ensure no cross-tenant data leakage occurs.
• Contribute to model behaviour evaluation — running prompt injection and adversarial tests as part of ongoing eval work.