AI Workflow Analyst

We’re Hiring: AI Workflow Analyst at Sophie Society!

At Sophie Society, we’re building AI tools that handle real Amazon tasks—like campaign audits, search term analysis, and listing reviews.

Before anything goes live, it needs to be tested properly.

This role is responsible for that. You make sure our AI outputs are accurate, useful, and safe to use on real accounts.

What you’ll do

• Create test cases from real Amazon data (search term reports, campaigns, listings)

• Run AI tools and review the outputs

• Check for:

- Wrong keyword classification

- Bad campaign structure decisions

- Missing or incorrect insights

- Hallucinated data

• Write simple scoring criteria to evaluate outputs

• Identify what’s causing issues (prompt, input data, or model)

• Work with the builder to fix and improve results

• Approve or block tools before they go live

Requirements

• Strong hands-on use of ChatGPT and Claude

• Ability to write clear prompts and improve them through testing

• Strong attention to detail (you catch errors others miss)

• Clear, structured writing

• Comfortable with:

- Git, Markdown, YAML

- Reading basic Python or TypeScript

Nice to have

• Experience with Amazon PPC (campaigns, search terms, structure)

• Experience with tools like LangSmith, Promptfoo, or similar

• Experience working in an agency

What we offer

• Fully remote, comfortable work environment — work from anywhere

• Full-time (40h/week) — with flexibility around your schedule

• $35,000–$60,000 USD — based on experience