QA Automation Engineer

Altoros – an established provider of software development services is looking for a QA Automation Engineer.

Requirements

It’s a part-time position.

• Own the test strategy across FE, BE, and LLM / agent workflows; define the test pyramid and coverage goals.
• Build and maintain automation for :
• UI / E2E flows (critical paths, multi‑step agentic scenarios).
• API / contract testing (REST / GraphQL), schema / compatibility checks.
• Data & integration tests across services and external partners.
• LLM Quality & Evals
• Design golden datasets and rubrics for LLM output evaluation (pass / fail thresholds, human‑in‑the‑loop review).
• Set up regression / eval harnesses for prompts, tools, and agents; track drift and hallucination rates.
• Implement guardrails and adversarial / red‑team tests for safety, privacy, bias, and toxicity.
• Non‑functional testing : baseline and monitor performance, reliability, and resilience for key user journeys.
• CI / CD ownership : integrate tests into pipelines, gate releases, reduce flakiness, and surface quality KPIs (e.g., failure rate, MTTR, coverage).
• Test data & environments : manage fixtures, synthetic data, and stable seeds for deterministic runs.
• Quality operations : triage defects, drive root‑cause analysis, and keep crisp, proactive communication with PM, Design, and Engineering.
• Documentation : maintain clear test plans, playbooks, and release criteria.

Nice to have :

Join Altoros Team!