Software Engineer, Data & AI Platform (m/f/x)

Job description

About us We’re Cortea , a Berlin startup transforming audits with AI . Manual, document-heavy audits waste expert time while demand keeps rising. Our AI-powered software and specialized AI agents remove the repetitive work so auditors can focus on judgment. Backed by top-tier VCs with >15m funding, with a working product and paying customers, we’re rapidly scaling. We value first-principles thinking, speed, trust, and kindness . We build side by side in our Berlin office . Your Role We are looking for an Engineer with strong data engineering and AI systems experience to build the data, evaluation, and observability foundation for production-grade LLM agents used in complex audit workflows. This role sits at the intersection of backend engineering, data engineering, AI infrastructure, and LLM operations . You will work hands-on in our backend and agent architecture, building the systems that help us evaluate, monitor, debug, optimize, and continuously improve AI agents in production. This is not a traditional analytics, BI, or dashboarding role. You should expect to write production code, design infrastructure, work inside backend systems, and directly improve the quality, cost, reliability, and performance of LLM-based agents. What you’ll do You will help building and operating the technical infrastructure around our AI agents, with a focus on data infrastructure, evaluation, observability, and optimization. Your work will include: Building online and offline evaluation systems for LLM agents, including pipelines that use golden datasets, ground-truth data, human review workflows, and experiment results. Creating automated quality gates so changes to prompts, context, models, or agent logic can be tested before reaching production. Analyzing large volumes of agent traces and executions to identify failure modes, quality regressions, latency issues, reliability gaps, and cost optimization opportunities. Working with columnar data stores and analytical databases such as BigQuery, ClickHouse, or similar technologies. Building reliable data retention and replay mechanisms for long-term analysis of production agent behaviour. Creating observability tooling for trace analysis, experiment monitoring, production dashboards, logging, tracing, and debugging. Working inside our core backend and agent architecture, including building new agents or improving existing ag…