Senior Software Engineer, Applied ML Systems

Experience: 6+ years building and operating production ML systems that drive commercial decisions at scale. 

Location: Remote 

To streamline and fast-track screening, please submit your details here (if you haven’t already): https://airtable.com/appbtkr4odapnb5I6/pag05ROZwgz5AaLDG/form 

We’ll review your responses as part of the initial screening process. Please make sure you complete and submit all details through the form to be considered for the next stage. Submissions outside the form may not be considered.

Why This Role Matters

Terrabase builds decisioning infrastructure for enterprise customers: ranked recommendations, scoring pipelines, and policy-governed outputs that drive real commercial action. Our ML systems do not live in notebooks. They run multi-stage evaluation harnesses, apply structured governance rules, backtest against historical outcomes, and ship ranked outputs that customers act on daily.

This role owns the decisioning system end to end. That means the models, the eval harness, the policy layer, the production services, and the technical roadmap for where all of it goes next.

What You Will Do

Own the decisioning and ranking pipeline. Design, extend, and operate the end-to-end system: candidate generation in DuckDB, multi-stage scoring with LightGBM and AutoGluon, post-score policy application, and final ranked output delivery. You understand each layer well enough to debug latency, correctness, and coverage problems quickly, and to design the next version.

Lead the evaluation harness. Our eval pipeline runs multiple gates before any output ships: data health checks, specification validation, business rules enforcement, resolution checks, LLM-as-judge scoring, backtest against historical outcomes, and final output validation. You will own this harness, extend it as the system grows, and ensure every model or pipeline change is measurable and reproducible before it reaches a customer.

Apply policy logic with rigor. Our ML systems operate under structured governance rules that determine which offers apply to which customer segments, under what conditions. You will implement, test, and audit these rules in code, not configure them in a spreadsheet. Every exclusion must be traceable and explainable.

Engineer features that move metrics. Identify and build the behavioral signals, engagement indicators, contract features, and value-band attributes that improve model performance. Close the loop from feature hypothesis through offline evaluation to production monitoring. Own the data contracts between upstream sources and the scoring pipeline.

Build and maintain the production pipeline and service layer. The decisioning system is not a batch notebook. You will write and operate the Python pipeline and service layer that wraps model inference, handles edge cases, versions model artifacts, and connects to downstream consumers. You own CI, test coverage, reproducible training runs, monitoring, and production incidents.

Drive technical direction. Write design documents, lead code review, and set the engineering standard for the decisioning system. Help define the roadmap: what gets built, in what order, and why. Mentor contributors who work alongside you on this system.

Work forward-deployed. You will engage directly with customer stakeholders to understand business context, interpret model outputs, and translate commercial requirements into system constraints. You are accountable for customer delivery, not just model accuracy.

What We Are Looking For

• 6+ years building and operating production ML systems, not prototypes or research work
• Strong Python skills across the full ML lifecycle: data pipelines, feature engineering, model training, inference services, and monitoring
• Production experience with gradient boosting models (LightGBM, XGBoost)
• Hands-on with DuckDB or similar in-process analytical engines for large-scale data processing
• Evaluation discipline: held-out metrics, backtesting against historical data, multi-gate eval pipelines, LLM-as-judge patterns
• Experience applying business rules, policy overrides, or constraint layers on top of model outputs
• Engineering fundamentals: CI pipelines, data contracts, versioned artifacts, test coverage, incident response
• Technical leadership: design docs, code review, roadmap input, mentoring
• Comfort with forward-deployed work: you can run a meeting with a non-technical stakeholder and turn the output into a system requirement
• Comfort inheriting an existing production codebase, improving its structure, and raising reliability without rewriting everything from scratch

Bonus Points

• Experience with next-best-offer engines, customer-level targeting, or recommendation systems at scale
• Experience with AutoML frameworks (AutoGluon or similar) in a production scoring pipeline
• Thompson sampling, multi-armed bandits, or portfolio-level optimization experience
• Exposure to structured data from telecoms, financial services, or retail sectors
• Prior work owning a decisioning or ranking system as the technical lead

Life at Terrabase

We are a sharp, focused, fully remote team that ships to real enterprise customers weekly. You will own a system that drives measurable commercial outcomes, with high autonomy, generous cloud budgets, and a culture that prizes rigor over hype.

Terrabase is an equal-opportunity employer. We celebrate diversity and are committed to building an inclusive environment for every team member.