Tech Lead - Backend

Experience: 5+ years production software engineering, with 2+ years working directly on LLM or agent systems in production. 

Location: Remote 

To streamline and fast-track screening, please submit your details here (if you haven’t already): https://airtable.com/appbtkr4odapnb5I6/pagqo91lKv3VJg3GT/form 

We’ll review your responses as part of the initial screening process. Please make sure you complete and submit all details through the form to be considered for the next stage. Submissions outside the form may not be considered.

Why This Role Matters

Terrabase builds agent infrastructure that enterprise customers rely on daily for SQL generation, forecasting, data analysis, and artifact delivery. Our orchestration layer routes between specialized sub-agents, manages typed handoff contracts, runs structured eval suites, and enforces correctness across every turn.

This is not a research-prototype role. You will build and evolve agent architecture, but always in service of making the system observable, typed, evaluated, recoverable, and boringly reliable in production.

What You Will Do

Own the harness architecture and middleware stack. Our LangGraph orchestrator routes between sub-agents through a layered middleware stack: file upload handling, source resolution, local context, workspace sync, state hydration, aggregation barriers, and typed handoff contracts. You will extend this stack, enforce its contracts in code, and keep it operational as routing logic and agent surfaces evolve.

Maintain typed contracts and boundaries. Agent handoffs at Terrabase carry typed contracts with barrier conditions and retry predicates. You will design these contracts, enforce them with strict typing, manage backward compatibility when contracts change, and write the contract tests that prevent silent regressions.

Own the eval suites. We run structured eval suites across routing decisions, context-resolution accuracy, multi-turn coherence, visual reference alignment, and artifact correctness. You will extend coverage, write new evals where gaps exist, and build CI gates that block releases when regressions are detected. A routing change or prompt change with no eval coverage does not ship.

Triage production failures and close the loop. When an agent turn fails in production, you will trace it in LangSmith, identify the failure class, and convert it into a durable regression test. You will own the release gates, keep prompts and runtime contracts in sync, manage feature flag rollout risk, and remove dead paths as the system evolves.

Own SQL and artifact correctness. Our agents generate SQL over customer schemas and produce structured artifacts (reports, dashboards, data sheets) under a strict schema contract. You will own the correctness layer: source grounding, schema-aware validation, provenance surfaces, and the eval infrastructure that catches generated artifact failures before they reach customers.

Build and maintain HITL workflows. Human-in-the-loop checkpoints let users intervene, redirect, or approve mid-chain. You will design these workflows, enforce their resumable state contracts, and ensure they degrade gracefully when interrupted.

Instrument for traceability. You will extend LangSmith tracing coverage, add structured span annotations, and build the tooling that lets us diagnose a bad agent turn from production trace data alone, without requiring a local reproduction.

What We Are Looking For

• 5+ years production software engineering, with strong Python fundamentals
• 2+ years working hands-on with LLM-based systems: agent loops, tool use, context management, or inference pipelines
• Experience with LangGraph, LangChain, OpenAI/Anthropic tool-use systems, or equivalent multi-step agent/runtime orchestration
• Practical eval engineering: you have built or extended eval harnesses, written automated test cases for agent behavior, and treated evaluation as an ongoing engineering discipline
• Strong engineering hygiene: strict typing, small interfaces, contract tests, clear schema migrations, and CI discipline
• Ability to debug from production traces and artifacts, not only local reproductions
• Comfort working across prompts, Python runtime code, TypeScript product surfaces, data systems, and eval infrastructure
• Systems thinking: you design for observability, recovery, and state management, not just the happy path
• Maintenance ownership mindset: you triage, close loops, and leave systems more debuggable than you found them
• Pragmatic judgment: you can distinguish between reliability-critical infrastructure and speculative abstraction

Bonus Points

• HITL workflow design: checkpoints, approvals, mid-chain interrupts, resumable state
• Context engineering depth: chunking strategies, retrieval-augmented generation, semantic routing, re-ranking
• Experience with LangSmith, Weights and Biases, or similar trace and evaluation platforms
• Prior work shipping agent systems to enterprise customers where SQL or data correctness is a hard requirement
• Experience with mypy, Pydantic contracts, or strict typing disciplines in a production Python codebase

Life at Terrabase

We are a sharp, focused, fully remote team building agent infrastructure that enterprise customers trust with their data. You will work directly alongside the engineer who designed this harness, with broad ownership, generous compute budgets, and a culture that treats reliability as a product requirement, not a research topic.

Terrabase is an equal-opportunity employer. We celebrate diversity and are committed to building an inclusive environment for every team member.

Job responsibility The Lead Developer contributes, to create a customer’s solution, in the building phase of the software development lifecycle. The Software Engineer is responsible for performing the detailed design of application and technical architecture components and classes in accordance to the specification provided by the Solution Architect, for coding of SW components, and for the early testing phases (component testing), and system testing support

Hands on experience in Java, Python, scripting
- Design, implement, deploy and support web-based applications and web services using server technologies stacks that include Java, MySQL
- Experience in Java/J2EE using web frameworks Play or Spring boot, REST API Development and ORM (JPA, Hibernate eBean etc.)
- In depth knowledge of Design patterns and Data structures
- Databases - MySQL, Oracle, SQL
- Solid understanding of concepts like Web Services, SOA, REST APIs, Message Queue, Caching, Distributed/Scalable Architecture
- Secure application development best practices, such as OWASP
- Document and maintain all design documents and involve in review process.
- Experienced problem solving and debugging skills.
- Good verbal and written communication and interpersonal skills
- Experience in Agile methodology is added advantage
- Knowledge of Cloud Knowledge /Azure
- Must know about Unix knowledge
- experience in Elastic search

We are seeking a motivated and detail-oriented Junior Backend Engineer to join our team. The role involves developing and maintaining backend systems using Django and Python, while also managing on-premises hardware and deployments. You will work closely with cross-functional teams to ensure our software is reliably configured, deployed, and maintained in an on-prem environment.

Key Responsibilities

Backend Development: Design, develop, and maintain backend services and APIs using Python (Django or Flask).

On-Prem Deployment: Install, configure, and manage software on on-premises hardware, including servers and networking equipment.

Database Management: Design database schemas, write efficient queries, and optimize data access layers (PostgreSQL, MySQL, etc.).

API Integration: Implement and maintain RESTful APIs and integrate with third-party services.

Troubleshooting & Debugging: Diagnose and resolve application and infrastructure issues to ensure high availability.

Version Control & CI/CD: Use Git for version control and contribute to continuous integration and delivery pipelines (GitHub Actions, Jenkins, etc.).

Security & Compliance: Work with the team to ensure software and infrastructure meet security and compliance requirements.

Documentation: Maintain clear, comprehensive documentation for your code, configurations, and on-prem deployment processes.

Skills And Qualifications 

• 2-3 years of hands-on experience in back-end development with Node.js. ● Knowledge of Node.js frameworks 
• Good understanding of server-side templating languages 
• Understanding of code versioning tools, such as Git ● Have worked in any of the cloud based platform AWS, GCP, Docker, Kubernetes. 

• At least one year experience with PHP or NodeJS.
• Experience with at least one Relational database like MongoDB, MySQL or NoSQL
• Experience with multiple PHP frameworks will be plus(e.g Wordpress, Laravel, Codeigniter,Symfony, cakePHP)
• Proficiency with multiple PHP Frameworks.
• Proficiency designing and building APIs.
• Comfortable in a work environment that requires strong problem-solving skills and independent self-direction coupled with team collaboration and open communication
• Thorough knowledge of Ajax, Javascript, HTML,CSS and other web technologies XHTML and HTML5 coding is desirable.
• Experience in both Linux and Windows environment development. Understanding of RESTful web services.
• Work in a fast-paced, creative atmosphere to develop new ideas that adapt to evolving user needs
• Experience using Git / GitHub. Excellent communication skills      

Node.js Developer Responsibilities:

• Developing and maintaining all server-side network components.
• Ensuring optimal performance of the central database and responsiveness to front-end requests.
• Collaborating with front-end developers on the integration of elements.
• Designing customer-facing UI and back-end services for various business processes.
• Developing high-performance applications by writing testable, reusable, and efficient code.
• Implementing effective security protocols, data protection measures, and storage solutions.
• Running diagnostic tests, repairing defects, and providing technical support.
• Documenting Node.js processes, including database schemas, as well as preparing reports.
• Recommending and implementing improvements to processes and technologies.
• Keeping informed of advancements in the field of Node.js development.

Node.js Developer Requirements:

• Bachelor's degree in computer science, information science, or similar.
• At least 4 years' experience as a Node.js developer.
• Extensive knowledge of JavaScript, web stacks, libraries, and frameworks.
• Knowledge of front-end technologies such as HTML5 and CSS3.
• Superb interpersonal, communication, and collaboration skills.
• Exceptional analytical and problem-solving aptitude.
• Great organizational and time management skills.
• Availability to resolve urgent web application issues outside of business hours.

• Bachelor's Degree or equivalent experience.
• 5-8 years of experience with software development, preferably in Agile environment.
• Advanced knowledge of in software development tools, programming languages, databases, and environments, including but not limited to:
  • Python 3.x, Django, functional Python, procedural Python, object oriented Python, Asynchronous Design, Micro Services, memcache, REDIS, Celery, ElastiSearch, RESTful web services, Relational (Oracle & MySQL) and Non-Relational (NOSQL & DynamoDB) databases, AWS.

• Architect new and optimize existing software codebases and systems used to crawl, launch, run, and monitor the Anakin family of app crawlers
• Deeply own the lifecycle of software, including rolling out to operations, managing configurations, maintaining and upgrading, and supporting end-users
• Configure and optimize the automated testing and deployment systems used to maintain over 1000+ crawlers across the company
• Analyze data and bugs that require in-depth investigations
• Interface directly with external customers including managing relationships and steering requirements

• Extremely effective, self-driven builder
• 2+ years of experience as a backend software engineer
• 2+ years of experience with Python
• 2+ years of experience with AWS services such as EC2, S3, Lambda, etc.
• Should have managed a team of software engineers
• Deep experience with network debugging across all OSI layers (Wireshark)
• Knowledge of networks or/and cybersecurity

• Broad understanding of the landscape of software engineering design patterns and principles
• Ability to work quickly and accurately in a highly stressful environment during removing bugs in run-time within minutes
• Excellent communicator, both written and verbal

• Must be available to work extended hours and weekends when needed to meet critical deadlines
• Must have an aversion to politics and BS. Should let his/her work speak for him/her.
• Must be comfortable with uncertainty. In almost all the cases, your job will be to figure it out.
• Must not be bounded to comfort zone. Often, you will need to challenge yourself to go above and beyond.

We are looking for strong Backend developers to join our growing Team and help build Software applications and Tools that help streamline enterprise processes and build efficiency.

Roles & Responsibilities:

-Well versed with Javascript / Typescript, Node.js, Node.js frameworks like Nest.js, express.js, Postgres DB (relational databases), Redis / Memcache, Queues.

• Proficient in algorithms and data structures.
• Having strong experience in designing data intensive and scalable systems.
• Comfortable with microservices based architecture.
• Knowledge of software engineering processes and unit testing.
• Past experience of mentoring a team of at least two people.
• Prior experience with both relational and non-relational databases.
• Prior experience with caching.
• Ability to debug and hotfix a problem with production in a swift amount of time.
• Communication skills.