Senior AI Engineer (Customer Facing)

About LeadSquared

LeadSquared is a leading sales execution and marketing automation platform trusted by 2,000+ businesses globally, including healthcare, education, financial services, and real estate. Headquartered in Bengaluru with offices across the US, UK, UAE, and Southeast Asia, we empower sales teams to close faster, smarter, and at scale.

Our AI team is at the forefront of integrating cutting-edge large language model capabilities into enterprise workflows — building intelligent agents, copilots, and automation systems that redefine how businesses operate.

Role Overview

We are looking for a Senior AI Engineer with hands-on experience building LLM-powered agents and agentic AI systems. You will design, develop, and deploy autonomous AI pipelines that solve complex, multi-step business problems — from lead qualification and follow-up automation to intelligent CRM workflows and beyond.

This role is ideal for someone who is deeply excited about the frontier of AI, can move fast, and wants their work to directly impact millions of sales professionals worldwide.

Key Responsibilities

•

Design and build LLM-powered agentic systems using frameworks such as LangChain, LlamaIndex, AutoGen, or CrewAI to automate complex, multi-step workflows.

•

Develop and maintain Retrieval-Augmented Generation (RAG) pipelines with vector databases (Pinecone, Weaviate, Chroma, pgvector) for domain-specific knowledge grounding.

•

Build and integrate tool-use and function-calling capabilities into AI agents, enabling dynamic interaction with internal APIs, databases, and third-party services.

•

Implement prompt engineering strategies including chain-of-thought, few-shot prompting, and structured output parsing to ensure reliable agent behavior.

•

Design evaluation frameworks and observability pipelines (LangSmith, Helicone, custom metrics) to monitor agent performance, accuracy, and cost.

•

Collaborate with product, sales, and domain teams to translate business requirements into AI-driven solutions and features.

•

Optimize LLM inference for latency and cost using techniques like caching, model distillation, quantization, and batching.

•

Stay current with the rapidly evolving LLM ecosystem and proactively propose improvements and new approaches.

•

Contribute to internal best practices, documentation, and knowledge-sharing across the engineering org.

Required Qualifications

Experience

•

2–4 years of professional software engineering experience, with at least 1–2 years focused on LLM/AI systems.

•

Proven experience shipping LLM-based products or agentic AI systems into production environments.

Technical Skills

•

Strong proficiency in Python and familiarity with async programming patterns for AI pipelines.

•

Hands-on experience with LLM APIs: OpenAI (GPT-4o), Anthropic (Claude), Google (Gemini), or open-source models (Llama, Mistral).

•

Experience with agentic frameworks: LangChain, LangGraph, LlamaIndex, AutoGen, CrewAI, or similar.

•

Solid understanding of RAG architectures, embedding models, and semantic search.

•

Experience with vector databases and similarity search infrastructure.

•

Knowledge of REST APIs, microservices architecture, and containerization (Docker/Kubernetes).

Problem-Solving & Mindset

•

Strong ability to decompose ambiguous, open-ended problems into structured AI system designs.

•

Experience with prompt debugging, LLM evaluation, and iterative refinement workflows.

•

Ability to balance research exploration with engineering pragmatism to ship reliable systems.

Preferred Qualifications

•

Experience with multi-agent orchestration and agent memory systems (short-term and long-term).

•

Familiarity with fine-tuning or RLHF workflows for domain adaptation.

•

Background in NLP, information retrieval, or conversational AI.

•

Prior experience in B2B SaaS or CRM domain is a plus.

•

Contributions to open-source AI/ML projects or published research/blogs.

•

Experience with cloud platforms: AWS, GCP, or Azure — particularly AI/ML services

Role Overview

We are looking for an experienced AI/ML Engineer with strong expertise in Python, Generative AI, LLMs, LangChain, and LangGraph. The candidate will be responsible for designing and developing AI-powered applications, intelligent agents, and scalable LLM-based solutions.

Key Responsibilities

• Design and develop AI/ML and Generative AI applications using Python and modern LLM technologies.
• Build LLM-based applications and AI agents using LangChain and LangGraph.
• Develop agentic workflows involving tool calling, memory, reasoning, and multi-step orchestration.
• Integrate LLMs such as OpenAI, Azure OpenAI, Anthropic, Gemini, or other foundation models.
• Develop RAG (Retrieval-Augmented Generation) pipelines using vector databases.
• Work with embeddings, prompt engineering, semantic search, and document processing.
• Develop scalable APIs and backend services using Python, FastAPI, or Flask.
• Build and integrate AI solutions with existing applications and enterprise systems.
• Deploy and maintain AI/ML solutions on cloud platforms.
• Collaborate with data scientists, software engineers, and product teams to develop business-focused AI solutions.

Required Skills

• Strong hands-on experience in Python programming.
• Strong experience in AI/ML and Generative AI.
• Hands-on experience with LLMs and LLM-based application development.
• Strong experience with LangChain and/or LangGraph.
• Experience building AI Agents / Agentic AI workflows.
• Strong understanding of RAG, embeddings, vector databases, and prompt engineering.
• Experience with vector databases such as FAISS, Pinecone, Chroma, Weaviate, or Azure AI Search.
• Experience developing REST APIs using FastAPI/Flask.
• Good understanding of Machine Learning, NLP, and deep learning concepts.
• Experience with Azure / AWS / GCP cloud platforms.

Good to Have

• Experience with multi-agent systems and agent orchestration.
• Knowledge of MLOps / LLMOps.
• Experience with Docker, Kubernetes, and CI/CD.
• Knowledge of LLM evaluation, monitoring, observability, and AI governance.
• Experience with Azure OpenAI, Azure AI Foundry, or AWS Bedrock.

Applied AI Engineer

About the Role

We are building an AI-powered platform that transforms credit origination by automating manual processes and enabling lenders to underwrite more loans with greater accuracy and lower risk.

We are looking for an experienced Applied AI Engineer to design and build production-grade Agentic AI systems capable of complex, multi-step reasoning across financial data, legal documents, and other unstructured information.

As an early engineering team member, you will take ownership of core AI architecture, multi-agent orchestration, reasoning workflows, evaluation frameworks, and scalable AI infrastructure.

Key Responsibilities

• Design and develop stateful, multi-agent AI systems for complex reasoning and decision-making.
• Build and optimize LLM pipelines using frameworks such as LangChain, DSPy, and GEPA.
• Implement RAG, prompt optimization, memory management, tool integration, and human-in-the-loop workflows.
• Develop AI evaluation frameworks using LLM-as-a-Judge and scenario-based testing.
• Implement AI observability and monitoring to track accuracy, latency, cost, and system performance.
• Build ETL pipelines and work with structured and unstructured data, SQL/NoSQL databases, vector databases, and graph-based data models.
• Integrate external tools and systems using modern agent protocols such as Model Context Protocol (MCP).
• Deploy and maintain AI services on AWS or Azure.
• Implement CI/CD pipelines, containerization, and Infrastructure as Code for reliable production deployments.

Required Qualifications

• 5+ years of professional software development experience in Python or JavaScript.
• Proven experience building and deploying production-level Agentic AI systems or complex LLM reasoning applications.
• Hands-on experience with LangChain and LLMOps, including tools such as Langfuse, Weights & Biases (W&B), or Helicone.
• Strong understanding of LLMs, RAG, prompt engineering, multi-agent orchestration, memory management, and tool calling.
• Experience with DSPy, GEPA, or advanced LLM inference and optimization techniques.
• Strong data engineering fundamentals, including ETL, SQL/NoSQL databases, and vector databases.
• Familiarity with MCP and human-in-the-loop workflows.
• Experience with AWS or Azure and modern DevOps practices, including Docker, CI/CD, and Infrastructure as Code.

Good to Have

• Experience with RLHF or RLAIF.
• Knowledge of model fine-tuning techniques such as LoRA and QLoRA.
• Experience with graph databases and graph-based data models.
• Exposure to financial services, credit underwriting, risk analysis, or document-intensive AI applications.

Why Join Us?

• Work on advanced Agentic AI and complex reasoning systems beyond conventional chatbot applications.
• Build foundational AI architecture and production-grade LLM workflows.
• Take ownership of technical decisions and system design as an early engineering team member.
• Solve challenging problems at the intersection of AI, finance, and intelligent automation.

Hiring for AI Engineer

Exp: 6 - 12 yrs

Edu : BE/B.Tech/MCA

Work Location : Pune / Mumbai

Skill Set:

Total experience ranging from 6–10 years in software engineering/AI roles

Min 5 years strong programming experience in Python or Typescript is a MUST

Min 2.5 years hands-on experience in AI with LLMs, RAG pipelines, and AI frameworks

2+ years shipping LLM systems in production

Experience with cloud platforms (AWS/Azure/GCP)

Job Title: Senior AI Engineer

Experience: 5–10 years in software engineering, with 3+ years in AI/ML engineering preferred

Employment Type: Full-time

Department: Artificial Intelligence / Engineering

Location: Bangalore

We are looking for a highly skilled Senior AI Engineer to design, develop, deploy, and scale intelligent AI-powered applications and solutions. The ideal candidate will have strong expertise in machine learning, Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), AI agents, and production-grade software engineering.

The candidate will work closely with product managers, data scientists, software engineers, and business stakeholders to translate complex business requirements into scalable, reliable, secure, and cost-effective AI solutions.

Key Responsibilities

1. AI Solution Design and Development

• Design, develop, and deploy end-to-end AI and machine learning solutions.
• Build intelligent applications using LLMs, Generative AI, NLP, and deep learning techniques.
• Develop scalable AI architectures and reusable components for enterprise applications.
• Evaluate AI models and select appropriate approaches based on accuracy, latency, cost, and business requirements.

2. Generative AI and LLM Engineering

• Develop LLM-powered applications using models from providers such as OpenAI, Anthropic, Google, or open-source ecosystems.
• Design and implement RAG pipelines, document processing workflows, semantic search, and knowledge retrieval systems.
• Build AI agents and multi-step workflows using appropriate orchestration frameworks.
• Implement prompt engineering, structured outputs, tool calling, and context management.
• Optimize model performance through evaluation, fine-tuning where appropriate, and inference optimization.

3. Machine Learning and Data Engineering

• Develop and operationalize machine learning models for classification, prediction, recommendation, and other business use cases.
• Work with structured, unstructured, and multimodal data.
• Build data preprocessing, feature engineering, model training, and validation pipelines.
• Apply appropriate model evaluation techniques and statistical methods.

4. Production Deployment and MLOps / LLMOps

• Deploy AI solutions using cloud platforms such as AWS, Microsoft Azure, or Google Cloud.
• Build automated CI/CD pipelines for AI applications and model deployment.
• Implement model versioning, experiment tracking, monitoring, and performance evaluation.
• Optimize inference latency, throughput, infrastructure utilization, and operating costs.
• Establish observability for AI applications, including token usage, retrieval quality, latency, failures, and model behavior.

5. Software Engineering and System Architecture

• Develop production-quality applications and APIs using Python and modern backend frameworks.
• Design RESTful APIs, asynchronous processing systems, and microservices where appropriate.
• Integrate AI services with enterprise applications, databases, and third-party systems.
• Use relational databases, vector databases, caching, and distributed systems effectively.
• Apply software engineering best practices, including testing, code reviews, documentation, and design patterns.

6. Security, Quality, and Responsible AI

• Implement safeguards against prompt injection, data leakage, insecure tool execution, and unauthorized access.
• Apply appropriate privacy, security, and compliance controls to AI systems.
• Evaluate hallucinations, bias, reliability, and failure modes.
• Design human-in-the-loop workflows and fallback mechanisms for critical use cases.

7. Technical Leadership and Collaboration

• Lead technical design discussions and mentor junior AI engineers.
• Define engineering standards, coding practices, and AI development guidelines.
• Collaborate with cross-functional teams to deliver measurable business outcomes.
• Conduct code reviews, evaluate emerging technologies, and recommend practical innovations.
• Communicate technical trade-offs and solution architecture to both technical and non-technical stakeholders.

Required Technical Skills

Programming and Backend Development

• Strong proficiency in Python.
• Experience with FastAPI, Flask, or Django.
• Knowledge of REST APIs, asynchronous programming, data structures, algorithms, and software design principles.
• Experience with Git, automated testing, and CI/CD.

AI and Machine Learning

• Strong understanding of supervised and unsupervised learning, model evaluation, and statistical concepts.
• Hands-on experience with PyTorch, TensorFlow, or scikit-learn.
• Knowledge of NLP, embeddings, transformers, and deep learning fundamentals.

Generative AI and LLMs

• Hands-on experience building production applications using LLM APIs or open-source models.
• Strong understanding of RAG architecture, embeddings, chunking, reranking, and retrieval evaluation.
• Experience with vector databases such as Pinecone, Weaviate, Qdrant, Milvus, or pgvector.
• Familiarity with orchestration frameworks such as LangChain, LangChain, or comparable tools.
• Experience with tool calling, structured generation, agent workflows, and LLM evaluation.

Cloud and Infrastructure

• Experience with AWS, Azure, or Google Cloud.
• Working knowledge of Docker and Kubernetes.
• Familiarity with model serving, scalable APIs, monitoring, and production deployments.
• Experience with MLOps or LLMOps practices.

Databases and Data Processing

• Strong SQL skills and experience with relational databases.
• Familiarity with NoSQL databases, data pipelines, and distributed processing.
• Experience working with unstructured data, document ingestion, and retrieval systems.

Preferred Qualifications

• Experience with fine-tuning, quantization, or serving open-source LLMs.
• Experience with multimodal AI, including text, image, audio, or video.
• Knowledge of agent evaluation, workflow orchestration, and multi-agent systems.
• Experience with AI security, guardrails, governance, and responsible AI.
• Experience optimizing AI workloads for latency, scalability, and cost.
• Contributions to open-source AI projects, research, or technical publications.

Education

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
• Equivalent practical experience and demonstrated engineering expertise may be considered.

Key Success Metrics

• Quality, accuracy, and reliability of deployed AI solutions.
• Production adoption and measurable business impact.
• Reduction in inference costs and response latency.
• RAG retrieval quality and reduction in unsupported or incorrect responses.
• System availability, scalability, and operational stability.
• Delivery timelines, code quality, and technical mentoring effectiveness.

Job Description:

We are looking for a hands-on AI Engineer with experience in Generative AI and Agentic AI to build and deploy production-ready AI solutions.

Key Responsibilities:

• Develop and deploy GenAI and Agentic AI applications.
• Build RAG pipelines, LLM workflows, and AI agents.
• Develop solutions using Python, LangChain, LangGraph, LlamaIndex, or similar frameworks.
• Implement tool calling, context retrieval, and LLM orchestration.
• Integrate AI solutions with APIs and cloud platforms.
• Work with AWS/Azure/GCP, Docker, and CI/CD.

Required Skills:

• Strong Python programming skills.
• 3+ years of GenAI/Agentic AI experience.
• RAG and LLM orchestration.
• LangChain / LangGraph / LlamaIndex / AutoGen / CrewAI / Semantic Kernel.
• MCP and A2A knowledge.
• Cloud, APIs, Docker, and CI/CD experience.

Preferred Experience:

Hands-on experience building and deploying production-ready AI solutions.

Job Description – AI Engineer (End-to-End Development & Deployment)

Role Summary

We are looking for an AI Engineer with hands-on experience in designing, developing, deploying, and maintaining Generative/Agentic AI solutions in production. The ideal candidate should have end-to-end ownership of AI applications, from development to deployment, monitoring, and optimization.

Key Responsibilities

●        Design, build, and deploy Generative/Agentic AI solutions.

●        Develop applications using LLMs, RAG, AI agents, and vector databases.

●        Build scalable APIs and integrate AI solutions with enterprise applications.

●        Implement CI/CD pipelines, containerization, and MLOps best practices.

●        Monitor, optimize, and maintain production AI systems.

●        Collaborate with cross-functional teams to deliver business-driven AI solutions.

Required Skills

●       Strong programming skills in Python.

●       Experience with vector databases (e.g., Pinecone, FAISS, ChromaDB) and graph memory systems

●       Knowledge of atleast one agent development framework: Google ADK (preferred), LangChain/LangGraph/LlamaIndex, CrewAI

●       Experience with LLMs, RAG, GenAI, AgenticAI Agents

●       Hands-on experience with FastAPI, and REST APIs.

●       Knowledge of Docker, Kubernetes, Git, CI/CD.

●       Experience with AWS, Azure, or GCP. 

●       Experience with security compliance, monitoring and observability tools such as AWS CloudWatch, Azure Monitor, Google Cloud Monitoring.

🚨 Hiring – Data Scientist | Python + Agentic AI

💼 Experience: 5+ Years

Must Have:

• Strong Data Science experience

• Python

• Agentic AI / AI Agents

• Generative AI / LLMs

• RAG / Vector Databases

• LangChain / LangGraph or similar Agent Frameworks

• Machine Learning & NLP

Job Summary

We are looking for an experienced AI/ML Engineer to design, develop, deploy, and maintain machine learning and AI solutions that address complex business problems. The ideal candidate should have strong hands-on experience in Python, Machine Learning, Generative AI, LLMs, and AI/ML deployment, with the ability to work across the complete AI/ML lifecycle.

Key Responsibilities

• Design, develop, and deploy scalable Machine Learning and AI models for real-world business use cases.
• Build and optimize ML pipelines covering data preparation, feature engineering, model development, evaluation, and deployment.
• Develop solutions using Generative AI, Large Language Models (LLMs), NLP, and deep learning.
• Work with LLMs, prompt engineering, embeddings, vector databases, and Retrieval-Augmented Generation (RAG) architectures.
• Integrate AI/ML models with enterprise applications and APIs.
• Fine-tune and evaluate ML/LLM models based on business requirements.
• Implement MLOps practices for model versioning, deployment, monitoring, and continuous improvement.
• Collaborate with Data Scientists, Software Engineers, Architects, Product Managers, and business stakeholders.
• Conduct model performance evaluation, optimization, and troubleshooting.
• Ensure AI solutions meet requirements around security, scalability, reliability, responsible AI, and data privacy.
• Stay current with emerging AI/ML technologies, frameworks, and industry best practices.

Required Skills

• Strong programming experience in Python.
• Strong understanding of Machine Learning algorithms, statistics, and data structures.
• Hands-on experience with ML frameworks such as TensorFlow, PyTorch, Scikit-learn, or equivalent.
• Experience with Generative AI and LLMs such as OpenAI, Azure OpenAI, Claude, Gemini, or open-source models.
• Strong knowledge of Prompt Engineering, RAG, embeddings, vector databases, and AI agents.
• Experience with NLP, deep learning, or computer vision is an advantage.
• Experience developing and consuming REST APIs and microservices.
• Working knowledge of SQL and NoSQL databases.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Understanding of Docker, Kubernetes, CI/CD, and MLOps.
• Familiarity with Git and modern software development practices.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or a related field.
• 4 years of relevant experience in AI/ML engineering or a related field.
• Experience building and deploying production-grade AI/ML solutions.
• Enterprise application development experience.
• Experience with Azure OpenAI, AWS Bedrock, Vertex AI, or similar managed AI platforms.
• Experience with LangChain, LlamaIndex, Semantic Kernel, or comparable AI frameworks is a plus.
• Experience with AI/ML model monitoring, evaluation, and optimization.

What You Bring

• Strong problem-solving and analytical skills.
• Ability to translate business requirements into practical AI/ML solutions.
• Strong software engineering and debugging capabilities.
• Ability to work independently as well as collaboratively in a cross-functional environment.
• Good communication skills with the ability to explain complex AI concepts to technical and non-technical stakeholders.

Keywords

AI Engineer | ML Engineer | Machine Learning | Generative AI | LLM | Python | NLP | Deep Learning | RAG | Prompt Engineering | AI Agents | Azure OpenAI | AWS Bedrock | MLOps | TensorFlow | PyTorch | Scikit-learn | Vector Database | Cloud AI

Location: Gurugram

Work mode: Hybrid

Senior AI Engineer

Code Generation, Agent Architecture & LLM Systems

📍 Mumbai (On-site) | Full-time | 5+ years

About the Role:

Unico Connect is an AI-first technology partner that builds custom mobile, web, and AI products for clients across multiple geographies.

We are hiring a Senior AI Engineer for a dedicated client engagement focused on building an AI-powered application builder platform - a product where users describe software in plain English and the system generates, previews, and iteratively refines working code.

The mandatory requirement for this role is hands-on production experience shipping LLM-powered systems with agent architectures, with experience in code generation or developer tooling contexts a strong advantage.

The role is product-focused and deeply hands-on. You will own everything between the user's prompt and correct code landing in the project: the agentic loop, code generation pipeline, context management, evaluation suite, and model cost strategy.

You will work alongside the Senior MLOps Engineer who operationalises the infrastructure around your system, and collaborate closely with backend, frontend, and DevOps engineers.

Responsibilities:

Agent Architecture

Design and own the agentic loop for the platform - request interpretation, planning, tool-calling sequence (read file, edit file, run build, search code, install package), and stop conditions.

Make and revisit architectural decisions on single-agent vs. multi-agent designs, including planner/executor splits and dedicated build-repair sub-agents.

Code Generation Pipeline

Own the end-to-end generation flow: task classification, context gathering, planning, targeted edits, verification, and commit.

Implement diff/search-replace-based file editing with fuzzy matching and fallback strategies.

Enforce scope discipline so the agent makes minimal diffs and does not modify code it was not asked to touch.

Self-Repair Loop

Build and tune the automated repair loop that pipes compiler, lint, build, and runtime errors back to the model with retry budgets and model escalation.

This loop is the primary quality lever - the difference between 60-70% and 90%+ build success rates.

Context Management

Build file-relevance retrieval so the agent sees the right files, not the whole codebase: dependency graphs, AST/tree-sitter-based chunking, embeddings, recency signals, and hybrid retrieval.

Implement conversation summarisation and memory for long sessions, and address long-project degradation through codebase summaries and periodic consistency passes.

Own token budgeting and prompt caching strategy.

Prompt Engineering as a Discipline

Own the system prompt and per-task prompt variants (new feature, bug fix, styling change).

Maintain few-shot examples and enforce coding conventions, stack rules, and prohibited behaviours such as no hardcoded secrets and no whole-file rewrites.

Version prompts like code with changelogs and rollback capability.

Evaluation and Quality Measurement

Design and own the evaluation suite: representative test prompts run on every prompt and model change, scored on build success rate, instruction adherence, and output quality including LLM-as-judge and visual/screenshot checks where relevant.

Define regression gates that block quality-degrading changes from shipping.

Treat evals the way engineers treat automated testing: versioned, automated, and tracked over time.

This responsibility is non-negotiable at this level.

Model Strategy and Cost

Design model routing - cheap and fast models for classification and small edits, frontier models for complex generation.

Drive cost optimisation through prompt caching, diff-based edits over full-file rewrites, and tighter context selection.

Track cost per agent run and tokens per task; evaluate new model releases against the eval suite and lead migrations when results justify it.

Safety and Reliability of Agent Behaviour

Defend against prompt injection from user content and fetched web content.

Ensure secrets never appear in generated client code.

Define what the agent's tools may and may not do in collaboration with the platform team.

Contribute to output moderation and abuse-pattern awareness.

Mentorship and Engineering Standards

Run code reviews, define engineering conventions for AI work, and raise the engineering bar across the AI team.

Work closely with the Senior MLOps Engineer on handoff of eval design, prompt configurations, and model routing logic.

Requirements:

Hands-on Production Ownership of LLM-Powered Systems with Agent Architectures (Mandatory)

Must have personally shipped and operated at least one complex production AI system - agentic, multi-step, or code generation - with end-to-end ownership of architecture, evaluation, and cost.

POCs, internal demos, and tutorial-grade work do not qualify.

5+ Years of Professional Software or AI Engineering Experience

With at least 3 years focused on LLM applications, AI engineering, or production AI systems.

Candidates with strong backend backgrounds and a clear, substantive pivot into LLM systems qualify.

Strong Python Proficiency and Service Development

Production-grade Python with FastAPI or equivalent: type hints, async patterns, streaming responses, testing, and packaging.

Not notebook-only.

Depth Across LLM APIs and Agent Systems

Production experience with at least two of OpenAI, Anthropic Claude, Google Gemini, or open-weight models (vLLM, Ollama, Together).

Production experience with at least one agent framework (LangGraph, CrewAI, AutoGen, LlamaIndex Agents) or hand-rolled equivalent.

Hands-on with tool calling, structured outputs, and multi-step reasoning.

Demonstrated, Systematic Evaluation Practice - Non-Negotiable

Must have built evaluation harnesses that gate production releases, not ad-hoc testing.

Hands-on with at least one of LangSmith, Langfuse, Promptfoo, Ragas, or DeepEval.

Candidates with no systematic answer to evaluation should not be considered at senior level regardless of other strengths.

Cost Discipline for Production AI

Track record of measurable cost optimisation on production AI features.

Able to speak in specifics: cost per request, savings achieved through caching or model routing, context reduction decisions.

AWS Working Knowledge

Hands-on with EC2, S3, IAM, and Docker.

Comfort with CI/CD workflows and deploying AI services.

Awareness of LLM Security Failure Modes

Familiar with prompt injection patterns, understands that system prompt rules alone are insufficient, and has experience with output validation and content safety in production.

Nice to Have

• Experience with AST/tree-sitter tooling, diff-based editing systems, or compiler-adjacent work
• MCP server authoring
• Open-source AI contributions
• Published technical writing on LLM systems
• Multi-modal model experience
• Fine-tuning exposure (LoRA, QLoRA, PEFT)