AI Engineer

Principal Software Engineer

Location: Bangalore India

Department: AI Engineering / Quality Assurance

Experience Level: Mid to Senior

We are looking for a highly motivated Senior AI Test / Automation Engineer to design and scale automated validation frameworks for AI/ML models, LLM-based applications, and agentic systems. This role is critical to ensure that AI solutions meet enterprise standards for quality, reliability, safety, and compliance before and after production deployment.

Key Responsibilities

• Build and maintain AI test automation frameworks for pre-qualification and continuous validation of models and agent workflows
• Develop comprehensive test suites, including:
• Unit, integration, and end-to-end (E2E)
• Functional, regression, performance, and safety testing
• Validate AI system behavior, including:
• Non-deterministic LLM outputs
• Hallucinations and edge cases
• Multi-step agent decision-making
• Design and manage evaluation systems:
• Golden datasets
• Benchmarking pipelines (accuracy, latency, reliability)
• Automate testing within CI/CD pipelines for model updates, prompt changes, and tool integrations
• Implement observability and telemetry to enable traceability, monitoring, and audit readiness
• Collaborate cross-functionally with ML, MLOps, Product, and Security teams to define quality gates and release criteria
• Track and report quality KPIs, including test coverage, defect leakage, and system reliability
• Drive root-cause analysis and continuous improvement across the AI testing lifecycle

Required Skills

Core Engineering

• Strong programming skills in Python; familiarity with Bash, TypeScript, or Go
• Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress
• Proficiency in CI/CD tools (GitHub Actions, Jenkins, GitLab CI)
• Experience with cloud platforms (AWS, Azure, GCP) and containers (Docker, Kubernetes)

AI / ML & Agentic Systems

• Hands-on experience with LLM ecosystems (OpenAI, Anthropic, Bedrock)
• Familiarity with:
• RAG architectures and vector databases (Pinecone, Weaviate)
• Agent frameworks (LangChain, LlamaIndex, AutoGen)

AI Testing Techniques

• Experience with non-deterministic testing approaches (statistical assertions, tolerance thresholds)
• Knowledge of evaluation methods:
• LLM-as-a-judge
• BLEU, ROUGE, semantic similarity scoring
• Experience with prompt and agent regression testing
• Understanding of AI safety testing, including adversarial testing, bias/fairness validation, and jailbreak detection

Tooling (Preferred)

• AI testing & observability tools: LangSmith, TruLens, Arize, Weights & Biases
• Evaluation tools: DeepEval, Ragas, PromptFoo, Giskard
• Monitoring: Prometheus, Grafana, OpenTelemetry

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and cross-functional collaboration
• Data-driven mindset with focus on quality KPIs
• Detail-oriented with a strong bias toward automation and scalability

Experience Requirements

• 7+ years in QA, SDET, or test automation engineering
• Proven experience building and scaling automation frameworks
• Hands-on experience with AI/ML systems or LLM-based applications
• Experience testing RAG pipelines or agentic workflows
• Owned end-to-end AI test strategy and architecture
• Defined quality metrics and release gates
• Delivered scalable validation pipelines for production AI systems
• Supported audit and compliance readiness

Preferred

• Experience in enterprise or regulated environments (SOC2, ISO 27001, etc.)
• Exposure to:
• Shift-left testing practices
• Production observability and monitoring
• Chaos or resilience testing

Senior-Level Differentiators

Education

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field

Nice-to-have:

• ISTQB certification
• Cloud/ML certifications (AWS, Azure, GCP)
• AI testing certifications

What Success Looks Like

• AI systems that are accurate, reliable, and safe
• Fully automated test pipelines integrated into CI/CD
• Measurable improvements in defect leakage and model quality
• Strong observability and auditability across AI systems
• Scalable validation frameworks supporting rapid AI innovation

Job Details

Position Title: AI Evangelist

Role Purpose

We are seeking a highly experienced AI Evangelist & Core AI Engineer who combines deep technical expertise in Artificial Intelligence with the ability to influence, educate, demonstrate, and accelerate AI adoption across the enterprise.

The role requires deep understanding of AI, Machine Learning, Deep Learning, Generative AI, Large Language Models, Agentic AI, AI engineering, and modern software development practices, combined with significant hands-on experience implementing scalable AI applications in complex enterprise environments.

Key Responsibilities

1. Core AI Engineering & Architecture

• Design, develop, and implement enterprise-grade AI and Generative AI applications.
• Build scalable applications using LLMs, multimodal models, AI agents, RAG, vector databases, knowledge graphs, APIs, and enterprise data platforms.
• Architect AI solutions capable of supporting enterprise-scale workloads with appropriate considerations for:
• Scalability
• Reliability
• Security
• Performance
• Observability
• Cost optimization
• Governance
• Develop and implement advanced Retrieval-Augmented Generation (RAG) architectures including hybrid search, reranking, metadata filtering, semantic retrieval, and knowledge grounding.
• Build multi-agent and agentic AI applications capable of reasoning, tool usage, workflow execution, orchestration, and enterprise system integration.
• Design AI applications integrating structured and unstructured enterprise information.
• Evaluate architecture choices between commercial models, open-source models, small language models, domain-specific models, and hosted AI platforms.
• Implement model routing, prompt orchestration, caching, guardrails, evaluation pipelines, and fallback strategies.
• Work closely with cloud, platform, security, enterprise architecture, DevOps, and application engineering teams.

2. Deep AI / ML / DL Expertise

The candidate should have strong conceptual and practical understanding across the modern AI technology landscape, including:

• Artificial Intelligence /Machine Learning/ Deep Learning/ Neural Networks /LLM/SLM/RAG/Graph RAG / MLOps / LLMOps / GenAIOps

3. Generative AI & LLM Engineering

• Build production-grade applications utilizing leading commercial and open-source LLM ecosystems.
• Design scalable LLM platforms supporting multiple enterprise applications and use cases.
• Develop reusable LLM services, APIs, components, agent frameworks, and reference architectures.

4. AI Agents & Automation

• Design and implement AI agents and agentic workflows that automate complex business and engineering processes.
• Build agents capable of interacting with enterprise applications, APIs, databases, documents, development environments, and workflow platforms.
• Evaluate emerging agent architectures and orchestration frameworks pragmatically.
• Develop reusable patterns for:
• Single-agent systems
• Multi-agent systems
• Human-in-the-loop workflows
• Autonomous workflows
• Tool-calling agents
• Coding agents
• Enterprise knowledge agents
• Establish appropriate controls for AI agent identity, permissions, auditability, security, and human oversight.

5. AI-Enabled Software Engineering

Drive adoption of AI throughout the Software Development Life Cycle.

Role​

Build core software for an on-device agentic AI platform. The work includes agent orchestration, 

local model integration, tool and service execution, context management, secure automation, and 

integration with third-party services.​

Responsibilities​

•    Design and implement agent runtimes, routing, tool calling, state management, and recovery.​

•    Integrate local language and code models using structured outputs and function calling.​

•    Integrate third-party APIs and SDKs with authentication, validation, and error handling.​

•    Develop workflow orchestration, context management, and short- and long-term memory 

components.​

•    Implement secure automation with sandboxing, approvals, verification, and rollback.​

•    Create logs, traces, tests, and evaluation harnesses for reliable production deployment.​

•    Optimize latency, memory use, model selection, and runtime performance for PC hardware.​

•    Translate product scenarios into clear technical requirements.​

Required Qualifications​

•    BS/MS in Computer Science, Computer Engineering, or a related field.​

•    5+ years of professional software engineering experience.​

•    Strong C# and .NET development skills.​

•    Experience building agentic applications, workflow engines, automation systems, or tool-

calling software.​

•    Experience integrating LLMs or small language models with structured outputs and function 

calling.​

•    Experience integrating third-party APIs, SDKs, and services into production software.​

•    Strong knowledge of asynchronous programming, concurrency, state machines, and fault 

handling.​

•    Experience with Windows application, service, or systems development.​

•    Strong software testing, debugging, and production engineering practices.​

Preferred Qualifications​

•    On-device inference using ONNX Runtime, Windows ML, DirectML, OpenVINO, 

llama.cpp, or similar runtimes.​

•    Model quantization, hardware acceleration, memory profiling, and performance optimization.​

•    Agent frameworks and protocols such as Semantic Kernel, AutoGen, LangGraph, or MCP.​

•    Windows automation using PowerShell, Win32/WinRT, Task Scheduler, UI Automation, or 

Playwright.​

•    Knowledge graphs, ontologies, vector retrieval, or hybrid agent-memory systems.​

•    Sandboxing, least-privilege security, prompt-injection defenses, secrets management, and 

audit logging.​

•    Automated evaluation, telemetry, diagnostics, and hardware-in-the-loop testing.​

Technical Skills​

Required​

• C# / .NET​

• Agent orchestration and tool calling​

• Third-party API and SDK integration​

• Asynchronous programming    • Windows development​

• State machines / workflows​

• Authentication and permissions​

• Automated testing​

• Secure software design​

Preferred​

• ONNX Runtime / Windows ML / DirectML​

• Semantic Kernel / AutoGen / LangGraph​

• MCP​

• PowerShell / Win32 / WinRT    • Knowledge graphs / vector retrieval​

• Telemetry / diagnostics​

• Sandboxing / policy controls​

• Model optimization 

Role Overview 

We are looking for an AI Engineer to design, build, and ship production AI systems, including agentic AI applications, for enterprise clients. This is a hands-on engineering role: you will write production code, build and evaluate models and agents, and work closely with architects and product teams to take solutions from prototype to scale. 

Key Responsibilities 

Design and build agentic AI systems: agent workflows, tool/function-calling, memory, and human-in-the-loop patterns. Build and productionise RAG pipelines, prompt-based applications, and LLM integrations across providers. Develop and maintain data and ML pipelines: feature engineering, model training, evaluation, and monitoring. Integrate AI systems with enterprise applications (CRMs, ERPs, ITSM tools) via APIs, events, and MCP-based tool servers. Implement guardrails, prompt-injection defences, and evaluation frameworks to keep AI systems safe and reliable in production. 

Write clean, tested, production-grade code and participate actively in code and design reviews. 

Collaborate with architects, product managers, and delivery teams to translate requirements into working AI solutions. Troubleshoot and optimise AI systems for accuracy, latency, and cost in production. 

Required Qualifications 

8–12 years of hands-on software engineering experience, with a strong, unbroken technical track record. Hands-on experience building and shipping AI/ML systems in production, not just POCs. 

Practical experience with agentic AI systems and at least one major agent framework (LangGraph, CrewAI, AutoGen, OpenAI Agents SDK, Bedrock Agents/Strands, or Semantic Kernel). 

Experience with LLM/GenAI systems: RAG pipelines, prompt engineering, structured outputs, and tool calling across providers. 

Strong Python skills (TypeScript/Node.js a plus), with production-grade testing, CI/CD, and API design practices. Working knowledge of ML fundamentals: model evaluation, feature engineering, and experimentation. Cloud-native experience on AWS and/or Azure: containers, serverless, event backbones, and vector databases. Understanding of LLM safety and reliability practices: guardrails, prompt-injection defences, and observability. 

Applied AI Engineer

About the Role

We are building an AI-powered platform that transforms credit origination by automating manual processes and enabling lenders to underwrite more loans with greater accuracy and lower risk.

We are looking for an experienced Applied AI Engineer to design and build production-grade Agentic AI systems capable of complex, multi-step reasoning across financial data, legal documents, and other unstructured information.

As an early engineering team member, you will take ownership of core AI architecture, multi-agent orchestration, reasoning workflows, evaluation frameworks, and scalable AI infrastructure.

Key Responsibilities

• Design and develop stateful, multi-agent AI systems for complex reasoning and decision-making.
• Build and optimize LLM pipelines using frameworks such as LangChain, DSPy, and GEPA.
• Implement RAG, prompt optimization, memory management, tool integration, and human-in-the-loop workflows.
• Develop AI evaluation frameworks using LLM-as-a-Judge and scenario-based testing.
• Implement AI observability and monitoring to track accuracy, latency, cost, and system performance.
• Build ETL pipelines and work with structured and unstructured data, SQL/NoSQL databases, vector databases, and graph-based data models.
• Integrate external tools and systems using modern agent protocols such as Model Context Protocol (MCP).
• Deploy and maintain AI services on AWS or Azure.
• Implement CI/CD pipelines, containerization, and Infrastructure as Code for reliable production deployments.

Required Qualifications

• 5+ years of professional software development experience in Python or JavaScript.
• Proven experience building and deploying production-level Agentic AI systems or complex LLM reasoning applications.
• Hands-on experience with LangChain and LLMOps, including tools such as Langfuse, Weights & Biases (W&B), or Helicone.
• Strong understanding of LLMs, RAG, prompt engineering, multi-agent orchestration, memory management, and tool calling.
• Experience with DSPy, GEPA, or advanced LLM inference and optimization techniques.
• Strong data engineering fundamentals, including ETL, SQL/NoSQL databases, and vector databases.
• Familiarity with MCP and human-in-the-loop workflows.
• Experience with AWS or Azure and modern DevOps practices, including Docker, CI/CD, and Infrastructure as Code.

Good to Have

• Experience with RLHF or RLAIF.
• Knowledge of model fine-tuning techniques such as LoRA and QLoRA.
• Experience with graph databases and graph-based data models.
• Exposure to financial services, credit underwriting, risk analysis, or document-intensive AI applications.

Why Join Us?

• Work on advanced Agentic AI and complex reasoning systems beyond conventional chatbot applications.
• Build foundational AI architecture and production-grade LLM workflows.
• Take ownership of technical decisions and system design as an early engineering team member.
• Solve challenging problems at the intersection of AI, finance, and intelligent automation.

What you need • • • • • • Delivered inside a regulated environment: code review, change control, a real audit trail. Based in, or able to relocate to, Bengaluru or Mumbai for the engagement. 7–15 years building production software, with .NET (C#, ASP.NET Core) as your primary stack. Python you have shipped and maintained — services, pipelines or AI components other people depend on, not occasional scripting. Hands-on work with LLMs in production: retrieval-augmented generation, prompt and agent design, and a clear account of how you evaluated whether it worked. Comfortable across SQL, REST and asynchronous integration patterns, on Azure or AWS, with CI/CD, containers and infrastructure as code. Good to have • • Migration work from .NET Framework to modern .NET. Azure OpenAI, Semantic Kernel, LangChain or LlamaIndex. • • Snowflake, including Cortex. Real estate, lease management or corporate services domain exposure

5-8 years building production software, with .NET (C#, ASP.NET Core) as your primary stack. Python you have shipped and maintained — services, pipelines or AI components other people depend on, not occasional scripting. Hands-on work with LLMs in production: retrieval-augmented generation, prompt and agent design, and a clear account of how you evaluated whether it worked. Comfortable across SQL, REST and asynchronous integration patterns, on Azure or AWS, with CI/CD, containers and infrastructure as code. deliered inside a regulated environment: code review, change control, a real audit trail. Based in, or able to relocate to, Bengaluru or Mumbai for the engagement

About the team

SecurITe’s mission is to build an Agentic‑AI driven security platform that protects critical infrastructure from modern cyber threats. Our focus is on delivering highly performant, resilient, and intelligent network security systems that help defenders stay ahead of adversaries.

About the Role

We’re looking for a seasoned Senior Quality Engineer to provide technical leadership and architectural oversight for our next‑generation cybersecurity AI platform. In this high-impact role, you will define the technical strategy for quality assurance, ensuring our agentic AI transforms cyber defense with unparalleled reliability.

You will be responsible for the end-to-end quality lifecycle, from architectural reviews to the deployment of scalable automation frameworks. Beyond technical execution, you will serve as a mentor to junior team members, fostering a culture of technical excellence and driving the strategy that ensures our solutions meet the rigorous demands of critical infrastructure protection.

What You’ll Do

●    Defining and driving comprehensive QA strategies and roadmaps for the cybersecurity platform.

●    Designing, developing, and executing test plans, test cases, and automated scripts to ensure software quality.

●    Performing functional, regression, performance, scalability and security testing to identify bugs or defects.

●    Collaborating with developers, product managers, and other stakeholders to understand product requirements and testing needs.

●    Identifying, documenting, and tracking software defects, ensuring clear communication of issues and their resolutions.

●    Leading deep-dive root-cause analysis for critical system defects and security vulnerabilities.

●    Conducting thorough reviews of product specifications and software design to identify potential areas of concern before testing.

●    Architecting and designing complex, scalable test automation frameworks to optimize CI/CD velocity.

●    Ensuring the software meets customer and business requirements by validating the functionality and performance.

●    Assisting in continuously improving QA processes, tools, and best practices to enhance software testing efficiency and effectiveness.

●    Supporting user acceptance testing (UAT) and assisting clients with product validation.

●    Mentoring junior and mid-level engineers, providing technical guidance and conducting architectural reviews.

Required Experience

●    A Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.

●    8-10 years of proven experience in quality engineering, specifically within network cybersecurity, Identity Providers, or AI-integrated platforms.

●    Expertise in manual and automated testing.

●    Deep domain expertise in complex system validation and advanced automation practices at scale.

●    Proficiency in programming languages like Python to build and run automated test scripts.

●    "Strong knowledge of software testing methodologies, performance testing tools (e.g., JMeter, k6), and security traffic generation/simulation tools (e.g., Ixia BreakingPoint, Scapy, or Snort/Suricata traffic generators)."

●    Understanding of continuous integration/continuous deployment (CI/CD) pipelines and version control systems like Git.

●    Strong communication skills for documenting test results and interacting with cross-functional teams.

●    Excellent analytical skills, attention to detail, and problem-solving ability.

●    Ability to work independently as well as collaboratively in a team environment.

●    A curious mindset with a willingness to quickly learn new technologies and testing tools.

Required Skills & Qualifications

●    Familiarity with cloud-based testing environments (GCP, AWS, Azure).

●    Experience with cybersecurity products or cloud services or IDP or Web UI

The Mindset

●    Problem Solver: You thrive on complex, ambiguous challenges and engineer elegant solutions.

●    Ownership‑Driven: You take initiative, move fast, and deliver outcomes without hand‑holding.

●    Continuous Learner: You stay ahead of the curve in AI, ML, and emerging technologies.

●    Startup DNA: You excel in fast‑moving environments where priorities evolve and impact is immediate.

Immediate Joiner - 15 days Notice period

• Primary Skillsets: Python, Generative AI, Agentic AI, LLM, experience with designing and developing production grade AI solutions, AWS/Azure/GCP, LLMOps, Data Engineering (Databricks/Snowflake), Regulated GenAI, Data and Decisions, and Agentic Systems.

• Preferred: Pharma or life sciences experience, especially in regulated (GxP) environments, experience setting up a team or practice from scratch.

Position Overview 

The AI Observability Engineer will be instrumental in implementation of scalable, cloud-native solutions to meet the growing needs of our Data & Development team. The successful candidate will demonstrate the ability to abstract complexity and create reusable, scalable patterns that accelerate development. The AI Observability Engineer will build and maintain a robust framework to ensure the reliability and maintainability of DPR Construction's complex AI systems. 

Responsibilities 

• Standardize observability practices across AI/ML and other development teams including logging, metrics, tracing, and model performance monitoring, ingesting data from multiple platforms 
• Lead hands-on implementation of automation-first DevOps and MLOps practices, enabling infrastructure-as-code and consistent, repeatable environment provisioning 
• Design and manage intelligent DataOps pipelines with automated data quality monitoring and anomaly detection 
• Deploy, maintain and monitor containerized ML workloads 
• Extend existing CI/CD pipelines to support automated infrastructure changes and ML workflows 
• Implement AI-driven data validation, schema and concept drift detection and metadata management. 
• Establish governance frameworks for AI systems, including bias detection, explainability, and auditability 
• Extend existing Azure RBAC strategy by automating role and permission management to reduce manual intervention 
• Develop automated test suites for model performance, regression, edge cases and bias validation 
• Monitor model KPIs (accuracy, precision, recall, latency, calibration) 
• Ensure reproducability of experiments and production models 
• Act as a technical point of contact for DevOps and MLOps practices, developing reusable patterns, documentation, and proof-of-concepts to drive adoption 

Qualifications 

• Bachelor’s degree in computer science, Data Science, Information Systems, or a related field 
• 5+ years of experience in DevOps, MLOps, Data Engineering, Software Engineering or Site Reliability Engineering 
• Strong understanding of cloud infrastructure and experience working with at least one major cloud provider, preferably Azure 
• Proficiency in at least one objected-oriented programming language, preferably python with hands-on experience in ml frameworks like TensorFlow, PyTorch or Scikit-learn