Data Analyst

AI/LLM Test Automation Engineer (SDET)

Location: Bangalore (Hybrid preferred)

Experience: 5-8 Years

Job Summary

We are seeking a Senior Test Automation Engineer (SDET) with expertise in AI/LLM testing and Azure DevOps CI/CD to build robust automation frameworks for cutting-edge AI applications. The role combines deep programming skills (Java/Python), modern DevOps practices, and specialized LLM testing to ensure high-quality AI product delivery.

Key Responsibilities

• Design, develop, and maintain automation frameworks using Java/Python for web, mobile, API, and backend testing.
• Create and manage YAML-based CI/CD pipelines in Azure DevOps for end-to-end testing workflows.
• Perform AI/LLM testing including prompt validation, content generation evaluation, model behavior analysis, and bias detection.
• Write and maintain BDD Cucumber feature files integrated with automation suites.
• Execute manual + automated testing across diverse application layers with focus on edge cases.
• Implement Git branching strategies, code reviews, and repository best practices.
• Track defects and manage test lifecycle using ServiceNow or similar tools.
• Conduct root-cause analysis, troubleshoot complex issues, and drive continuous quality improvements.

Mandatory Skills & Experience

✅ 5+ years SDET/Automation experience

✅ Java/Python scripting for test frameworks (Selenium, REST Assured, Playwright)

✅ Azure DevOps YAML pipelines (CI/CD end-to-end)

✅ AI/LLM testing (prompt engineering, model validation, RAG testing)

✅ Cucumber BDD (Gherkin feature files + step definitions)

✅ Git (branching, PRs, GitFlow)

✅ ServiceNow/Jira defect tracking

✅ Manual + Automation testing (web/mobile/API/backend)

Technical Stack

Programming: Java, Python, JavaScript

CI/CD: Azure DevOps, YAML Pipelines

Testing: Selenium, Playwright, REST Assured, Postman

BDD: Cucumber (Gherkin), JBehave

AI/ML: Prompt validation, LLM APIs (OpenAI, LangChain)

Version Control: Git, GitHub/GitLabDefect

Tracking: ServiceNow, Jira, Azure Boards

Preferred Qualifications

• Exposure to AI testing frameworks (LangSmith, Promptfoo)
• Experience with containerization (Docker) and Kubernetes
• Knowledge of performance testing for AI workloads
• AWS/GCP cloud testing experience
• ISTQB or relevant QA certifications

What We Offer

• Work on next-gen AI/LLM products with global impact
• Modern tech stack with Azure-native DevOps
• Flexible hybrid/remote work model
• Continuous learning opportunities in AI testing

ABOUT Company

We are changing the way cataloging is done across the Globe. Our vision is to empower the smallest of sellers, situated in the farthest of corners, to create superior product images and videos, without the need for any external professional help. Imagine 30M+ merchants shooting Product Images or Videos using their Smartphones, and then choosing Filters for Amazon, Asos, Airbnb, Doordash, etc to instantly compose High-Quality "tuned-in" product visuals, instantly. They have built the world’s leading image editing AI software, to capture and process beautiful product images for online selling. We are also fortunate and proud to be backed by the biggest names in the investment community including the likes of Accel Partners, Angellist and prominent Founders and Internet company operators, who believe that there is an intelligent and efficient way of doing Digital Production than how the world operates currently.

ABOUT THE ROLE

We are looking for builders of the 1st order. You should get a rush from creating things from scratch and take pleasure in watching users try to tear them apart through sheer volume and strange, unthinkable use cases. And when it breaks, you are to — make everything smooth, secure, and stronger than ever! If you inhale resilience and exhale code, apply now!

You will work closely with founders to build features, tools, and scalable architecture As with any startup, responsibilities will vary frequently. You’ll wear multiple hats, and work across many different technologies, might talk to clients directly, and have an active say in the direction of our product. We want someone who is passionate about this type of environment!

JOB RESPONSIBILITIES

• Design and build applications for the iOS and Android platforms
• Ability to lead a team of engineers as they execute on building new features for IOS and Android platforms
• Ensure the performance, quality, and responsiveness of applications
• Collaborate with a team to define, design, and ship new features
• Identify and correct bottlenecks and fix bugs
• Help maintain code quality, organization, and automation
• Work proactively in a fast-paced dynamic environment
• Provide work estimates and deliver projects on schedule
• Owns success – Takes responsibility for successful delivery of the solutions.

JOB REQUIREMENTS

• Minimum of a bachelor's degree in engineering or relevant field and Tier 1 college
• Overall 6-10 yrs exp with Min 4 years relevant in iOS, Android, Swift, Kotlin and Objective C
• Strong understanding of Hybrid mobile skills such as Ionic, Cordova, PWAs, Flutter, ReactJS and React Native , Cordova or Capacitor
• Good understanding on native mobile skills include iOS, Android, Swift, Kotlin and Objective C.
• Good to have knowledge JS frameworks such as Angular
• Good hands-on experience on various design and architecture aspects - MVP, MVVM etc.
• Good knowledge of writing unit tests for mobile
• Strong knowledge of mobile app performance, using various tools and Android Studio IDE
• Strong background in Agile and working closely with product management
• Should be able to conduct scrum meetings and ensure team is not blocked
• Leadership skills necessary to manage and develop a team
• Organizational competencies and project management skills to keep projects, processes, and the entire engineering team on track
• Self-motivated attitude with the ability to multitask and thrive in a timeline-driven environment
• Interpersonal communication skills with expertise in distilling complicated topics to a broader audience

Responsibilities:

● Work with development teams and product managers to ideate software solutions.

● Design user-side and server-side architecture.

● Build the front-end of applications through appealing visual design.

● Develop and manage well-functioning databases and applications on MYSQL.

● Write effective APIs.

● Test software to ensure responsiveness and efficiency.

● Troubleshoot, debug and upgrade software.

● Create security and data protection settings.

● Build features and applications with a mobile responsive design.

● Write technical documentation.

Requirements:

● Proven experience as a Full Stack Developer or similar role.

● Experience developing web and mobile applications. ● Knowledge of multiple front-end languages and libraries (e.g. HTML/ CSS, JavaScript).

● Knowledge of back-end language(Node.js) and JavaScript frameworks (Angular, React, Vue but preferably React).

● Familiarity with MySQL databases, web servers (e.g. Apache) and UI/UX design.

● Excellent communication and teamwork skills.

● Great attention to detail Organizational skills. The knowledge of following, will fetch you brownie points during interviews:

● Ruby on Rails.

● Familiarity with AWS.

● Familiarity with Docker

Marketing Lead (Customer Growth & Retention) - Y Combinator backed Shopify-like platform

About Merito :-

The role requires you to:

• Answer client queries related to use of Superset software product, on email channel, and largely phone channel
• To understand queries, resolve them by providing ways to achieve use cases using Superset tool
• Ensure customer success by providing timely resolution
• Write documentation and FAQs

What you need to have: