QA Lead (AI/ML Products)

Principal Software Engineer

Location: Bangalore India

Department: AI Engineering / Quality Assurance

Experience Level: Mid to Senior

We are looking for a highly motivated Senior AI Test / Automation Engineer to design and scale automated validation frameworks for AI/ML models, LLM-based applications, and agentic systems. This role is critical to ensure that AI solutions meet enterprise standards for quality, reliability, safety, and compliance before and after production deployment.

Key Responsibilities

• Build and maintain AI test automation frameworks for pre-qualification and continuous validation of models and agent workflows
• Develop comprehensive test suites, including:
• Unit, integration, and end-to-end (E2E)
• Functional, regression, performance, and safety testing
• Validate AI system behavior, including:
• Non-deterministic LLM outputs
• Hallucinations and edge cases
• Multi-step agent decision-making
• Design and manage evaluation systems:
• Golden datasets
• Benchmarking pipelines (accuracy, latency, reliability)
• Automate testing within CI/CD pipelines for model updates, prompt changes, and tool integrations
• Implement observability and telemetry to enable traceability, monitoring, and audit readiness
• Collaborate cross-functionally with ML, MLOps, Product, and Security teams to define quality gates and release criteria
• Track and report quality KPIs, including test coverage, defect leakage, and system reliability
• Drive root-cause analysis and continuous improvement across the AI testing lifecycle

Required Skills

Core Engineering

• Strong programming skills in Python; familiarity with Bash, TypeScript, or Go
• Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress
• Proficiency in CI/CD tools (GitHub Actions, Jenkins, GitLab CI)
• Experience with cloud platforms (AWS, Azure, GCP) and containers (Docker, Kubernetes)

AI / ML & Agentic Systems

• Hands-on experience with LLM ecosystems (OpenAI, Anthropic, Bedrock)
• Familiarity with:
• RAG architectures and vector databases (Pinecone, Weaviate)
• Agent frameworks (LangChain, LlamaIndex, AutoGen)

AI Testing Techniques

• Experience with non-deterministic testing approaches (statistical assertions, tolerance thresholds)
• Knowledge of evaluation methods:
• LLM-as-a-judge
• BLEU, ROUGE, semantic similarity scoring
• Experience with prompt and agent regression testing
• Understanding of AI safety testing, including adversarial testing, bias/fairness validation, and jailbreak detection

Tooling (Preferred)

• AI testing & observability tools: LangSmith, TruLens, Arize, Weights & Biases
• Evaluation tools: DeepEval, Ragas, PromptFoo, Giskard
• Monitoring: Prometheus, Grafana, OpenTelemetry

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and cross-functional collaboration
• Data-driven mindset with focus on quality KPIs
• Detail-oriented with a strong bias toward automation and scalability

Experience Requirements

• 7+ years in QA, SDET, or test automation engineering
• Proven experience building and scaling automation frameworks
• Hands-on experience with AI/ML systems or LLM-based applications
• Experience testing RAG pipelines or agentic workflows
• Owned end-to-end AI test strategy and architecture
• Defined quality metrics and release gates
• Delivered scalable validation pipelines for production AI systems
• Supported audit and compliance readiness

Preferred

• Experience in enterprise or regulated environments (SOC2, ISO 27001, etc.)
• Exposure to:
• Shift-left testing practices
• Production observability and monitoring
• Chaos or resilience testing

Senior-Level Differentiators

Education

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field

Nice-to-have:

• ISTQB certification
• Cloud/ML certifications (AWS, Azure, GCP)
• AI testing certifications

What Success Looks Like

• AI systems that are accurate, reliable, and safe
• Fully automated test pipelines integrated into CI/CD
• Measurable improvements in defect leakage and model quality
• Strong observability and auditability across AI systems
• Scalable validation frameworks supporting rapid AI innovation

About the Role

We are looking for an experienced Senior QA Engineer to join our engineering team in Hyderabad. This is primarily a Manual QA role for someone who takes strong ownership of product quality and system stability. The ideal candidate should be proactive in identifying issues before they impact users, maintaining strong regression coverage, monitoring production behaviour, and working closely with developers to continuously improve application reliability.

Experience with test automation is a strong advantage, but the primary focus of this role is hands-on manual testing and quality ownership.

Key Responsibilities

· Own end-to-end quality across new features, existing functionality, integrations, and production releases.

· Perform functional, regression, integration, API, exploratory, smoke, and end-to-end testing across web applications and backend services.

· Build and maintain comprehensive test cases, regression suites, checklists, and release validation processes.

· Proactively monitor applications and production behaviour, identify unusual patterns or failures, and investigate issues before they become customer-impacting problems.

· Work closely with developers and product teams to reproduce issues, identify root causes, validate fixes, and prevent regressions.

· Test APIs and third-party integrations using tools such as Postman and validate data directly using MySQL.

· Take ownership of release quality, production stability, defect prevention, and continuous improvement of QA processes.

· Contribute to automation testing where appropriate and help identify high-value areas for automation.

Required Skills & Experience

· 5+ years of professional QA/testing experience, with strong hands-on experience in manual testing.

· Strong experience with functional testing, regression testing, integration testing, exploratory testing, release testing, and defect management.

· Strong working knowledge of MySQL, including writing queries to validate data, investigate issues, and troubleshoot application behaviour.

· Experience testing REST APIs, third-party integrations, and backend services, preferably using Postman or similar tools.

· Good understanding of web applications, client-server architecture, APIs, databases, authentication, and common application workflows.

· Experience with production monitoring, log analysis, incident investigation, root-cause analysis, and post-release validation.

· Ability to create clear test scenarios, identify edge cases, reproduce complex issues, and communicate defects effectively.

· Strong ownership, problem-solving, attention to detail, and communication skills.

Automation – Good to Have

Experience with Playwright, Selenium, Cypress, or similar automation frameworks would be an advantage. The candidate should be able to identify scenarios that are suitable for automation, contribute to automated regression coverage, and work with developers to improve overall test efficiency.

What We Are Looking For

We are looking for someone who thinks beyond simply executing test cases. The successful candidate should take personal ownership of application quality and stability, proactively look for weaknesses in the system, question assumptions, identify risks, and continuously improve regression coverage.

They should be comfortable monitoring production behaviour, investigating problems independently, following issues through to resolution, and ensuring that fixes do not introduce new regressions. The candidate must be comfortable working with an international team and willing to work from our Hyderabad office during UK business hours.

• 12 - 16 years of experience in Enterprise Software Product Engineering (Development/Quality) 
• Working experience with Enterprise SaaS product companies 
• 6+ years of experience in leading/managing multi-discipline activities - manual, automation, performance and security testing. 
• Experience with AI powered test automation frameworks like Playwright, TestRigor etc is preferable. 
• Experience with security and performance testing tools like BurpSuite, Locust, K6 etc   
• Strong experience in generating and presenting data/metric driven reports to exec/C-level stakeholders. 
• Self drive to achieve team and organization goals with high ROI 
• Result oriented collaborator and enabler of success. 
• Preferred: 
• Enterprise software experience withing the Life Sciences industry and good understanding of GxP requirements 
• Knowledge of cloud native, distributed architecture 

Responsibilities:

• Lead the overall strategy, execution, reporting of ValGenesis’s platform/integration testing activities 
• Own the strategic vision and establish the Software QA processes, tools, frameworks. Ensure, Monitor and Measure of the same in terms of rollout, adoption, results. 
• Change advocate of lean and efficient working – Processes, Tools, Resources. 
• Work with the engineering and cross-functional leaders along with the developers/testers to achieve continuous and incremental quality improvement across all products. 
• Own and deliver automation, security, performance testing frameworks/tools/strategies.  
• Enable the adoption of such tools/frameworks during development phase. 
• Build tools to monitor and generate metrics of all QA activities - test definition, execution, results, production incidents etc. 
• Build a strategy and deliver capabilities to fully automate computer software validation for all product releases. 
• Build and sustain high performance teams – Hiring, Performance and Talent Management 

Strong Senior QA/SDET Profile with hands-on automation building and QA-function ownership

2

Mandatory (Experience 1): Must have 6+ years in QA/test automation/SDET roles with clear progression in scope (Engineer to Senior to Lead)in a B2B SaaS product company

3

Mandatory (Experience 2): Must have direct experience scaling a QA function at a SaaS company and building automation frameworks or QA processes from scratch

4

Mandatory (Tech skill 1): Must have hands-on UI automation experience with Selenium (Java + TestNG) or Playwright / Cypress (JavaScript / TypeScript) using maintainable design patterns like Page Object Model

5

Mandatory (Tech skill 2): Must have API testing & automation experience with Postman and RestAssured (or equivalent) for functional, integration, and contract testing

6

Mandatory (Tech skill 3): Must have experience with CI/CD integration (Jenkins or GitHub Actions) and Git for version control

7

Mandatory (Skill 1): Must have a strong grasp of Agile / Scrum delivery and JIRA for defect and test management

8

Mandatory (Skill 2): Must have experience leading or mentoring QA engineers and collaborating cross-functionally with developers and product teams

9

Mandatory (Skill 3): Must have solid experience using AI in test automation and manual testing exposure

10

Mandatory (Communication): Must have strong communication skills for cross-functional collaboration

11

Mandatory (Company): Must come from a B2B SaaS

Job Role: Senior QA Engineer

Company: Appmaker by StarApps

Location: Kochi, Kerala (On-Site)

Experience: 3–5 Years

Employment Type: Full-time

About Appmaker by StarApps

At StarApps, we build tools like Appmaker that help thousands of global e-commerce merchants turn their web stores into native mobile apps.

About the Role

We're looking for an experienced QA Engineer to take overall ownership of our QA cycle. You'll do hands-on testing, build strong quality control so fewer issues reach QA in the first place, and

guide our current QA team. Your goal: catch issues early, reduce the number of QA cycles per release, and ship high-quality apps faster.

Key Responsibilities

• QA Ownership: Own the end-to-end QA cycle, from requirement review to final releasesign-off.
• Quality Control: Set up quality checks, entry/exit criteria and checklists that stop issues early and reduce repeat QA cycles.
• Team Guidance: Mentor and guide the current QA team, review their test cases and bug reports, and raise the team's testing standards.
• Test Planning: Create test plans and test cases from the requirements and designs shared, covering functional, regression and edge cases.
• Device & Design Parity Testing: Test on real Android and iOS devices, and run design parity checks against approved designs.
• Bug Management: Report, prioritise and track bugs, document regressed bugs, and verify fixes with the dev team.
• Process Improvement: Find the root causes of recurring bugs and improve QA processes, tools and documentation.
• Communication: Keep management updated on release quality, risks and issues found.
• AI-Assisted Testing: Use AI tools to generate test cases and speed up testing, while reviewing the output carefully.

What We're Looking For

• Experience: 3–5 years in software QA, with strong experience testing mobile apps.
• Ownership: Proven experience owning QA for releases and reducing QA cycles or bug
• leakage.
• Leadership: Experience guiding or mentoring junior QA engineers.
• Testing Expertise: Strong knowledge of test planning, functional, regression,
• smoke/sanity testing and the bug life cycle.
• QA Tools: Hands-on experience with QA tools like BrowserStack, Appium and bug
• trackers such as Jira, and the judgment to pick the right tool for the job.
• AI Tools: Experience or strong interest in using AI tools for testing.
• Attention to Detail: You notice when a price, image, spacing or button looks off.
• Communication: Clear written and spoken English for reports, updates and working with developers and management.

Bonus Points

• Experience testing Shopify or e-commerce apps.
• Experience with test automation (Appium, Playwright or similar) or API testing (e.g.
• Postman).
• Experience setting up QA processes from scratch in a fast-moving team.

Why Join Us?

• Real Impact: Own the quality of apps used by thousands of global brands and millions of shoppers.
• Leadership: Shape how QA works at Appmaker and grow the QA team.
• Modern QA: Work with AI-driven testing tools and a modern workflow.
• Growth & Culture: High ownership and a supportive team in our Kochi office.

About the team

SecurITe’s mission is to build an Agentic‑AI driven security platform that protects critical infrastructure from modern cyber threats. Our focus is on delivering highly performant, resilient, and intelligent network security systems that help defenders stay ahead of adversaries.

About the Role

We’re looking for a seasoned Senior Quality Engineer to provide technical leadership and architectural oversight for our next‑generation cybersecurity AI platform. In this high-impact role, you will define the technical strategy for quality assurance, ensuring our agentic AI transforms cyber defense with unparalleled reliability.

You will be responsible for the end-to-end quality lifecycle, from architectural reviews to the deployment of scalable automation frameworks. Beyond technical execution, you will serve as a mentor to junior team members, fostering a culture of technical excellence and driving the strategy that ensures our solutions meet the rigorous demands of critical infrastructure protection.

What You’ll Do

●    Defining and driving comprehensive QA strategies and roadmaps for the cybersecurity platform.

●    Designing, developing, and executing test plans, test cases, and automated scripts to ensure software quality.

●    Performing functional, regression, performance, scalability and security testing to identify bugs or defects.

●    Collaborating with developers, product managers, and other stakeholders to understand product requirements and testing needs.

●    Identifying, documenting, and tracking software defects, ensuring clear communication of issues and their resolutions.

●    Leading deep-dive root-cause analysis for critical system defects and security vulnerabilities.

●    Conducting thorough reviews of product specifications and software design to identify potential areas of concern before testing.

●    Architecting and designing complex, scalable test automation frameworks to optimize CI/CD velocity.

●    Ensuring the software meets customer and business requirements by validating the functionality and performance.

●    Assisting in continuously improving QA processes, tools, and best practices to enhance software testing efficiency and effectiveness.

●    Supporting user acceptance testing (UAT) and assisting clients with product validation.

●    Mentoring junior and mid-level engineers, providing technical guidance and conducting architectural reviews.

Required Experience

●    A Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.

●    8-10 years of proven experience in quality engineering, specifically within network cybersecurity, Identity Providers, or AI-integrated platforms.

●    Expertise in manual and automated testing.

●    Deep domain expertise in complex system validation and advanced automation practices at scale.

●    Proficiency in programming languages like Python to build and run automated test scripts.

●    "Strong knowledge of software testing methodologies, performance testing tools (e.g., JMeter, k6), and security traffic generation/simulation tools (e.g., Ixia BreakingPoint, Scapy, or Snort/Suricata traffic generators)."

●    Understanding of continuous integration/continuous deployment (CI/CD) pipelines and version control systems like Git.

●    Strong communication skills for documenting test results and interacting with cross-functional teams.

●    Excellent analytical skills, attention to detail, and problem-solving ability.

●    Ability to work independently as well as collaboratively in a team environment.

●    A curious mindset with a willingness to quickly learn new technologies and testing tools.

Required Skills & Qualifications

●    Familiarity with cloud-based testing environments (GCP, AWS, Azure).

●    Experience with cybersecurity products or cloud services or IDP or Web UI

The Mindset

●    Problem Solver: You thrive on complex, ambiguous challenges and engineer elegant solutions.

●    Ownership‑Driven: You take initiative, move fast, and deliver outcomes without hand‑holding.

●    Continuous Learner: You stay ahead of the curve in AI, ML, and emerging technologies.

●    Startup DNA: You excel in fast‑moving environments where priorities evolve and impact is immediate.

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

Senior QA Engineer

Make quality part of the engineering system.

AbleCredit builds software for complex enterprise workflows where AI, APIs, business rules, human decisions, and real-world operations have to work together reliably. We care about building products that work in production - not just in demos. We're looking for a senior quality engineer who can move quality upstream: from release-time testing to better automation, contracts, observability, and engineering practices.

What You'll Own?

• Own quality strategy across web, mobile, backend APIs, integrations, and end-to-end workflows.

• Design and maintain automated regression suites for critical customer journeys.

• Build API, integration, contract, and workflow tests that catch failures before production.

• Partner with engineering and product teams during design - not only after development is complete.

• Define release readiness, risk-based testing, and production validation practices.

• Investigate escaped defects, identify systemic causes, and drive preventive improvements.

• Improve test data, environments, tooling, and CI integration so teams can ship confidently.

What We're Looking For?

• 5+ years in software quality engineering or test automation.

• Strong API and backend testing experience, including asynchronous and integration-heavy systems.

• Experience testing mobile and web applications and understanding client/ server failure modes.

• Ability to write automation in JavaScript/ TypeScript, Python, Java, or a similar language.

• Strong debugging skills across logs, network requests, databases, and distributed services.

• Experience with CI/CD and integrating automated tests into release pipelines.

• A product mindset: prioritize risk and customer impact rather than maximizing test-case counts.

Nice to Have

• Performance, load, security, or reliability testing.

• Experience with Playwright, Cypress, Appium, Postman/ Newman, k6, or similar tooling.

• Experience with event-driven systems, queues, webhooks, and long running workflows.

• Experience testing AI-assisted or non-deterministic product behavior.

How We Think About Quality?

Quality is an engineering responsibility. Automation should create confidence, not dashboards. A production escape should improve the system that allowed it to happen. The goal is simple: ship quickly without making customers our test environment.

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

About the role

We need a senior QA professional to assess our current QA practices, set up the right processes, and guide junior team members on execution. You will then scale the practice with Playwright automation.

Responsibilities

Assess existing QA processes, tools and coverage; identify gaps and drive an improvement roadmap.

Define test strategy, QA processes and quality metrics (coverage, defect leakage, release sign-off).

Guide and mentor junior QA engineers on test design, execution and automation.

Build a scalable Playwright (TypeScript) framework for UI and API tests.

Integrate automation into CI/CD with quality gates and clear reporting.

Drive adoption of AI in testing: agentic AI for test generation, self-healing scripts and failure triage, using Playwright MCP and similar tools.

Define QA approaches for AI-powered features, including LLM output evaluation and agent workflow testing.

Partner with dev and product teams to make quality a shared responsibility.

Must have

8–10 years in QA, 4+ years in test automation.

2+ years hands-on Playwright with TypeScript/JavaScript.

Proven experience assessing and setting up a QA practice from scratch.

Strong API testing, CI/CD (GitHub Actions, Jenkins or similar) and Git skills.

Experience leading and coaching junior QA engineers.

Hands-on use of AI coding assistants and agentic AI tools (e.g. Playwright MCP, Claude, GitHub Copilot) in QA workflows.

Good to have

Selenium/Cypress to Playwright migration experience.

Performance, accessibility or visual testing exposure.

ISTQB or equivalent certification.

Experience building or testing MCP servers, AI agents or LLM-based applications (evals, prompt and guardrail testing).