AI Test Engineer

Strong QA Engineer Profile with manual + automation testing across web, API, and AI-driven features

2

Mandatory (Experience 1): Must have 2+ years in QA / software testing, covering both manual and automation testing

3

Mandatory (Experience 2): Must have experience designing, writing, and maintaining test plans, test cases, and test data for web, API, and AI-driven features

4

Mandatory (Experience 3): Must have experience performing functional, regression, integration, and exploratory testing across releases

5

Mandatory (Tech skill 1): Must have hands-on automation experience with a framework such as Selenium, Playwright, Cypress, or similar

6

Mandatory (Tech skill 2): Experience testing AI/ML or data products (validating accuracy, consistency, edge cases, bias, graceful failure handling)

7

Mandatory (Tech skill 3): Must have API testing experience with tools like Postman, REST Assured, or equivalent, and experience testing REST APIs and data pipelines

8

Mandatory (Tech skill 4): Must have working knowledge of Python, JavaScript, or Java for test automation

9

Mandatory (Tech skill 5): Must have SQL and data skills — able to write queries to validate data and back-end behaviour

10

Mandatory (Skill 1): Must have a strong grasp of QA methodologies, SDLC/STLC, and Agile/Scrum practices.

11

Mandatory (Skill 2): Must have an analytical mindset with sharp attention to detail and clear defect communication

12

Mandatory (Skill 3): Must have strong communication skills

13

Preferred (CI/CD): Experience integrating automated tests into CI/CD pipelines (Jenkins, GitHub Actions)

14

Preferred (Domain): Cloud/AWS and insurance domain exposure

Strong QA Engineer Profile with manual + automation testing across web, API, and AI-driven features

2

Mandatory (Experience 1): Must have 2+ years in QA / software testing, covering both manual and automation testing

3

Mandatory (Experience 2): Must have experience designing, writing, and maintaining test plans, test cases, and test data for web, API, and AI-driven features

4

Mandatory (Experience 3): Must have experience performing functional, regression, integration, and exploratory testing across releases

5

Mandatory (Tech skill 1): Must have hands-on automation experience with a framework such as Selenium, Playwright, Cypress, or similar

6

Mandatory (Tech skill 2): Experience testing AI/ML or data products (validating accuracy, consistency, edge cases, bias, graceful failure handling)

7

Mandatory (Tech skill 3): Must have API testing experience with tools like Postman, REST Assured, or equivalent, and experience testing REST APIs and data pipelines

8

Mandatory (Tech skill 4): Must have working knowledge of Python, JavaScript, or Java for test automation

9

Mandatory (Tech skill 5): Must have SQL and data skills — able to write queries to validate data and back-end behaviour

10

Mandatory (Skill 1): Must have a strong grasp of QA methodologies, SDLC/STLC, and Agile/Scrum practices.

11

Mandatory (Skill 2): Must have an analytical mindset with sharp attention to detail and clear defect communication

12

Mandatory (Skill 3): Must have strong communication skills

13

Preferred (CI/CD): Experience integrating automated tests into CI/CD pipelines (Jenkins, GitHub Actions)

14

Preferred (Domain): Cloud/AWS and insurance domain exposure

Title: AI/ML Test Engineer – GenAI

Location - Mumbai

Technical Skills

• Strong experience in Generative AI, LLMs, and Agentic AI systems

• Hands-on expertise with AI evaluation frameworks (RAGAS, DeepEval, TruLens, LangSmith, Promptfoo, etc.)

• Proficiency in Python and AI/ML development libraries

• Knowledge of Prompt Engineering, prompt testing, and optimization

Ability to define and track evaluation metrics such as accuracy, relevance, groundedness, hallucination rate, latency, and user satisfaction

• Experience in creating automated evaluation pipelines and benchmarking frameworks

• Strong understanding of AI safety, guardrails, bias testing, and responsible AI practices

• Familiarity with REST APIs, JSON, vector databases, and knowledge retrieval systems

• Experience in A/B testing, human-in-the-loop evaluation, and red teaming

• Strong experience in Manual Testing of AI/GenAI applications, including functional, exploratory, UAT, regression, and end-to-end testing

• Expertise in validating Agent Reasoning, Tool Calling, Workflow Execution, and Response Quality

• Hands-on experience in Automation Testing using Python frameworks

Key Responsibilities

• Design, execute, and automate evaluation strategies for Agentic AI applications.
• Develop evaluation datasets, test cases, and benchmark suites.
• Measure and improve agent performance, reasoning quality, tool usage, and workflow effectiveness.
• Analyze model outputs and identify hallucinations, biases, safety risks, and failure patterns.
• Collaborate with AI Engineers, Product Teams, and Domain Experts to improve agent quality and reliability.
• Generate evaluation reports, dashboards, and actionable recommendations.

About Us

We’re building the next generation of AI-powered business software, and we’re looking for people who want to shape that future with us. With Lumen, we’re reimagining how users interact with CRM — moving beyond screens, menus and dashboards to an intelligent interface where users can simply ask AI to take actions, retrieve knowledge, generate insights and get work done. With Agent Studio, we’re enabling businesses to build, test and deploy their own AI agents for real-world workflows. And with Invorto, we’re bringing AI to voice, allowing businesses to create intelligent voice agents tailored to their customer and operational use cases.

What makes this especially exciting is the stage and scale of the opportunity. You’ll get to work on genuinely hard problems across LLMs, agents, reasoning, orchestration, voice AI, evaluation, reliability and enterprise security — not as isolated experiments, but as products used in real business workflows. You’ll have the opportunity to build zero-to-one, own meaningful parts of the product end-to-end, work closely with customers, experiment rapidly, and see your work reach production at scale.

Why join now? Because the playbook for enterprise AI is still being written. You won’t just be implementing someone else’s roadmap — you’ll help define the product, architecture and experiences that become that playbook. Expect high ownership, fast iteration, hard technical and product problems, direct customer impact, and the chance to build AI systems that have to work reliably in the real world — not just in a demo.

About the Role

We are looking for a QA Engineer who specializes in testing agentic AI platforms. You will design and automate quality processes for systems that involve LLMs, autonomous agents, tool use and orchestration across Lumen and Agent Studio — ensuring that AI-driven workflows behave reliably, safely and predictably in production, not just in a demo.

What You’ll Do

• Design and build automated test suites and evaluation frameworks for agentic AI workflows, including multi-step and tool-calling behaviors.
• Use AI/LLM-based QA tools and evaluation frameworks to test model outputs, agent decisions and end-to-end task completion at scale.
• Define quality metrics and benchmarks for agent reliability, correctness, latency and safety, and track them over releases.
• Identify edge cases, failure modes and regressions specific to non-deterministic AI systems, and build automated checks to catch them early.
• Integrate automated agent/LLM testing into CI/CD pipelines to support fast, reliable iteration.
• Partner closely with AI/ML and backend engineers to reproduce issues, root-cause failures and validate fixes.
• Work with customers and customer-facing teams to understand real-world usage patterns and translate them into test scenarios.

What We’re Looking For

• 2–4 years of QA/test automation experience, including hands-on work testing agentic AI or LLM-based platforms.
• Practical experience using AI-focused QA/evaluation tools to test agent behavior, prompts and model outputs.
• Strong scripting/automation skills (Python preferred) to build and maintain test frameworks.
• Understanding of how LLM-based agents work — tool calling, orchestration, memory, reasoning chains — well enough to design meaningful test cases.
• Comfort working with non-deterministic systems and designing evaluation approaches beyond traditional pass/fail testing.
• Strong communication skills — this is a customer-facing role, and you will be expected to clearly articulate technical concepts, decisions and trade-offs to both technical and non-technical stakeholders, including customers.

Good to Have

• Experience testing voice AI or real-time conversational systems.
• Familiarity with CRM or enterprise SaaS platforms.
• Exposure to enterprise security or compliance testing for AI systems.

QA Test Engineer – Manual & Automation

Location: Bangalore, India

Experience: 2–5 years

Employment Type: Full-time

Work Mode: On-site

About Juntrax

Juntrax is a B2B Cloud SaaS product company building business operations software for SMEs. Our platform helps businesses manage their day-to-day operations through streamlined digital workflows and tools.

We are looking for a QA Test Engineer who can ensure the quality, reliability, and usability of our product through a combination of manual and automated testing.

Key Responsibilities

• Design, develop, and execute manual and automated test cases for web-based SaaS applications.
• Perform functional, regression, integration, system, and API testing.
• Create and maintain automated test scripts for critical product workflows.
• Identify, document, track, and retest bugs using appropriate bug-tracking tools.
• Collaborate closely with developers, UI/UX designers, product teams, and other stakeholders to understand requirements and ensure product quality.
• Participate in requirement analysis and contribute to test planning and test strategy.
• Perform cross-browser and responsive testing to ensure a consistent user experience.
• Validate new features, enhancements, bug fixes, and product releases.
• Maintain clear documentation of test scenarios, test cases, defects, and test results.
• Help improve QA processes, automation coverage, and overall testing efficiency.

Required Skills & Qualifications

• 2–5 years of experience in Software Quality Assurance / Testing.
• Strong understanding of manual testing concepts and QA methodologies.
• Hands-on experience with automation testing using tools/frameworks such as Selenium, Playwright, Cypress, or similar.
• Experience with API testing using tools such as Postman or similar.
• Good understanding of SQL and database validation.
• Experience with regression, integration, functional, and end-to-end testing.
• Familiarity with Git/version control and basic CI/CD concepts.
• Ability to write clear and effective test cases and defect reports.
• Good analytical and problem-solving skills.
• Strong attention to detail and a quality-focused mindset.

Good to Have

• Experience testing B2B SaaS or cloud-based applications.
• Experience with JavaScript/TypeScript, Python, Java, or another programming language.
• Experience integrating automated tests into CI/CD pipelines.
• Familiarity with Agile/Scrum development environments.
• Experience with performance or security testing.

What We’re Looking For

• Someone who can think beyond the happy path and identify edge cases and potential failure points.
• A QA professional who is comfortable working independently as well as with cross-functional teams.
• Someone who takes ownership of product quality from requirement analysis through release.
• A continuous learner who is interested in increasing automation coverage and improving QA processes.

Why Join Juntrax?

• Work directly on a growing B2B SaaS product.
• Gain exposure to the complete software development and release lifecycle.
• Opportunity to contribute to both manual and automation QA practices.
• Work closely with engineering, product, and leadership teams.
• Be part of a fast-paced startup environment where your work has a direct impact on the product.

We are seeking a forward-thinking Mid-Level QA Automation Engineer specializing in RPA, financial systems validation, and AI engineering. You will build and scale end-to-end automation workflows using UiPath for our financial services platform. Additionally, you will pioneer our testing evolution by leveraging prompt engineering, creating custom AI agents, and implementing Model Context Protocol (MCP) servers to bridge AI models with our testing infrastructure.

Key Responsibilities and Duties

   RPA Automation:

·    Design, develop, and maintain robust automated testing workflows using UiPath Studio.

 AI Agent Engineering:

·   Architect, build, and orchestrate custom AI agents to autonomously generate,execute, and self-heal test scripts.

Prompt Engineering:

·   Design, optimize, and manage advanced prompt templates to drive deterministic,high-quality code and test case generation from LLMs.

 MCP Integration:

·    Implement and configure Model Context Protocol (MCP) ecosystems to securely connect AI agents with local data, development tools, and testing environments. 

 Financial System Testing:

·   Validate end-to-end trade life cycles, market data processing, and capital market clearing workflows. 

Regression Orchestration

·   Manage continuous regression schedules via UiPath Orchestrator to ensure financial system stability. 

Mandatory skills

Project Mandatory Skills

Project Desired Skills

UiPath Studio and Orchestrator (UI/API automation activities)

Selenium or Playwright for web-based application testing

Building, deploying, and scripting autonomous AI agents for complex engineering workflows

Calypso treasury and capital markets platform

Prompt engineering — context injection, few-shot prompting, and guiding LLMs to exact test requirements

FitNesse for acceptance testing and collaborative documentation

Model Context Protocol (MCP) — using or developing MCP servers/clients to connect AI tools to external data sources and developer environments

TypeScript, Python, or Java to support agent tool calling and framework extensions

 Qualifications

• 3+ years of professional software QA automation or RPA development experience.
•  Strong understanding of capital markets, trading lifecycles, treasury workflows, or core banking architectures.
• Strong communication and collaboration skills to work across engineering, QA, and business teams.

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

Job Title: Test Automation Engineer

Experience Required: 2+ Yrs. of very recent experience

Work Location: Gurugram, Haryana (Regular Work from Office)

Work Timings: 04:30 PM 01:30 AM IST (Monday Friday)

Employer Company: Alldomainsoft.com Pvt. Ltd. (alldomainsoft.com)

US Client: Avive Solutions, Inc. (avive.life)

Note: Local candidates preferred. Unsuitable for Java based profiles.

About the Role

As a Test Automation Engineer at Avive, you will play a critical role in ensuring the quality and reliability of our software products. You will build and maintain automated test coverage using Python, Pytest, and BDD frameworks alongside Selenium, focused on validating web application behavior, data accuracy, user permissions, and simulated device communication protocols. Partnering closely with Software Engineering, you will help shape test strategy and expand coverage as new features and functionality are shipped.

What You'll Do including but not limited to:

1. Test Automation Development Design, build, and maintain automated test suites using Python, pytest, and BDD frameworks to validate web application functionality.

• Develop and maintain Selenium-based automation to verify data is displayed accurately across the website.
• Create automation that simulates device communication protocols to validate that data generated by connected devices is correctly received, processed, and reflected on the website.

2. Software Verification

• Expand and maintain automated regression suites that catch defects early and give the team confidence as new features are released.
• Identify gaps in existing test coverage and prioritize new automation based on risk and impact.
• Investigate test failures, determine root cause, and work with Software Engineering to resolve issues.

3. Process & Tooling

• Document, organize, and maintain test cases in Jira using Xray, ensuring traceability between requirements, test coverage, and results.
• Report, track, and prioritize bugs in Jira, working with Software Engineering through resolution.
• Evaluate and integrate new tools, libraries, or frameworks that improve test efficiency, coverage, or maintainability.
• Contribute to CI/CD pipeline integration for automated test execution.
• Continuously refine test strategy and best practices as the product and codebase evolve.

Who You Are

• BS/B.Tech./B.E. in Computer Science, Software Engineering, or a related field (equivalent experience considered).
• 2+ years of experience in software test automation.
• Strong proficiency in Python, with hands-on experience using pytest and BDD frameworks (e.g., pytest-bdd, Behave, or Cucumber)
• Solid experience building and maintaining automated UI tests with Selenium.
• Experience testing web applications, including validating data accuracy, user permissions/access control, and integrations with external or simulated data sources.
• Familiarity with Jira and Xray (or similar tools) for bug tracking and test case management.
• Comfortable working with APIs and simulating device or system communication protocols for test purposes.
• Exposure to CI/CD pipelines and integrating automated tests into a continuous delivery workflow is a plus.
• Strong analytical and debugging skills, with the ability to trace issues from symptom to root cause.
• Organized, independent, and results-oriented, with the ability to manage multiple test efforts simultaneously.
• Excellent communication skills and strong attention to detail, with a collaborative, process-oriented approach to working with Software Engineering.

LinkedIn: www.linkedin.com/in/rkd94

Apply Link: https://jobs.lever.co/AviveSolutions/c20d1002-7998-4e8b-9209-0a4c590d0c7c

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

What you'll own

The quality charter. Define and evolve the testing strategy — web application, dashboards, APIs, webhooks, and customer integrations. Establish release criteria, coverage expectations, and quality gates that engineers actually respect because they're fast and they catch real bugs. Automation at scale. Build and maintain end-to-end and API automation suites that run on every PR and give a trustworthy signal. Own the flake rate. Design for parallel execution, deterministic test data, and fast feedback — a suite nobody trusts is worse than no suite. Performance and reliability. Design and run load, stress, and soak tests against APIs and the platform. Profile results, isolate bottlenecks with the engineers who own the service, and set up the monitoring that tells us about degradation before a customer does. Leverage through tooling, including AI. Use whatever gets coverage up and cycle time down: LLM-assisted test case generation from specs, AI-assisted authoring and maintenance of automation, synthetic test data generation, log and failure triage. We expect you to be opinionated about where these tools genuinely help and where they produce impressive-looking noise. Raising the bar. Partner with product and engineering from spec through release so quality risk is surfaced during design, not after code freeze. Mentor the QA team and build the practices — bug triage discipline, root cause analysis, prevention over detection — that outlast any single release.

What we're looking for

Must have

3–6 years in QA roles, with meaningful ownership of automation frameworks rather than only executing existing suites.

Strong test automation skills with a modern framework — Playwright, Cypress, or Selenium — and solid coding ability in Python, JavaScript/TypeScript, or Java. You should be comfortable reading application code, not just testing around it.

API testing experience: REST, auth, webhooks and asynchronous event flows, contract testing, and tooling such as Postman, REST Assured, or Pact.

Hands-on CI/CD integration — GitHub Actions, GitLab CI, Jenkins, or equivalent — including making suites fast and reliable enough to gate merges.

Working knowledge of SQL for validating data correctness, and comfort with logs, traces, and observability tooling for debugging.

Rigorous, systematic thinking about edge cases and failure modes, and the communication skills to make quality risk legible to product and leadership.

Nice to have

Experience testing AI, ML, or LLM-based products, and comfort with the reality that outputs are probabilistic rather than deterministic. Familiarity with Docker, Kubernetes, or cloud infrastructure for standing up test environments.

Performance and load testing with k6, JMeter, Gatling, or Locust, plus the analytical ability to turn results into a specific, actionable bottleneck. Exposure to enterprise SaaS with real integration surface area, or to security and compliance testing expectations of enterprise buyers. - Experience as the first or most senior quality voice on a team.

What we offer

Competitive compensation with equity. Direct ownership of a charter that materially affects how enterprise customers experience our product.

Proximity to frontier AI systems in production, and the freedom to use modern tooling to do your job well.

A fast-moving team where the distance between an idea and a shipped change is short.