Automation Testing with AIML

Principal Software Engineer

Location: Bangalore India

Department: AI Engineering / Quality Assurance

Experience Level: Mid to Senior

We are looking for a highly motivated Senior AI Test / Automation Engineer to design and scale automated validation frameworks for AI/ML models, LLM-based applications, and agentic systems. This role is critical to ensure that AI solutions meet enterprise standards for quality, reliability, safety, and compliance before and after production deployment.

Key Responsibilities

• Build and maintain AI test automation frameworks for pre-qualification and continuous validation of models and agent workflows
• Develop comprehensive test suites, including:
• Unit, integration, and end-to-end (E2E)
• Functional, regression, performance, and safety testing
• Validate AI system behavior, including:
• Non-deterministic LLM outputs
• Hallucinations and edge cases
• Multi-step agent decision-making
• Design and manage evaluation systems:
• Golden datasets
• Benchmarking pipelines (accuracy, latency, reliability)
• Automate testing within CI/CD pipelines for model updates, prompt changes, and tool integrations
• Implement observability and telemetry to enable traceability, monitoring, and audit readiness
• Collaborate cross-functionally with ML, MLOps, Product, and Security teams to define quality gates and release criteria
• Track and report quality KPIs, including test coverage, defect leakage, and system reliability
• Drive root-cause analysis and continuous improvement across the AI testing lifecycle

Required Skills

Core Engineering

• Strong programming skills in Python; familiarity with Bash, TypeScript, or Go
• Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress
• Proficiency in CI/CD tools (GitHub Actions, Jenkins, GitLab CI)
• Experience with cloud platforms (AWS, Azure, GCP) and containers (Docker, Kubernetes)

AI / ML & Agentic Systems

• Hands-on experience with LLM ecosystems (OpenAI, Anthropic, Bedrock)
• Familiarity with:
• RAG architectures and vector databases (Pinecone, Weaviate)
• Agent frameworks (LangChain, LlamaIndex, AutoGen)

AI Testing Techniques

• Experience with non-deterministic testing approaches (statistical assertions, tolerance thresholds)
• Knowledge of evaluation methods:
• LLM-as-a-judge
• BLEU, ROUGE, semantic similarity scoring
• Experience with prompt and agent regression testing
• Understanding of AI safety testing, including adversarial testing, bias/fairness validation, and jailbreak detection

Tooling (Preferred)

• AI testing & observability tools: LangSmith, TruLens, Arize, Weights & Biases
• Evaluation tools: DeepEval, Ragas, PromptFoo, Giskard
• Monitoring: Prometheus, Grafana, OpenTelemetry

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and cross-functional collaboration
• Data-driven mindset with focus on quality KPIs
• Detail-oriented with a strong bias toward automation and scalability

Experience Requirements

• 7+ years in QA, SDET, or test automation engineering
• Proven experience building and scaling automation frameworks
• Hands-on experience with AI/ML systems or LLM-based applications
• Experience testing RAG pipelines or agentic workflows
• Owned end-to-end AI test strategy and architecture
• Defined quality metrics and release gates
• Delivered scalable validation pipelines for production AI systems
• Supported audit and compliance readiness

Preferred

• Experience in enterprise or regulated environments (SOC2, ISO 27001, etc.)
• Exposure to:
• Shift-left testing practices
• Production observability and monitoring
• Chaos or resilience testing

Senior-Level Differentiators

Education

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field

Nice-to-have:

• ISTQB certification
• Cloud/ML certifications (AWS, Azure, GCP)
• AI testing certifications

What Success Looks Like

• AI systems that are accurate, reliable, and safe
• Fully automated test pipelines integrated into CI/CD
• Measurable improvements in defect leakage and model quality
• Strong observability and auditability across AI systems
• Scalable validation frameworks supporting rapid AI innovation

Senior Automation Testing Engineer

Location: Bengaluru / Gurugram – Onsite

Experience: 5–11 Years

Employment Type: Full-Time

Key Responsibilities

• Develop and maintain automation test frameworks and scripts.
• Perform both manual and automation testing, including exploratory, functional, regression, and bug verification.
• Develop test cases and contribute to test planning, estimation, and Agile ceremonies.
• Perform non-functional testing for accuracy, reliability, and performance.
• Drive QA process and automation improvements.

Required Skills

• BE/B.Tech/ME/M.Tech in CS/ECE/EE or related field.
• 5–11 years of software testing experience.
• Strong hands-on experience with Python/Java, Selenium, Cucumber, and PyTest.
• Good knowledge of SDLC, testing methodologies, defect lifecycle, and risk assessment.
• Experience with version control and bug-tracking tools.

Good to Have: Linux, Qt/QML, Squish, ISTQB/CSM/PSM, and regulatory/compliance standards.

Strong Senior QA/SDET Profile with hands-on automation building and QA-function ownership

2

Mandatory (Experience 1): Must have 6+ years in QA/test automation/SDET roles with clear progression in scope (Engineer to Senior to Lead)in a B2B SaaS product company

3

Mandatory (Experience 2): Must have direct experience scaling a QA function at a SaaS company and building automation frameworks or QA processes from scratch

4

Mandatory (Tech skill 1): Must have hands-on UI automation experience with Selenium (Java + TestNG) or Playwright / Cypress (JavaScript / TypeScript) using maintainable design patterns like Page Object Model

5

Mandatory (Tech skill 2): Must have API testing & automation experience with Postman and RestAssured (or equivalent) for functional, integration, and contract testing

6

Mandatory (Tech skill 3): Must have experience with CI/CD integration (Jenkins or GitHub Actions) and Git for version control

7

Mandatory (Skill 1): Must have a strong grasp of Agile / Scrum delivery and JIRA for defect and test management

8

Mandatory (Skill 2): Must have experience leading or mentoring QA engineers and collaborating cross-functionally with developers and product teams

9

Mandatory (Skill 3): Must have solid experience using AI in test automation and manual testing exposure

10

Mandatory (Communication): Must have strong communication skills for cross-functional collaboration

11

Mandatory (Company): Must come from a B2B SaaS

We are seeking a forward-thinking Mid-Level QA Automation Engineer specializing in RPA, financial systems validation, and AI engineering. You will build and scale end-to-end automation workflows using UiPath for our financial services platform. Additionally, you will pioneer our testing evolution by leveraging prompt engineering, creating custom AI agents, and implementing Model Context Protocol (MCP) servers to bridge AI models with our testing infrastructure.

Key Responsibilities and Duties

   RPA Automation:

·    Design, develop, and maintain robust automated testing workflows using UiPath Studio.

 AI Agent Engineering:

·   Architect, build, and orchestrate custom AI agents to autonomously generate,execute, and self-heal test scripts.

Prompt Engineering:

·   Design, optimize, and manage advanced prompt templates to drive deterministic,high-quality code and test case generation from LLMs.

 MCP Integration:

·    Implement and configure Model Context Protocol (MCP) ecosystems to securely connect AI agents with local data, development tools, and testing environments. 

 Financial System Testing:

·   Validate end-to-end trade life cycles, market data processing, and capital market clearing workflows. 

Regression Orchestration

·   Manage continuous regression schedules via UiPath Orchestrator to ensure financial system stability. 

Mandatory skills

Project Mandatory Skills

Project Desired Skills

UiPath Studio and Orchestrator (UI/API automation activities)

Selenium or Playwright for web-based application testing

Building, deploying, and scripting autonomous AI agents for complex engineering workflows

Calypso treasury and capital markets platform

Prompt engineering — context injection, few-shot prompting, and guiding LLMs to exact test requirements

FitNesse for acceptance testing and collaborative documentation

Model Context Protocol (MCP) — using or developing MCP servers/clients to connect AI tools to external data sources and developer environments

TypeScript, Python, or Java to support agent tool calling and framework extensions

 Qualifications

• 3+ years of professional software QA automation or RPA development experience.
•  Strong understanding of capital markets, trading lifecycles, treasury workflows, or core banking architectures.
• Strong communication and collaboration skills to work across engineering, QA, and business teams.

Staff Engineer – Automation & Manual Testing

Company: Stryker

Industry: Medical Devices / Healthcare Technology

Location: Gurugram– Onsite

Experience: 5–10 Years

Employment Type: Full-Time

Required Skills & Qualifications

• 4–10 years of experience in Software Testing, covering both Automation and Manual Testing.
• Strong hands-on experience in Automation Framework Development – Mandatory.
• Strong programming experience in either Java or Python.
• Hands-on experience with one or more of:
• Selenium
• PyTest
• Cucumber
• Java/Python-based automation frameworks
• Experience in designing, developing, maintaining, and enhancing automation frameworks.
• Strong understanding of Manual Testing, including test case design, execution, regression, exploratory testing, and defect verification.
• Good knowledge of SDLC, STLC, testing methodologies, defect lifecycle, and risk assessment.
• Experience with version control and defect/bug tracking tools.
• Good understanding of Agile/Scrum methodologies.
• BE/B.Tech/ME/M.Tech in Computer Science, Information Technology, Electronics, Electrical Engineering, or a related engineering discipline.

Preferred Skills

• Experience working with both Java and Python.
• Experience with Linux, Qt/QML, or Squish.
• Exposure to performance/non-functional testing.
• Experience in regulated or compliance-driven product development environments.
• Experience in the medical devices/healthcare domain.
• ISTQB, CSM, PSM, or other relevant certifications.

What We're Looking For

• Strong automation engineers who have built or significantly enhanced automation frameworks, rather than only executing existing scripts.
• Candidates with solid hands-on coding experience in Java OR Python.
• Professionals comfortable working across both automation and manual testing.
• Strong problem-solving, debugging, and collaboration skills.
• Candidates who can contribute to test strategy as well as hands-on implementation.

Share CV at : snigdhaattheratebeanhr.com

Required Skills

• Strong programming experience in Java
• Hands-on experience with Selenium WebDriver
• Experience in API Testing using REST Assured
• Good understanding of REST APIs and HTTP methods
• Experience with Cucumber BDD framework
• Strong knowledge of TestNG
• Experience in designing and maintaining automation frameworks
• Familiarity with BDD (Behavior-Driven Development) methodology
• Experience with Maven/Gradle and Git version control
• Knowledge of SQL and database validation
• Exposure to CI/CD tools such as Jenkins is an added advantage

Key Responsibilities

• Design, develop, and maintain robust automation test scripts using Java and Selenium.
• Automate REST API test cases using REST Assured.
• Create and maintain BDD feature files using Cucumber.
• Develop and execute automated regression, smoke, and functional test suites.
• Integrate automated test execution with CI/CD pipelines.
• Analyze test results, identify defects, and work closely with developers for resolution.
• Participate in requirement analysis, sprint planning, and test estimation.
• Maintain automation frameworks and improve test coverage.
• Ensure software quality by following best testing practices and coding standards.

Preferred Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Agile/Scrum project experience is preferred.

About the team

SecurITe’s mission is to build an Agentic‑AI driven security platform that protects critical infrastructure from modern cyber threats. Our focus is on delivering highly performant, resilient, and intelligent network security systems that help defenders stay ahead of adversaries.

About the Role

We’re looking for a seasoned Senior Quality Engineer to provide technical leadership and architectural oversight for our next‑generation cybersecurity AI platform. In this high-impact role, you will define the technical strategy for quality assurance, ensuring our agentic AI transforms cyber defense with unparalleled reliability.

You will be responsible for the end-to-end quality lifecycle, from architectural reviews to the deployment of scalable automation frameworks. Beyond technical execution, you will serve as a mentor to junior team members, fostering a culture of technical excellence and driving the strategy that ensures our solutions meet the rigorous demands of critical infrastructure protection.

What You’ll Do

●    Defining and driving comprehensive QA strategies and roadmaps for the cybersecurity platform.

●    Designing, developing, and executing test plans, test cases, and automated scripts to ensure software quality.

●    Performing functional, regression, performance, scalability and security testing to identify bugs or defects.

●    Collaborating with developers, product managers, and other stakeholders to understand product requirements and testing needs.

●    Identifying, documenting, and tracking software defects, ensuring clear communication of issues and their resolutions.

●    Leading deep-dive root-cause analysis for critical system defects and security vulnerabilities.

●    Conducting thorough reviews of product specifications and software design to identify potential areas of concern before testing.

●    Architecting and designing complex, scalable test automation frameworks to optimize CI/CD velocity.

●    Ensuring the software meets customer and business requirements by validating the functionality and performance.

●    Assisting in continuously improving QA processes, tools, and best practices to enhance software testing efficiency and effectiveness.

●    Supporting user acceptance testing (UAT) and assisting clients with product validation.

●    Mentoring junior and mid-level engineers, providing technical guidance and conducting architectural reviews.

Required Experience

●    A Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.

●    8-10 years of proven experience in quality engineering, specifically within network cybersecurity, Identity Providers, or AI-integrated platforms.

●    Expertise in manual and automated testing.

●    Deep domain expertise in complex system validation and advanced automation practices at scale.

●    Proficiency in programming languages like Python to build and run automated test scripts.

●    "Strong knowledge of software testing methodologies, performance testing tools (e.g., JMeter, k6), and security traffic generation/simulation tools (e.g., Ixia BreakingPoint, Scapy, or Snort/Suricata traffic generators)."

●    Understanding of continuous integration/continuous deployment (CI/CD) pipelines and version control systems like Git.

●    Strong communication skills for documenting test results and interacting with cross-functional teams.

●    Excellent analytical skills, attention to detail, and problem-solving ability.

●    Ability to work independently as well as collaboratively in a team environment.

●    A curious mindset with a willingness to quickly learn new technologies and testing tools.

Required Skills & Qualifications

●    Familiarity with cloud-based testing environments (GCP, AWS, Azure).

●    Experience with cybersecurity products or cloud services or IDP or Web UI

The Mindset

●    Problem Solver: You thrive on complex, ambiguous challenges and engineer elegant solutions.

●    Ownership‑Driven: You take initiative, move fast, and deliver outcomes without hand‑holding.

●    Continuous Learner: You stay ahead of the curve in AI, ML, and emerging technologies.

●    Startup DNA: You excel in fast‑moving environments where priorities evolve and impact is immediate.

QA Automation Engineer – Selenium + Java (5+ Years)

Experience: 5+ Years

Location: Noida (Work from Office/Hybrid as per client requirement)

Employment Type: Contract (1+ Year)

Working Hours: 11:00 AM – 8:30 PM IST

About the Role

We are looking for an experienced QA Automation Engineer with strong expertise in Selenium and Core Java to join our team. The ideal candidate should have hands-on experience in designing robust automation frameworks, developing scalable test libraries, and working on enterprise-level migration testing projects. This role requires excellent problem-solving skills and the ability to leverage AI-powered testing solutions to improve automation efficiency.

Key Responsibilities

Design, develop, and maintain enterprise-level automation frameworks for large-scale applications.

Build scalable, reusable, and maintainable automation libraries to improve testing efficiency.

Develop automation solutions for database migration and application migration testing.

Create comprehensive regression test scenarios using the Gherkin (BDD) language.

Convert UI-based workflows into efficient API-based automated test suites.

Integrate automation scripts into CI/CD pipelines for continuous testing.

Work closely with Developers, Business Analysts, and Product Owners to understand business requirements.

Execute functional, regression, integration, API, and end-to-end automation testing.

Utilize AI-powered testing tools and services (such as Azure Surface or similar platforms) to optimize testing workflows.

Perform root cause analysis for test failures and collaborate with development teams to resolve issues.

Ensure automation code follows coding standards, design principles, and best practices.

Develop and maintain automation documentation, knowledge base articles, and reusable assets.

Participate in code reviews and contribute to continuous improvement initiatives.

Support test data preparation and environment management.

Monitor automation execution and provide detailed test reports to stakeholders.

Required Skills

5+ years of experience in QA Automation.

Strong programming skills in Core Java.

Excellent understanding of Object-Oriented Programming (OOP) concepts.

Hands-on experience with design patterns and automation framework development.

Strong expertise in Selenium WebDriver.

Experience with BDD frameworks such as Cucumber.

Experience with Hibernate.

Strong understanding of Microservices architecture.

Experience in API testing using REST APIs.

Hands-on experience with Sybase and Microsoft SQL Server (MSSQL).

Experience working on database migration projects.

Good understanding of SQL queries, joins, stored procedures, and database validation.

Knowledge of Git for version control.

Experience using Maven for dependency management and build automation.

Hands-on experience with unit testing and integration testing frameworks.

Experience with mocking frameworks.

Exposure to AI-driven testing tools and intelligent test automation solutions.

Strong analytical and debugging skills.

Excellent written and verbal communication skills.

Preferred Qualifications

Experience with enterprise migration projects.

Exposure to cloud environments such as Microsoft Azure.

Familiarity with CI/CD tools like Jenkins or Azure DevOps.

Experience with Agile/Scrum development methodologies.

Knowledge of API automation frameworks.

Ability to mentor junior QA engineers and contribute to automation strategy.

Required Technical Skills

Core Java

Selenium WebDriver

Cucumber (BDD)

Gherkin

Hibernate

Microservices

REST API Testing

Sybase

MSSQL

Git

Maven

JUnit/TestNG

Mocking Frameworks

AI-based Test Automation

Regression Testing

Integration Testing

API Automation

Soft Skills

Excellent communication skills

Strong analytical thinking

Problem-solving ability

Team collaboration

Attention to detail

Ability to work independently in a fast-paced environment

Why Join Us?

Opportunity to work on enterprise-scale migration projects.

Exposure to AI-powered automation technologies.

Collaborative and technology-driven work environment.

Long-term contract opportunity with challenging projects.

Opportunity to work with modern QA automation frameworks and cloud technologies.

Experience: 5+ Years

Location: Noida

Contract Duration: 1+ Year

Working Hours: 11:00 AM – 8:30 PM IST

Job Opportunity: QA Automation Engineer at FrontM

Location: Bangalore preferred / Hybrid as applicable

Experience: 5–6 years in Quality Assurance roles

Education: B.E/B.Tech in Computer Science, Engineering or a related technical discipline

Salary: Above market standards, flexible for the right candidate

Career growth: Long-term opportunity with potential to lead test automation strategy and quality engineering

About FrontM

FrontM builds software platforms for frontline workforces operating in remote and low-connectivity environments, with a strong focus on the maritime industry. The platform supports communication, collaboration, healthcare, learning, welfare and operational workflows across mobile, web, kiosk and connected device environments.

The platform is used across web, iOS, Android and integrated services in demanding real-world conditions, requiring disciplined test strategy, reliable automation and close attention to quality at every release.

Role Summary

As a QA Automation Engineer, you will bridge the gap between manual testing and automated solutions to ensure the delivery of high-quality software. You will design, develop and execute automated test scripts for FrontM’s web, mobile and API interfaces, working closely with developers and product managers so that quality is built in from the start.

This role requires strong hands-on experience with modern testing frameworks including Playwright, Appium and Cucumber, and comfort using AI tools to accelerate test case generation and script creation.

Key Responsibilities

Test Strategy & Automation Frameworks (≈40%)

• Review product requirements and technical design documents to create comprehensive, well-structured test plans and test cases covering both manual and automated strategies
• Design, build and maintain scalable automated test frameworks for web applications using Playwright
• Design, build and maintain automated tests for mobile applications on iOS and Android using Appium
• Write clear, business-readable test scenarios and implement BDD frameworks using Cucumber
• Use modern AI tools such as ChatGPT, GitHub Copilot or specialised QA AI tools to generate test cases, write boilerplate automation scripts and identify edge cases

API, Integration & CI/CD Testing (≈40%)

• Perform robust API integration testing to ensure seamless communication between microservices and external endpoints
• Integrate automated tests into the CI/CD pipeline to ensure continuous quality and rapid feedback for the development team
• Maintain and extend regression suites so they remain reliable as the platform evolves

Defect Management & Collaboration (≈20%)

• Identify, record, document and track bugs thoroughly through to resolution
• Perform thorough regression testing when bugs are resolved
• Work closely with cross-functional teams, including developers and product managers, to identify system requirements and ensure quality is built in from the start

Required Technical Skills

Test Automation & Frameworks

• Proven work experience in software quality assurance, with a strong foundation in manual testing methodologies and a transition into automation
• Strong hands-on experience writing automated tests using Playwright for web and Appium for mobile
• Solid experience implementing and maintaining Cucumber for Behaviour-Driven Development

API, Programming & CI/CD

• Deep understanding of RESTful APIs and hands-on API integration testing experience using tools such as Postman, RestAssured or Playwright API testing
• Proficiency in at least one relevant programming language such as JavaScript, TypeScript, Java or Python, with TypeScript/JavaScript highly preferred for Playwright
• Experience with Git and CI/CD tools such as Jenkins, GitHub Actions or GitLab CI

AI-Assisted Testing & Quality Practices

• Demonstrated ability to prompt and use AI coding assistants and generative AI tools to build test scripts, generate test data and brainstorm test scenarios
• Understanding of Agile/Scrum development principles
• Experience with performance and security testing

Personal Attributes

• Strong ownership mindset for product quality and release readiness
• Methodical, detail-oriented approach to testing, documentation and defect tracking
• Clear written and spoken communication in English
• Ability to work independently and collaborate effectively with developers and product managers
• Comfortable working in fast-moving engineering teams
• Curiosity and willingness to adopt new tools, including AI, to improve QA efficiency

Why join FrontM?

Long-Term Career Growth

Opportunity to shape quality engineering for software used by global maritime and remote workforce customers, with scope to grow into test automation leadership roles.

Engineering Challenges That Matter

Test applications used in remote, low-bandwidth and operationally demanding environments, where reliability matters.

Broad Technical Ownership

Take responsibility across web, mobile and API testing, BDD, CI/CD integration and AI-assisted quality practices.

Apply now

Join a team focused on building reliable software for real-world use cases and help ensure the quality of systems used across the global maritime workforce.

Job Summary: QA Automation Lead – Playwright

Experience: 14+ Years

Location: Chennai / Pune

Notice Period: Immediate to 15 Days

Key Skills:

• Strong experience in QA Automation and Test Automation
• Hands-on experience with Playwright
• Strong experience in Java and Selenium
• Good knowledge of BDD & Cucumber
• Strong experience in SQL
• Experience in designing and maintaining automation frameworks
• Proven experience in a Lead role
• Strong understanding of automation testing methodologies and best practices

Roles & Responsibilities:

• Lead the design and development of automation frameworks using Playwright, Java, and Selenium.
• Develop and maintain BDD test scenarios using Cucumber.
• Perform test automation and database validation using SQL.
• Guide and mentor QA automation team members.
• Review test strategies, automation scripts, and test coverage.
• Collaborate with development and business teams to ensure quality throughout the SDLC.
• Identify automation opportunities and continuously improve testing processes.