Admissions Cousellor

AI/LLM Test Automation Engineer (SDET)

Location: Bangalore (Hybrid preferred)

Experience: 5-8 Years

Job Summary

We are seeking a Senior Test Automation Engineer (SDET) with expertise in AI/LLM testing and Azure DevOps CI/CD to build robust automation frameworks for cutting-edge AI applications. The role combines deep programming skills (Java/Python), modern DevOps practices, and specialized LLM testing to ensure high-quality AI product delivery.

Key Responsibilities

• Design, develop, and maintain automation frameworks using Java/Python for web, mobile, API, and backend testing.
• Create and manage YAML-based CI/CD pipelines in Azure DevOps for end-to-end testing workflows.
• Perform AI/LLM testing including prompt validation, content generation evaluation, model behavior analysis, and bias detection.
• Write and maintain BDD Cucumber feature files integrated with automation suites.
• Execute manual + automated testing across diverse application layers with focus on edge cases.
• Implement Git branching strategies, code reviews, and repository best practices.
• Track defects and manage test lifecycle using ServiceNow or similar tools.
• Conduct root-cause analysis, troubleshoot complex issues, and drive continuous quality improvements.

Mandatory Skills & Experience

✅ 5+ years SDET/Automation experience

✅ Java/Python scripting for test frameworks (Selenium, REST Assured, Playwright)

✅ Azure DevOps YAML pipelines (CI/CD end-to-end)

✅ AI/LLM testing (prompt engineering, model validation, RAG testing)

✅ Cucumber BDD (Gherkin feature files + step definitions)

✅ Git (branching, PRs, GitFlow)

✅ ServiceNow/Jira defect tracking

✅ Manual + Automation testing (web/mobile/API/backend)

Technical Stack

Programming: Java, Python, JavaScript

CI/CD: Azure DevOps, YAML Pipelines

Testing: Selenium, Playwright, REST Assured, Postman

BDD: Cucumber (Gherkin), JBehave

AI/ML: Prompt validation, LLM APIs (OpenAI, LangChain)

Version Control: Git, GitHub/GitLabDefect

Tracking: ServiceNow, Jira, Azure Boards

Preferred Qualifications

• Exposure to AI testing frameworks (LangSmith, Promptfoo)
• Experience with containerization (Docker) and Kubernetes
• Knowledge of performance testing for AI workloads
• AWS/GCP cloud testing experience
• ISTQB or relevant QA certifications

What We Offer

• Work on next-gen AI/LLM products with global impact
• Modern tech stack with Azure-native DevOps
• Flexible hybrid/remote work model
• Continuous learning opportunities in AI testing

Company Cashflo

Salary: Upto 60LPA

ESOPs

Exp: 5-8Y

We are looking for someone who loves a challenge, is ambitious, super tenacious, and

persistent. S/he is a self-starter, thrives in a dynamic, small start-up environment, has a knack for

understanding customer needs, and loves to get sh*t done! If you check these boxes - we want

to talk to you!

Responsibilities:

Work closely with the Founder/CEO and to define and build the CashFlo platform

Identify customer problems, and latent needs & devise a product roadmap

Communicate and evangelize product output through low-fidelity designs

Collaborate with Sales and Customer Success to understand and eliminate obstacles to

adoption and usage

Work with Marketing to bring new features to life on various communication touchpoints

Collaborate with engineers and designers to arrive at detailed solutions to problems

Execute

Own the roadmap for one or more parts of the product

Work with engineering and design to plan launch scope

Together with engineering and design, break-down scope into launch iterations

Deliver product features as per timelines

Requirements:

Strong communication (written & verbal) and interpersonal skills

You should have worked at a startup or started your own company that shipped a product

An engineering / technical background that will allow you to reason about product

trade-offs and make good choices without significant input from engineers

At least 5 years of experience in a Product Management role

Attention to detail - great products are all about the details

Passion for getting the job done - A combination of grit, hustle and problem-solving skills

to deliver results on time

Prior exposure to the Fintech landscape and understanding of trends in financial services

• Excellent front end/UI skills (JS, HTML, Angular)
• Understanding of nodeJS, server-side technologies,exposure to databases.
• Deep knowhow of data structures, algorithms.
• Hands on development across technologies.

Primary Customer Facing Responsibilities:

• Handle customer service and support tickets efficiently, acting as the first point of contact over phone or email.
• Proactively communicate with customers regarding post-purchase support, including order installation, refunds, and payment issues.
• Utilize our client ticketing system to ensure prompt and effective resolution of customer queries.

Key Responsibilities:

Case Analysis and Critical Thinking:

• Develop comprehensive knowledge of client products and the ticketing system.
• Conduct thorough investigations to fully understand user issues, employing effective probing techniques.

Problem Solving:

• Provide accurate information and solutions for client software products or services.
• Offer alternative solutions when necessary, guiding users through the resolution process.

Post-Resolution Follow-Up:

• Ensure customer satisfaction by following up and updating customer status before case closure.

Client and Operational Responsibilities:

• Coordinate with team leads and managers for guidance on escalated cases.
• Record detailed events and problem resolutions in system logs.
• Forward customer feedback and suggestions to the appropriate internal team.
• Suggest improvements to processes and knowledge resources.
• Participate actively in team meetings and maintain effective communication with internal teams.
• Report to client leads and managers as required.

Requirements:

• Experience in help desk, software product support, and customer service.
• Tech-savvy with knowledge of computer operating systems, software, and hardware.
• Excellent written and verbal communication skills in English.
• Degree in a relevant field preferred.
• Proficient with Microsoft Office, Google Sheets, and other business software.
• Own a desktop/laptop with a stable internet connection.
• Demonstrated proactive, learning-oriented approach, with a focus on continuous process improvement.

Additional Information:

• This is a fully remote position with a 5-day work week.
• Requires approximately 9 hours of work per weekday.
• Compensation is competitive and billed hourly.
• Opportunity for long-term growth and additional responsibilities within the organization.

Our Commitment:

We value diversity and are committed to creating an inclusive environment for all employees. We encourage candidates of all backgrounds to apply.

Benefits:

• Work-from-home flexibility.
• Career advancement opportunities and professional development support.
• Supportive and collaborative team environment.

Application and Selection Process:

Initial Application:

• Interested candidates should submit their resume along with a cover letter detailing their relevant experience and why they are a good fit for this role via the Cutshort Application Portal.

Written Assessment:

• Selected candidates will be invited to complete a written assessment. This assessment is designed to evaluate technical and customer service skills and must be completed within 2 hours of commencement to ensure authenticity. Instructions and a deadline for the assessment will be provided upon selection.

Virtual Interview:

• Candidates who successfully pass the written assessment will be invited to a virtual interview with key stakeholders. This round will focus on assessing cultural fit, communication skills, and problem-solving abilities.

Onboarding and Training:

• Candidates who clear the interview stage will proceed to onboarding and training, marking the final stage of the selection process. This phase will familiarize them with company policies, tools, and the specific responsibilities of their role.

About Inevolution

Founded in 2009, InEvolution is a dynamic and evolving firm specializing in back-office, operations, and customer support services. We are a globally oriented team committed to delivering quality services with cost efficiency. Our ethos is to constantly adapt to the changing needs of the industry, effectively serving as an extension of our clients' teams.

We specialize in alleviating operational burdens from organizational leaders, allowing them to focus on growth. Our expertise spans across 24/7 client support, advanced technology utilization, proficient data management, and providing certified solutions. Our services include data hygiene, GDPR compliance, e-commerce campaign configuration, order management, reporting, sales support, and comprehensive customer support across various platforms. With our diverse experience in multiple business sectors, we offer robust solutions that cater to the unique needs of our clients.

Company Web : https://inevolution.in/

What The Role Is

We are looking for an GRC Operations Officer based in Chennai. This is a new role within the growing IT Compliance function, where you will be responsible for handling audits, implementation of information security policies etc,. The successful candidate will be comfortable working with the team on implementing frameworks and providing support for internal and external stakeholders. Reporting to the IT Compliance Officer for our Chennai team, this role is integral to the successful growth of the team as well as wider company performance.  

What You’ll Do

• Contribute and assist with continuous improvement of company policies, practices, and procedures
• Review, modify and maintain existing practices and policies to reflect our operations and values within specific industry-standard frameworks like ISO and NIST, among others
• Provide support for internal and third-party audits
• Respond to due diligence and TPRM requests from customers and other interested parties.
• Support internal staff with GRC-related questions and topics
• Develop, maintain and execute awareness programs
• Be a local representative of the company’s GRC group and manage the physical security requirements for the location
• Work independently and prioritize multiple tasks and adapt to needed changes
• Effectively communicate risks to diverse audiences, both in writing and verbally
• Apply a risk-based approach to planning, executing, and reporting on audit engagements and auditing process;

What You’ll Bring

• 2-5 years IT Security, IT risk, IT auditing, and/or IT Compliance experience within a technology company, accounting firm,  or others.
• Bachelor's degree or equivalent work experience working in compliance/GRC team.
• Exceptional organisational skills and attention to details.
• Knowledge of applicable domestic and internationally recognized information security management, governance, and compliance principles, practices, laws, rules and regulations;
• Information systems auditing, monitoring, controlling, and assessment process.

Perks & Benefits: 

• Competitive base salary
• Equity - every employee is a stakeholder in our enormous upside
• A tech-first company culture driven by entrepreneurial thinking and talent
• A great team working in unison towards the same mission
• Transparency is what our product is built on—and so is our culture
• Generous health insurance benefits for employees and their dependents
• Parental leave.
• Flexible work schedule and work-from-home options
• Flexible PTO

• Design and code feature in the front-end and back-end as per the specifications provided by the Product and Design teams.
• Ship features in fast-moving agile sprints.
• Work closely with Product Design and Product Management teams to brainstorm and chalk out details of features.
• Ability to build a feature from scratch & drive it to completion

Skills Must have:

• Good understanding of Javascript and Clean architecture.
• Good understanding of asynchronous programming, callback, and closures
• Experience in developing applications that use queues like SQS, RabbitMQ, Kafka, etc
• Experience with AWS services like S3, EC2, etc
• Strong experience with Javascript frontend framework/libraries like ReactJS
• Good knowledge of CSS and its preprocessors like SASS/LESS
• Good understanding of server-side templates.
• Experience with creating cross-browser compatible web apps
• Should be Strong in MySql / Postgres database
• Stronghold in Data structure and processing JSON

Good to have

• Basic knowledge of CI, CD
• Intermediate knowledge in any cloud services(AWS, GCP)
• Familiar with testing frameworks like Mocha, Cypress