Cutshort logo
For Employers
Mercor logo
AI Red-Teamer — Adversarial AI Testing (Advanced); English & Hindi
AI Red-Teamer — Adversarial AI Testing (Advanced); English & Hindi

AI Red-Teamer — Adversarial AI Testing (Advanced); English & Hindi at Mercor · Remote only · 0 - 10 years · ₹5L - ₹10L / yr · Remote only · Posted 17 Apr 2026

Halogion's logo

AI Red-Teamer — Adversarial AI Testing (Advanced); English & Hindi

Agency job
0 - 10 yrs
₹5L - ₹10L / yr
Remote only
Skills
Artificial Intelligence (AI)
Cyber Security
socio-technical probing
Penetration testing
exploit development
Reverse engineering

Location: Remote; Geography restricted to India

Type: Full-time or Part-time Contract Work

Fluent Language Skills Required: English & Hindi. Native-level fluency in English and Hindi is required for this position.

Why This Role Exists

At Mercor, we believe the safest AI is the one that’s already been attacked — by us. We are assembling a red team for this project - human data experts who probe AI models with adversarial inputs, surface vulnerabilities, and generate the red team data that makes AI safer for our customers.

This project involves reviewing AI outputs that touch on sensitive topics such as bias, misinformation, or harmful behaviors. All work is text-based, and participation in higher-sensitivity projects is optional and supported by clear guidelines and wellness resources. Before being exposed to any content, the topics will be clearly communicated.

What You’ll Do

  • Red team conversational AI models and agents: jailbreaks, prompt injections, misuse cases, bias exploitation, multi-turn manipulation
  • Generate high-quality human data: annotate failures, classify vulnerabilities, and flag systemic risks
  • Apply structure: follow taxonomies, benchmarks, and playbooks to keep testing consistent
  • Document reproducibly: produce reports, datasets, and attack cases customers can act on

Who You Are

  • You bring prior red teaming experience (AI adversarial work, cybersecurity, socio-technical probing)
  • You’re curious and adversarial: you instinctively push systems to breaking points
  • You’re structured: you use frameworks or benchmarks, not just random hacks
  • You’re communicative: you explain risks clearly to technical and non-technical stakeholders
  • You’re adaptable: thrive on moving across projects and customers

Nice-to-Have Specialties

  • Adversarial ML: jailbreak datasets, prompt injection, RLHF/DPO attacks, model extraction
  • Cybersecurity: penetration testing, exploit development, reverse engineering
  • Socio-technical risk: harassment/disinfo probing, abuse analysis, conversational AI testing
  • Creative probing: psychology, acting, writing for unconventional adversarial thinking

What Success Looks Like

  • You uncover vulnerabilities automated tests miss
  • You deliver reproducible artifacts that strengthen customer AI systems
  • Evaluation coverage expands: more scenarios tested, fewer surprises in production
  • Mercor customers trust the safety of their AI because you’ve already probed it like an adversary

Why Join Mercor

  • Build experience in human data-driven AI red teaming at the frontier of safety
  • Play a direct role in making AI systems more robust, safe, and trustworthy

The contract rate for this project will be aligned with the level of expertise required, the sensitivity of the material, and the scope of work. Competitive rates commensurate with experience.

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Mercor

Founded
Type
Size
Stage

About

N/A

Company social profiles

N/A

Similar jobs (10)

Resources Valley
Actively reviewing applications
at Resources Valley
1 recruiter
Manind Gupta
Posted by Manind Gupta
Jaipur
1 - 4 yrs
₹2L - ₹8L / yr
Cyber Security
cyber

Cyber Security Expert


We are seeking a highly skilled Cyber Security Expert with strong expertise in AI-driven

security tools and ethical hacking techniques. The ideal candidate will safeguard our digital

infrastructure, proactively identify vulnerabilities, and design intelligent defense mechanisms

against evolving cyber threats.

Key Responsibilities

• Threat Analysis: Identify, assess, and mitigate potential risks across systems and

networks.

• AI Security Tools: Deploy and manage AI-based solutions for intrusion detection,

anomaly monitoring, and predictive threat modelling.

• Ethical Hacking: Conduct penetration testing, simulate cyber-attacks, and recommend

remediation strategies.

• Incident Response: Lead investigations, contain breaches, and implement recovery

measures.

• Compliance & Governance: Ensure adherence to data protection laws, industry

standards, and organizational policies.

• Training & Awareness: Educate employees on best practices and emerging threats.


Required Skills & Qualifications

• Proven hands-on experience with AI-powered security platforms (e.g., SIEM, SOAR,

ML-based anomaly detection).

• Strong knowledge of ethical hacking tools (Metasploit, Burp Suite, Kali Linux, etc.).

• Expertise in network security, firewalls, IDS/IPS, and endpoint protection.

• Familiarity with cloud security (AWS, cloud platform).

• Proficiency in scripting languages (Python, Bash, PowerShell) for automation.

• Solid understanding of cryptography, authentication protocols, and secure coding

practices.



Read more
Cadence Design Systems
at Cadence Design Systems
2 recruiters
Furkan Khan
Posted by Furkan Khan
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
2 - 10 yrs
₹15L - ₹35L / yr
skill iconPython
CI/CD
Google Cloud Platform (GCP)
Large Language Models (LLM) tuning
Artificial Intelligence (AI)
+1 more

Principal Software Engineer


Location: Bangalore India

Department: AI Engineering / Quality Assurance

Experience Level: Mid to Senior


We are looking for a highly motivated Senior AI Test / Automation Engineer to design and scale automated validation frameworks for AI/ML models, LLM-based applications, and agentic systems. This role is critical to ensure that AI solutions meet enterprise standards for quality, reliability, safety, and compliance before and after production deployment.


Key Responsibilities

  • Build and maintain AI test automation frameworks for pre-qualification and continuous validation of models and agent workflows
  • Develop comprehensive test suites, including:
  • Unit, integration, and end-to-end (E2E)
  • Functional, regression, performance, and safety testing
  • Validate AI system behavior, including:
  • Non-deterministic LLM outputs
  • Hallucinations and edge cases
  • Multi-step agent decision-making
  • Design and manage evaluation systems:
  • Golden datasets
  • Benchmarking pipelines (accuracy, latency, reliability)
  • Automate testing within CI/CD pipelines for model updates, prompt changes, and tool integrations
  • Implement observability and telemetry to enable traceability, monitoring, and audit readiness
  • Collaborate cross-functionally with ML, MLOps, Product, and Security teams to define quality gates and release criteria
  • Track and report quality KPIs, including test coverage, defect leakage, and system reliability
  • Drive root-cause analysis and continuous improvement across the AI testing lifecycle


Required Skills

Core Engineering

  • Strong programming skills in Python; familiarity with Bash, TypeScript, or Go
  • Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress
  • Proficiency in CI/CD tools (GitHub Actions, Jenkins, GitLab CI)
  • Experience with cloud platforms (AWS, Azure, GCP) and containers (Docker, Kubernetes)

AI / ML & Agentic Systems

  • Hands-on experience with LLM ecosystems (OpenAI, Anthropic, Bedrock)
  • Familiarity with:
  • RAG architectures and vector databases (Pinecone, Weaviate)
  • Agent frameworks (LangChain, LlamaIndex, AutoGen)

AI Testing Techniques

  • Experience with non-deterministic testing approaches (statistical assertions, tolerance thresholds)
  • Knowledge of evaluation methods:
  • LLM-as-a-judge
  • BLEU, ROUGE, semantic similarity scoring
  • Experience with prompt and agent regression testing
  • Understanding of AI safety testing, including adversarial testing, bias/fairness validation, and jailbreak detection

Tooling (Preferred)

  • AI testing & observability tools: LangSmith, TruLens, Arize, Weights & Biases
  • Evaluation tools: DeepEval, Ragas, PromptFoo, Giskard
  • Monitoring: Prometheus, Grafana, OpenTelemetry

Soft Skills

  • Strong analytical and problem-solving skills
  • Excellent communication and cross-functional collaboration
  • Data-driven mindset with focus on quality KPIs
  • Detail-oriented with a strong bias toward automation and scalability

Experience Requirements

  • 7+ years in QA, SDET, or test automation engineering
  • Proven experience building and scaling automation frameworks
  • Hands-on experience with AI/ML systems or LLM-based applications
  • Experience testing RAG pipelines or agentic workflows
  • Owned end-to-end AI test strategy and architecture
  • Defined quality metrics and release gates
  • Delivered scalable validation pipelines for production AI systems
  • Supported audit and compliance readiness

Preferred


  • Experience in enterprise or regulated environments (SOC2, ISO 27001, etc.)
  • Exposure to:
  • Shift-left testing practices
  • Production observability and monitoring
  • Chaos or resilience testing

Senior-Level Differentiators


Education

  • Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field

Nice-to-have:

  • ISTQB certification
  • Cloud/ML certifications (AWS, Azure, GCP)
  • AI testing certifications

What Success Looks Like

  • AI systems that are accurate, reliable, and safe
  • Fully automated test pipelines integrated into CI/CD
  • Measurable improvements in defect leakage and model quality
  • Strong observability and auditability across AI systems
  • Scalable validation frameworks supporting rapid AI innovation



Read more
Securite Technology
Actively reviewing applications
Vijay Rohom
Posted by Vijay Rohom
Bengaluru (Bangalore)
8 - 10 yrs
₹26L - ₹40L / yr
IDP
User Interface (UI) Design

About the team


SecurITe’s mission is to build an Agentic‑AI driven security platform that protects critical infrastructure from modern cyber threats. Our focus is on delivering highly performant, resilient, and intelligent network security systems that help defenders stay ahead of adversaries.


About the Role

We’re looking for a seasoned Senior Quality Engineer to provide technical leadership and architectural oversight for our next‑generation cybersecurity AI platform. In this high-impact role, you will define the technical strategy for quality assurance, ensuring our agentic AI transforms cyber defense with unparalleled reliability.

You will be responsible for the end-to-end quality lifecycle, from architectural reviews to the deployment of scalable automation frameworks. Beyond technical execution, you will serve as a mentor to junior team members, fostering a culture of technical excellence and driving the strategy that ensures our solutions meet the rigorous demands of critical infrastructure protection.


What You’ll Do

●    Defining and driving comprehensive QA strategies and roadmaps for the cybersecurity platform.

●    Designing, developing, and executing test plans, test cases, and automated scripts to ensure software quality.

●    Performing functional, regression, performance, scalability and security testing to identify bugs or defects.

●    Collaborating with developers, product managers, and other stakeholders to understand product requirements and testing needs.

●    Identifying, documenting, and tracking software defects, ensuring clear communication of issues and their resolutions.

●    Leading deep-dive root-cause analysis for critical system defects and security vulnerabilities.

●    Conducting thorough reviews of product specifications and software design to identify potential areas of concern before testing.

●    Architecting and designing complex, scalable test automation frameworks to optimize CI/CD velocity.

●    Ensuring the software meets customer and business requirements by validating the functionality and performance.

●    Assisting in continuously improving QA processes, tools, and best practices to enhance software testing efficiency and effectiveness.

●    Supporting user acceptance testing (UAT) and assisting clients with product validation.

●    Mentoring junior and mid-level engineers, providing technical guidance and conducting architectural reviews.


Required Experience

●    A Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.

●    8-10 years of proven experience in quality engineering, specifically within network cybersecurity, Identity Providers, or AI-integrated platforms.

●    Expertise in manual and automated testing.

●    Deep domain expertise in complex system validation and advanced automation practices at scale.

●    Proficiency in programming languages like Python to build and run automated test scripts.

●    "Strong knowledge of software testing methodologies, performance testing tools (e.g., JMeter, k6), and security traffic generation/simulation tools (e.g., Ixia BreakingPoint, Scapy, or Snort/Suricata traffic generators)."

●    Understanding of continuous integration/continuous deployment (CI/CD) pipelines and version control systems like Git.

●    Strong communication skills for documenting test results and interacting with cross-functional teams.

●    Excellent analytical skills, attention to detail, and problem-solving ability.

●    Ability to work independently as well as collaboratively in a team environment.

●    A curious mindset with a willingness to quickly learn new technologies and testing tools.

Required Skills & Qualifications

●    Familiarity with cloud-based testing environments (GCP, AWS, Azure).

●    Experience with cybersecurity products or cloud services or IDP or Web UI


The Mindset

●    Problem Solver: You thrive on complex, ambiguous challenges and engineer elegant solutions.

●    Ownership‑Driven: You take initiative, move fast, and deliver outcomes without hand‑holding.

●    Continuous Learner: You stay ahead of the curve in AI, ML, and emerging technologies.

●    Startup DNA: You excel in fast‑moving environments where priorities evolve and impact is immediate.

 

Read more
NonStop io Technologies Pvt Ltd
Actively reviewing applications
Kalyani Wadnere
Posted by Kalyani Wadnere
Pune
4 - 7 yrs
Best in industry
Data Structures
Artificial Intelligence (AI)
skill iconMachine Learning (ML)
Scikit-Learn
TensorFlow
+4 more

About NonStop io Technologies

NonStop io Technologies is a value-driven company with a strong focus on process-oriented software engineering. We specialize in Product Development and have a decade's worth of experience in building web and mobile applications across various domains. NonStop io Technologies follows core principles that guide its operations and believes in staying invested in a product's vision for the long term. We are a small but proud group of individuals who believe in the 'givers gain' philosophy and strive to provide value in order to seek value. We are committed to and specialize in building cutting-edge technology products and serving as trusted technology partners for startups and enterprises. We pride ourselves on fostering innovation, learning, and community engagement. Join us to work on impactful projects in a collaborative and vibrant environment.


Brief Description:

We're seeking an AI/ML Engineer to join our team. As AI/ML Engineer, you will be responsible for designing, developing, and implementing artificial intelligence (AI) and machine learning (ML) solutions to solve real-world business problems. You will work closely with engineering teams, including software engineers, domain experts, and product managers, to deploy and integrate Applied AI/ML solutions into the products that are being built at NonStop io. Your role will involve researching cutting-edge algorithms and data processing techniques, and implementing scalable solutions to drive innovation and improve the overall user experience.


Responsibilities

● Applied AI/ML engineering; Building engineering solutions on top of the AI/ML tooling available in the industry today. Eg: Engineering APIs around OpenAI

● AI/ML Model Development: Design, develop, and implement machine learning models and algorithms that address specific business challenges, such as natural language processing, computer vision, recommendation systems, anomaly detection, etc.

● Data Preprocessing and Feature Engineering: Cleanse, preprocess, and transform raw data into suitable formats for training and testing AI/ML models. Perform feature engineering to extract relevant features from the data

● Model Training and Evaluation: Train and validate AI/ML models using diverse datasets to achieve optimal performance. Employ appropriate evaluation metrics to assess model accuracy, precision, recall, and other relevant metrics

● Data Visualization: Create clear and insightful data visualizations to aid in understanding data patterns, model behaviour, and performance metrics

● Deployment and Integration: Collaborate with software engineers and DevOps teams to deploy AI/ML models into production environments and integrate them into various applications and systems

● Data Security and Privacy: Ensure compliance with data privacy regulations and implement security measures to protect sensitive information used in AI/ML processes

● Continuous Learning: Stay updated with the latest advancements in AI/ML research, tools, and technologies, and apply them to improve existing models and develop novel solutions

● Documentation: Maintain detailed documentation of the AI/ML development process, including code, models, algorithms, and methodologies for easy understanding and future reference.


Qualifications & Skills

● Bachelor's, Master's, or PhD in Computer Science, Data Science, Machine Learning, or a related field. Advanced degrees or certifications in AI/ML are a plus

● Proven experience as an AI/ML Engineer, Data Scientist, or related role, ideally with a strong portfolio of AI/ML projects

● Proficiency in programming languages commonly used for AI/ML. Preferably Python

● Familiarity with popular AI/ML libraries and frameworks, such as TensorFlow, PyTorch, scikit-learn, etc.

● Familiarity with popular AI/ML Models such as GPT3, GPT4, Llama2, BERT etc.

● Strong understanding of machine learning algorithms, statistics, and data structures

● Experience with data preprocessing, data wrangling, and feature engineering

● Knowledge of deep learning architectures, neural networks, and transfer learning

● Familiarity with cloud platforms and services (e.g., AWS, Azure, Google Cloud) for scalable AI/ML deployment

● Solid understanding of software engineering principles and best practices for writing maintainable and scalable code

● Excellent analytical and problem-solving skills, with the ability to think critically and propose innovative solutions

● Effective communication skills to collaborate with cross-functional teams and present complex technical concepts to non-technical stakeholders

Read more
Mellivo AI
Actively reviewing applications
Sahil M
Posted by Sahil M
Thane, Mumbai
2 - 4 yrs
Best in industry
Cyber Security
skill iconPython
JSON
Malware analysis
Threat analysis

About Mellivo.ai

Mellivo.ai builds AI-powered knowledge automation and decision systems. We are also developing a privacy-first cybersecurity product to help people understand risks from suspicious apps, harmful links and device security settings. The product brings together on-device analysis, expert-authored detection rules and clear, actionable guidance.

Your Role

We are looking for a Cybersecurity Engineer with 2–4 years of experience to help design the security intelligence behind our PLATFORM. You will turn threat research into practical detection rules and user guidance, and work with developers to test and strengthen the product.

● Research threats, suspicious application behavior, malicious signing indicators, phishing links and common digital fraud patterns.

● Define explainable detection rules, required device facts, severity levels, evidence requirements and appropriate remediation guidance.

● Author and validate expert rules through the product’s restricted rule language and developer tools, in collaboration with backend engineers.

● Curate threat indicators and reviewed findings with source provenance, confidence, freshness and correction or revocation procedures.

● Design synthetic and authorized test cases; measure false positives and missed detections, and refine rules using reproducible evidence.

● Assess which signals are available on Android and iOS within OS permissions and privacy limits, and document coverage gaps.

● Threat-model the app, APIs and administration portal; perform authorized security testing and verify remediation with engineers.

● Review secure storage, authentication, access controls, signed updates and data flows to protect private device information.

Required Skills

Threat Research and Detection

● Practical knowledge of mobile threats, phishing, malicious URLs, application permissions and common abuse techniques.

● Ability to investigate indicators, distinguish suspicion from confirmed evidence, and write clear, testable detection logic.

● Experience with Python or a comparable scripting language for analysis, test fixtures and repeatable security checks.

Application Security

● Understanding of Android and iOS security models, application sandboxing, permissions and secure data handling.

● Hands-on web or API security testing, threat modeling and actionable vulnerability reporting.

● Familiarity with OWASP mobile and application security guidance, HTTP, TLS, authentication and authorization.

Preferred Skills

● Android application analysis using tools such as MobSF, JADX or apktool, and controlled dynamic analysis with Frida.

● Experience with Burp Suite, OWASP ZAP or Wireshark in authorized test environments.

● Threat intelligence normalization, rule engines, detection evaluation or investigation of fraud affecting users in India.

● Relevant practical certifications or published research, responsible disclosures and security projects; certifications are an advantage, not a requirement.

Who We Are Looking For

You investigate carefully, distinguish evidence from assumptions and communicate findings to both engineers and non-specialists. You care about false positives, user trust and responsible research, and can turn security knowledge into useful product behavior.

Qualifications

● 2–4 years of relevant cybersecurity experience, with hands-on work in threat research, detection engineering or application security.

● Bachelor’s degree in Cybersecurity, Computer Science, Information Technology, Engineering or a related field, or equivalent practical experience.

● Demonstrated work through security assessments, detection rules, research reports, product contributions or a relevant portfolio.

Why Join Mellivo.ai

Help define the detection capabilities of a privacy-first cybersecurity product. Work closely with mobile and backend engineers, turn research into understandable protection features, and influence how the product explains risk and supports safer user decisions.

Technology Stack

Research and Analysis: Python, JSON, threat indicators and controlled analysis environments

Security Testing: Burp Suite or OWASP ZAP, MobSF, JADX, Frida and Wireshark as appropriate

Product: Android and iOS security models, versioned expert rules and synthetic test fixtures

Wider Platform: REST APIs, C# and ASP.NET Core, PostgreSQL and a React developer portal

Read more
P99soft
Actively reviewing applications
Meghana Toluganti
Posted by Meghana Toluganti
Hyderabad
4 - 7 yrs
₹15L - ₹30L / yr
Generative AI
Artificial Intelligence (AI)
Large Language Models (LLM)
skill iconPython
API
+7 more

Title: AI Engineer - Product & Adoption Capability

Experience: 4-7 years

Location: Hyderabad

Mode of work: Hybrid


Role Summary:

P99Soft is hiring for an applied AI engineering role that also requires GTM-type capability - the ability to think about how AI tools actually get adopted by end users, not a dedicated go-to-market or marketing title. The engineer is expected to understand user workflows, spot adoption barriers, and shape how tooling gets rolled out, while remaining hands-on with applied AI work day to day.


Key Responsibilities:

  • Build and support the rollout of AI-powered tools, driving awareness, onboarding, and adoption among target end users.
  • Understand and map user workflows to identify where and why adoption of new AI tooling breaks down, and design solutions to close those gaps.
  • Partner closely with engineering and architecture stakeholders to translate what's technically possible into what users will actually adopt and use.
  • Gather and synthesize user feedback to inform product decisions and prioritization for the AI tooling roadmap.
  • Apply hands-on applied AI knowledge to evaluate tool capability and limitations firsthand, rather than relying solely on engineering input.
  • Communicate technical AI concepts to non-technical stakeholders in a way that drives buy-in and correct usage.


Required Experience & Skills:

  • Applied, hands-on AI experience is non-negotiable - genuine exposure to building or working closely with generative/agentic AI systems, not just using AI assistants as an end user.
  • Working GTM-type capability: comfort defining requirements, prioritizing based on user needs, and driving adoption of a tool - this can come from prior product exposure rather than a formal GTM title.
  • Strong user empathy and the ability to identify practical adoption barriers for enterprise tools, distinct from purely technical blockers.
  • Comfort operating with generative/applied AI systems (LLMs, agentic tools) as distinct from traditional data science approaches (regression, classification).
  • Background can come from startup, founder, or self-taught paths — a traditional data science or enterprise pedigree is not required, provided applied AI fundamentals are genuinely strong.


Nice to Have:

  • Some depth in AI systems internals (e.g., exposure to debugging model-serving or library/version compatibility issues) is a plus, though this role does not require building custom training pipelines from scratch.
  • Experience operating in a regulated, quality-driven, or highly technical enterprise environment.


Benefits

  • Competitive salary and performance-based bonuses.
  • Comprehensive health insurance.
  • Paid time off and flexible working hours.
  • Professional development opportunities and support for continuing education.
  • Udemy Learning Licenses to enhance skills and stay up-to-date with the latest technology.


P99Soft is an Equal-Opportunity Employer

P99soft is an equal-opportunity employer. At P99soft, we are committed to providing equal employment opportunities regardless of job history, disability, gender identity, religion, race, color, caste, marital/parental status, veteran status, or any other special status. We stand against the discrimination of employees and individuals and are proud to be an equitable workplace that welcomes individuals from all walks of life if they fit the designated roles and responsibilities.


Life @ P99Soft

At P99soft, we believe in creating a work environment that balances professional excellence with personal well-being. Our vibrant office culture is built on mutual respect, inclusivity, and a shared passion for innovation. We host regular team-building activities, workshops, and social events to foster a sense of community and collaboration among our employees. Join us and be part of a team that values your contributions and supports your growth.


About Us

P99soft is a leading IT Services and IT Consulting company dedicated to making our customers' business operations seamless by providing them with digital services and applications using the latest technologies. We are committed to bringing excellence to all our creations. We need passion, empathy, and being customer-focused to achieve technical excellence. We believe in transparency through open communication and work with honesty and integrity while dealing with our customers and partners.

Our Vision: “To lead digital transformation for our customers, delighting them at every step, while empowering our employees and fostering success.”

Our Mission: “To attain our objectives within a framework of integrity, transparency, and respect for our clients, employees, partners, and the community, fostering mutual trust and sustainable growth.”


Journey towards digital transformations begins here. Our talented teams help you make informed decisions, enhance operation efficiencies, create state-of-the-art visual interfaces, and design dynamic workflows to bring a better customer experience. Our engineering teams take complete ownership of assigning the right talent and architecting solutions using the latest technologies.

Read more
Techjays
Actively reviewing applications
at Techjays
4 candid answers
1 product
SriKrishna T
Posted by SriKrishna T
Remote, Coimbatore
2 - 10 yrs
Best in industry
Software Testing (QA)
Artificial Intelligence (AI)
Anthropic Claude
Prompt engineering
skill iconMachine Learning (ML)
+2 more

TECHJAYS

THE AI REIMAGINATION COMPANY

AI Quality Engineer


Position : AI Quality Engineer

Experience : 3–10 Years

Location : Coimbatore / Hybrid

Joining : Immediate / Within 30 Days


Primary SkillsManual & Functional Testing, AI/ML Application Testing, LLM Testing, Prompt Testing, Test Case Design, API Testing, Regression Testing, Defect Management

Preferred SkillsRAG Testing, Chatbot Testing, AI Response Evaluation, Selenium, Playwright, Cypress, Python, JavaScript, SQL, Postman, JIRA, BDD


About Techjays

At Techjays, we're helping businesses reimagine how they build, operate, and scale with AI at the core. We focus on developing intelligent, scalable, and production-grade AI solutions that solve real-world business challenges.

Our global team brings together experienced technology professionals from organizations such as Google, NetApp, ADP, Cognizant Consulting, and Capgemini. We work with global clients to build AI-powered applications, intelligent automation platforms, and enterprise software solutions.

We're looking for an AI Quality Engineer with strong functional testing expertise and prior hands-on experience testing AI/ML-powered applications.

This role goes beyond traditional software testing. You will be responsible for validating the quality, accuracy, reliability, and behaviour of AI-powered features, ensuring that intelligent applications perform consistently across different scenarios and real-world use cases.

The ideal candidate should understand both traditional QA methodologies and the unique testing challenges associated with AI systems, including non-deterministic outputs, prompt behaviour, hallucinations, response relevance, and edge-case validation.

Prior hands-on experience testing AI/ML-based products or applications is mandatory. Candidates with only traditional manual testing experience and no AI project exposure will not be a fit.

Key Responsibilities

1. AI Application & LLM Testing

  • Perform functional and quality testing of AI-powered applications, LLM-based products, chatbots, and intelligent automation systems.
  • Design test scenarios to validate AI-generated responses for accuracy, relevance, consistency, completeness, and contextual understanding.
  • Test prompts, prompt variations, and conversational flows to identify unexpected outputs and behavioural inconsistencies.
  • Evaluate AI responses for hallucinations, incorrect information, irrelevant answers, and context retention issues.
  • Validate AI application behaviour across different user inputs, edge cases, and complex scenarios.
  • Test RAG-based applications by validating document retrieval, response relevance, contextual accuracy, and source grounding.
  • Evaluate AI-powered search, recommendation engines, and other intelligent product features.
  • Collaborate with AI engineers and data science teams to define acceptance criteria and quality benchmarks.

2. Functional & Manual Testing

  • Analyse business and technical requirements and convert them into comprehensive test scenarios and test cases.
  • Execute functional, regression, integration, system, and user acceptance testing across web and mobile applications.
  • Validate end-to-end application workflows, business logic, user interactions, and data processing.
  • Identify functional gaps, unexpected behaviour, and usability issues.
  • Perform exploratory testing to uncover defects that may not be covered by predefined test cases.
  • Ensure application functionality remains stable across different releases and environments.
  • Review requirements early in the development lifecycle to identify potential quality risks.

3. AI Quality Evaluation & Validation

  • Develop structured testing approaches for non-deterministic AI outputs using behaviour-based, range-based, and scenario-based validation.
  • Evaluate AI response quality against expected outcomes and predefined acceptance criteria.
  • Validate prompt sensitivity, context handling, conversation memory, and multi-turn interactions.
  • Identify failure patterns involving hallucinations, prompt injection, incorrect retrieval, and inconsistent responses.
  • Test AI system performance across different input formats, languages, and user scenarios where applicable.
  • Maintain test datasets and evaluation scenarios for repeatable AI quality assessment.
  • Work with engineering teams to identify root causes of AI-related defects and verify fixes.

4. API & Integration Testing

  • Perform manual API testing using Postman or similar tools.
  • Validate API request and response structures, status codes, authentication, authorization, and error handling.
  • Test integrations between AI services, backend systems, databases, and external APIs.
  • Validate data flow between application components and identify integration issues.
  • Test API behaviour under different input conditions, including invalid inputs and boundary cases.
  • Verify API responses and ensure consistency between frontend and backend functionality.

5. Defect Management & Quality Reporting

  • Identify, document, track, and manage defects throughout the testing lifecycle.
  • Maintain detailed test cases, test execution reports, defect logs, and test documentation.
  • Collaborate with developers to reproduce issues, analyse root causes, and validate fixes.
  • Track test coverage, defect trends, and application quality metrics.
  • Communicate release risks, unresolved defects, and testing status to relevant stakeholders.
  • Participate in release readiness reviews and provide quality sign-off for assigned modules.

6. Automation & Continuous Improvement

  • Identify repetitive test scenarios that can be automated to improve testing efficiency.
  • Support automation initiatives using Selenium, Playwright, or Cypress.
  • Work with developers to improve testability and integrate testing practices into CI/CD workflows.
  • Explore AI-assisted testing tools and approaches to improve test coverage and efficiency.
  • Continuously improve testing methodologies for both conventional software applications and AI-powered systems.

Required Skills & Experience

1. Functional & Manual QA

  • 3–5 years of hands-on experience in manual and functional software testing.
  • Strong understanding of software testing methodologies, SDLC, STLC, and defect management.
  • Experience designing and executing test cases for functional, regression, integration, and end-to-end testing.
  • Strong knowledge of Agile/Scrum methodologies and sprint-based QA processes.
  • Experience testing web applications, APIs, and mobile applications.
  • Ability to analyse requirements and identify functional gaps, edge cases, and potential risks.

2. Mandatory AI/ML Testing Experience

  • Prior hands-on experience testing AI/ML-powered applications or products.
  • Practical experience testing LLM-based applications, AI chatbots, conversational systems, or intelligent automation features.
  • Understanding of prompt testing and evaluating AI-generated responses.
  • Ability to validate AI outputs for accuracy, relevance, consistency, and contextual correctness.
  • Experience identifying hallucinations, unexpected AI behaviour, and response-quality issues.
  • Understanding of AI application workflows, model outputs, and the challenges associated with non-deterministic systems.
  • Experience collaborating with AI/ML engineers or data science teams during testing and validation.

3. API & Integration Testing

  • Hands-on experience with manual API testing using Postman.
  • Understanding of REST APIs, JSON, request/response validation, authentication, and error handling.
  • Experience validating integrations between frontend, backend, and external services.
  • Basic understanding of database validation and SQL queries.

4. Testing Tools & Technologies

  • Experience with JIRA and Confluence for defect tracking and test documentation.
  • Familiarity with Postman, SQL, and browser developer tools.
  • Understanding of test management and reporting practices.
  • Basic knowledge of BDD frameworks such as Cucumber/Gherkin is an advantage.

Preferred Skills

  • Experience testing RAG-based applications, including document retrieval and contextual response validation.
  • Exposure to vector databases and semantic search testing.
  • Familiarity with LangChain, LangGraph, LlamaIndex, or similar AI frameworks.
  • Understanding of LLM evaluation frameworks and AI quality assessment methodologies.
  • Experience with AI evaluation tools such as Ragas, DeepEval, or similar frameworks.
  • Exposure to prompt injection, adversarial testing, and responsible AI validation.
  • Working knowledge of Selenium, Playwright, or Cypress.
  • Basic scripting knowledge in Python or JavaScript.
  • Familiarity with CI/CD pipelines and automated testing practices.
  • Exposure to cross-browser testing tools such as BrowserStack or Sauce Labs.

What We're Looking For Beyond Skills

  • AI Testing Mindset: Understands that testing AI applications requires more than checking whether a feature works. Can evaluate response quality, accuracy, relevance, and behaviour.
  • Analytical Thinking: Ability to identify unusual scenarios, inconsistent outputs, and hidden quality issues.
  • Strong QA Fundamentals: Applies structured testing methodologies while adapting them to AI-powered applications.
  • Attention to Detail: Documents defects clearly and identifies patterns in application and AI behaviour.
  • Ownership: Takes responsibility for testing quality from requirement analysis through release validation.
  • Collaboration: Works effectively with developers, AI engineers, product managers, and data science teams.
  • Continuous Learning: Interested in exploring emerging AI technologies and evolving testing methodologies.
  • Communication: Clearly communicates test results, defect impact, quality risks, and release readiness.

What We Offer

  • Competitive compensation aligned with experience and expertise.
  • Flexible work environment (Coimbatore office / Hybrid).
  • Paid holidays and flexible time off.
  • Medical insurance covering self and family, up to ₹4 lakh per person.
  • Opportunity to work on real-world AI-powered products and enterprise applications.
  • Hands-on exposure to LLMs, RAG systems, agentic AI, and modern AI engineering practices.
  • Opportunities to develop expertise in AI quality engineering and automation testing.
  • A collaborative, multicultural environment with opportunities for professional growth.
  • A culture that values clarity, integrity, ownership, and continuous learning.

Join Techjays

If you're passionate about software quality and have hands-on experience validating AI-powered applications, we'd love to hear from you.

Join us in building reliable, intelligent, and production-ready AI systems.

Read more
Appknox
Actively reviewing applications
at Appknox
1 video
6 recruiters
Vasudha Srivastav
Posted by Vasudha Srivastav
Bengaluru (Bangalore)
4 - 7 yrs
Best in industry
Bug Bounty
Exploit Development
Agent Driven Pentesting
Reverse engineering
Penetration testing
+6 more

A BIT ABOUT US

Appknox is one of the top Mobile Application security companies recognized by Gartner and G2. A profitable B2B SaaS startup headquartered in Singapore & working from Bengaluru.

The primary goal of Appknox is to help businesses and mobile developers secure their mobile applications with a focus on delivery speed and high-quality security audits.

Appknox has helped secure mobile apps at Fortune 500 companies with major brands spread across regions like India, South-East Asia, Middle-East, US, and expanding rapidly. We have secured 300+ Enterprises globally.

We are a 65+ incredibly passionate team working to make an impact and helping some of the biggest companies globally. We work in a highly collaborative, very fast-paced work environment. If you have what it takes to be part of the team, we are excited and let’s speak further.



The Opportunity

To join the security team engaging with multiple clients, helping them with end to end security audits, also research about new topics and vulnerabilities to be added to the scanner, present it in conferences.


What An Ideal Candidate Would Look Like: 

  • Skills - Application Penetration Testing (Web, iOS and Android), experience with IoT testing, source code audits.
  • Technology Stack: AWS, GCP, Objective C, Java, Python
  • Responsibilities: Engage with clients for scoping call, perform security audits, remediation call with clients to patch the issues, research on new technologies/vulnerabilities


Minimum Requirements

  • 2-4+ years of experience in application security and vulnerability research
  • Strong foundation in mobile app security - Android or Ios
  • Proven track record of discovering and disclosing CVEs in mobile platforms or popular applications (provide - github/bug bounty profiles)
  • Strong understanding of exploit mitigations and proven ability to bypass them
  • Should be able to architect automated detection logic for new vulnerabilities and integrate them into our security products
  • Develop AI-driven agents to automate dynamic analysis
  • Should be able to Leverage AI/LLMs to augment Pentesting efforts
  • Ability to work independently in a fast-paced environment, balancing deep research with practical deliverables


Good to have Requirements

  • Understanding of AI/ML security risks


Responsibilities

  • Security assessment of web/mobile applications on various platforms
  • Focusing on Mobile Application Security Research for new vulnerabilities
  • Static and Dynamic Code Analysis
  • Develop and interpret security standards and guides
  • Automation of exploit development 
  • Understand and explain the results with impact on business and compliance status
  • Continuously learning and training on latest tools and technique


Work Expectations

Within 1 month

Training on processes, security workflows and develop understanding on internal tools.


Within 3 months

Actively contributing in exploit development and automation of it with the team.


Within 6 months

Expected to achieve Subject Matter Expert status on our core security products. We expect you to validate your findings against real-world conditions, with a strong emphasis on translating internal discoveries into actionable bug bounty submissions and earned CVEs to benchmark your impact against the external security community.


Within 1 Year

By the end of your first year, you will be expected to produce and submit a piece of novel, peer-reviewed security research. This deliverable must be of a quality suitable for top-tier industry conferences.


Personality traits we really admire

  • A confident and dynamic working persona, which can bring fun to the team, and a sense of humour, is an added advantage.
  • Great attitude to ask questions, learn and suggest process improvements.
  • Has attention to details and helps identify edge cases.
  • Highly motivated and coming up with fresh ideas and perspectives to help us move towards our goals faster.
  • Follow timelines and absolute commitment to deadlines.


Interview Process - would be team specific

  • Round 1- CTF round 
  • Round 2 -Profile Evaluation; HR
  • Round 3 -Technical Interview with security team members
  • Round 4 -Technical Interview with the Hiring Manager
  • Round 5 -Technical round with CTO
  • Round 6 -HR Round


Compensation

  • As per Industry Standards


Why Join Us

  • Freedom & Responsibility: If you are a person who enjoys challenging work & pushing your boundaries, then this is the right place for you. We appreciate new ideas & ownership as well as flexibility with working hours.
  • Great Salary & Equity: We keep up with the market standards & provide pay packages considering updated standards. Also as Appknox continues to grow, you’ll have a great opportunity to earn more & grow with us. Moreover, we also provide equity options for our top performers.
  • Holistic Growth: We foster a culture of continuous learning and take a much more holistic approach to train and develop our assets: the employees. We shall also support you all on that journey of yours.
  • Transparency: Being a part of a start-up is an amazing experience, one of the reasons being open communication & transparency at multiple levels. Working with Appknox will give you the opportunity to experience it all first-hand.
Read more
Prentis
Actively reviewing applications
Anika fromTitanHolding
Posted by Anika fromTitanHolding
Bengaluru (Bangalore)
3 - 5 yrs
₹25L - ₹30L / yr
Large Language Models (LLM)
AI Agents
skill iconPython
Retrieval Augmented Generation (RAG)
REST APIs
+3 more

Product Engineer — Role Summary

We are looking for a Product Engineer to build user-facing products that combine AI capabilities with practical applications. You will work at the intersection of software engineering and AI, developing autonomous, agent-driven systems that solve complex educational and research problems.

Key Responsibilities:

  • Product Development: Design, build, and deploy production-ready applications powered by LLMs and AI agents.
  • Data Engineering: Build scalable ETL/ELT pipelines to process structured and unstructured data, including text and audio, for RAG and model fine-tuning.
  • Agentic Workflows: Develop multi-step AI agents with tool calling, APIs, databases, search, reasoning, and memory.
  • Rapid Prototyping: Turn ideas and research concepts into interactive, production-ready applications.
  • AI Integration: Use frameworks such as LangChain, LlamaIndex, AutoGen, or custom orchestrators to integrate AI into scalable systems.
  • User Experience: Transform raw AI outputs into reliable, intuitive, and responsive user experiences.
  • Collaboration: Work closely with ML researchers and data engineers to integrate custom and fine-tuned models.
  • Observability: Monitor agent behavior, manage edge cases, reduce hallucinations, and improve reliability in production.

The ideal candidate combines strong software engineering, AI/LLM expertise, data engineering, and product thinking, with the ability to take an AI concept from prototype to production.

Read more
Staffnixcom
Actively reviewing applications
Mayank Choudhary
Posted by Mayank Choudhary
Hyderabad , Indore, Ahmedabad
3 - 5 yrs
₹21L - ₹27L / yr
LLM

Strong Senior AI Engineer Profile with production-grade LLM and Python expertise

2

Mandatory (Experience 1): Must have at least 3+ years of professional AI/ML engineering experience with demonstrated track record of delivering production-grade AI systems in real-world environments

3

Mandatory (Tech skill 1): Must have strong programming skills in Python and SQL, with hands-on ML/data libraries (scikit-learn, pandas, numpy) and deep-learning frameworks (PyTorch or TensorFlow), plus REST API design

4

Mandatory (Experience 2): Must have hands-on experience building and deploying production-grade ML/LLMs including RAG pipelines, document parsing, information extraction, and text processing on large-scale unstructured data (preprocessing, chunking, embeddings, feature engineering)

5

Mandatory (Tech skill 2): Must have strong NLP / extraction-focused ML depth — transformers, embeddings, vector databases, RAG, LLM integrations, and agentic workflows.

6

Mandatory (Tech skill 3): Must have hands-on experience with AWS (SageMaker, Bedrock, EC2, Lambda) and Azure (AI Foundry, Azure OpenAI) for model training, fine-tuning, deployment, and inference at scale

7

Mandatory (Tech skill 4): Must have experience with multi-agentic frameworks / orchestration tools (Claude Code, LangGraph, LangChain, CrewAI)

8

Mandatory (Tech skill 5): Must have hands-on experience with MLOps ecosystem including experiment tracking (MLflow, Weights & Biases), model versioning/registry, automated retraining, and CI/CD (GitHub Actions, Azure DevOps, Docker, Kubernetes)

9

Mandatory (Tech skill 6): Must have experience with evaluation frameworks (precision, recall, F1, field-level accuracy) and iterative model improvement, plus AI observability (Prometheus, Grafana, SLOs).

10

Mandatory (Team Leadership): Must have experience leading projects or teams, managing technical deliverables, and mentoring, with strong problem-solving in ambiguous challenges

11

Mandatory (Education) - B.Tech or B.E. If graduated in BCA or BCS, they should have a Maters or Post graduation

12

Mandatory (Company): B2B Tech/Consulting/B2B Software

Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos