QA Automation Analyst - Contractual

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

Total exp: 5-8 Years

Client: MRI Software

Location: Vadodara

Budget: Look for decent with experience

Mode interview: 1 Virtual

Mode of Work: Hybrid

QA Engineer:

Job Summary/Objective 

Quality Assurance Engineers at MRI collaborate closely with Product Management and Product Development teams to deliver high-quality, high-performance products for our clients. In this role, QA Engineers are responsible for creating and contributing to the documentation of user stories and acceptance criteria, working alongside product owners and development teams. They define manual and automation testing strategies, tools, scenarios, and plans for both web and mobile products, ensuring comprehensive functional and performance testing. 

 QA Engineers will execute static, dynamic, and performance testing while providing detailed reporting and feedback throughout the development lifecycle. They are expected to demonstrate strong technical skills, including expertise in back-end development, database queries, and keen attention to detail. Effective written, verbal, and organizational communication is essential. 

 In addition, QA Engineers will leverage AI to optimize and automate repetitive tasks, thereby enhancing the overall efficiency and effectiveness of QA processes, ensuring a faster, more accurate testing cycle.

Tools and technology:

1.     Tools: Jira, Xray, TFVC, Git

2.     API: REST, SOAP, SoapUI, Postman

3.     Automation: C#, JavaScript, TypeScript, Ruby, Playwright, SpecFlow

4.     Database: SQL

About Us

We’re building the next generation of AI-powered business software, and we’re looking for people who want to shape that future with us. With Lumen, we’re reimagining how users interact with CRM — moving beyond screens, menus and dashboards to an intelligent interface where users can simply ask AI to take actions, retrieve knowledge, generate insights and get work done. With Agent Studio, we’re enabling businesses to build, test and deploy their own AI agents for real-world workflows. And with Invorto, we’re bringing AI to voice, allowing businesses to create intelligent voice agents tailored to their customer and operational use cases.

What makes this especially exciting is the stage and scale of the opportunity. You’ll get to work on genuinely hard problems across LLMs, agents, reasoning, orchestration, voice AI, evaluation, reliability and enterprise security — not as isolated experiments, but as products used in real business workflows. You’ll have the opportunity to build zero-to-one, own meaningful parts of the product end-to-end, work closely with customers, experiment rapidly, and see your work reach production at scale.

Why join now? Because the playbook for enterprise AI is still being written. You won’t just be implementing someone else’s roadmap — you’ll help define the product, architecture and experiences that become that playbook. Expect high ownership, fast iteration, hard technical and product problems, direct customer impact, and the chance to build AI systems that have to work reliably in the real world — not just in a demo.

About the Role

We are looking for a QA Engineer who specializes in testing agentic AI platforms. You will design and automate quality processes for systems that involve LLMs, autonomous agents, tool use and orchestration across Lumen and Agent Studio — ensuring that AI-driven workflows behave reliably, safely and predictably in production, not just in a demo.

What You’ll Do

• Design and build automated test suites and evaluation frameworks for agentic AI workflows, including multi-step and tool-calling behaviors.
• Use AI/LLM-based QA tools and evaluation frameworks to test model outputs, agent decisions and end-to-end task completion at scale.
• Define quality metrics and benchmarks for agent reliability, correctness, latency and safety, and track them over releases.
• Identify edge cases, failure modes and regressions specific to non-deterministic AI systems, and build automated checks to catch them early.
• Integrate automated agent/LLM testing into CI/CD pipelines to support fast, reliable iteration.
• Partner closely with AI/ML and backend engineers to reproduce issues, root-cause failures and validate fixes.
• Work with customers and customer-facing teams to understand real-world usage patterns and translate them into test scenarios.

What We’re Looking For

• 2–4 years of QA/test automation experience, including hands-on work testing agentic AI or LLM-based platforms.
• Practical experience using AI-focused QA/evaluation tools to test agent behavior, prompts and model outputs.
• Strong scripting/automation skills (Python preferred) to build and maintain test frameworks.
• Understanding of how LLM-based agents work — tool calling, orchestration, memory, reasoning chains — well enough to design meaningful test cases.
• Comfort working with non-deterministic systems and designing evaluation approaches beyond traditional pass/fail testing.
• Strong communication skills — this is a customer-facing role, and you will be expected to clearly articulate technical concepts, decisions and trade-offs to both technical and non-technical stakeholders, including customers.

Good to Have

• Experience testing voice AI or real-time conversational systems.
• Familiarity with CRM or enterprise SaaS platforms.
• Exposure to enterprise security or compliance testing for AI systems.

Job Description:

As a QA Associate at Binocs, you will play a pivotal role in ensuring the highest quality standards for our software products. With a direct reporting line to the QA Lead, you will be involved in designing, implementing, and executing test plans to identify software issues. This role is ideal for someone with a strong technical background in engineering, experience in fintech or finance software companies, and a passion for quality assurance and detail-oriented testing.

Key Responsibilities:

• Test Plan Development: Create comprehensive and well-structured test plans and test cases based on product specifications.
• Quality Assurance: Execute test cases (manual or automated) and analyze results to ensure software quality.
• Issue Documentation: Document and track bugs, as well as collaborate with the development team to ensure timely resolution.
• Process Improvement: Identify, recommend, and implement improvements in testing processes to enhance efficiency and quality.
• Software Validation: Validate that product features meet requirement specifications and are free of defects.
• Cross-functional Collaboration: Work closely with the product management and development teams to understand product vision and deliver software that aligns with customer and market needs.

Qualifications:

• Bachelor’s degree in Engineering, Computer Science, or related field.
• 2+ years of experience in quality assurance, preferably within fintech or finance software companies.
• Familiarity with software QA methodologies, tools, and processes.
• Experience in writing clear, concise, and comprehensive test plans and test cases.
• Hands-on experience with both white box and black box testing.
• Strong knowledge of software QA metrics, including bug tracking and reporting tools.
• Excellent analytical skills and attention to detail.
• Effective communication skills and the ability to work in a fast-paced environment.

Benefits:

Joining Binocs means being part of a passionate team dedicated to transforming the loan management landscape. We offer a dynamic, innovative environment where your work directly contributes to our vision of efficient, transparent, and data-driven loan management. Our competitive compensation package, including equity, reflects our commitment to attracting and retaining top talent.

About Us:

Binocs.co empowers institutional lenders with next-generation loan management software, streamlining the entire loan lifecycle and facilitating seamless interaction among stakeholders.

Team: Binocs.co is led by a passionate team with extensive experience in financial technology, lending, AI, and software development.

Investors: Our journey is backed by renowned investors who share our vision for transforming the loan management landscape: Beenext, Arkam Ventures, Accel, Saison Capital, Blume Ventures, Premji Invest, and Better Capital.

Required Skills:-

• 4+ years of experience in Software Testing (Manual & Automation).
• Strong experience with Selenium or Playwright.
• Hands-on experience in API Testing (Postman, REST Assured, Swagger, etc.).
• Experience with CI/CD pipelines (Jenkins, GitLab CI, Azure DevOps, GitHub Actions).
• Strong understanding of Functional, Regression, and Non-functional Testing.
• Excellent analytical and debugging skills.

AI-Specific Skills

• Hands-on experience in GenAI / AI Testing.
• Experience testing LLM-based applications.
• Strong understanding of prompt engineering and prompt validation.
• Knowledge of LLM behavior, model variability, and non-deterministic outputs.
• Experience validating AI outputs for accuracy, hallucinations, bias, and safety.
• Test data management for AI applications.
• Understanding of Responsible AI testing concepts.

About the team

SecurITe’s mission is to build an Agentic‑AI driven security platform that protects critical infrastructure from modern cyber threats. Our focus is on delivering highly performant, resilient, and intelligent network security systems that help defenders stay ahead of adversaries.

About the Role

We’re looking for a seasoned Senior Quality Engineer to provide technical leadership and architectural oversight for our next‑generation cybersecurity AI platform. In this high-impact role, you will define the technical strategy for quality assurance, ensuring our agentic AI transforms cyber defense with unparalleled reliability.

You will be responsible for the end-to-end quality lifecycle, from architectural reviews to the deployment of scalable automation frameworks. Beyond technical execution, you will serve as a mentor to junior team members, fostering a culture of technical excellence and driving the strategy that ensures our solutions meet the rigorous demands of critical infrastructure protection.

What You’ll Do

●    Defining and driving comprehensive QA strategies and roadmaps for the cybersecurity platform.

●    Designing, developing, and executing test plans, test cases, and automated scripts to ensure software quality.

●    Performing functional, regression, performance, scalability and security testing to identify bugs or defects.

●    Collaborating with developers, product managers, and other stakeholders to understand product requirements and testing needs.

●    Identifying, documenting, and tracking software defects, ensuring clear communication of issues and their resolutions.

●    Leading deep-dive root-cause analysis for critical system defects and security vulnerabilities.

●    Conducting thorough reviews of product specifications and software design to identify potential areas of concern before testing.

●    Architecting and designing complex, scalable test automation frameworks to optimize CI/CD velocity.

●    Ensuring the software meets customer and business requirements by validating the functionality and performance.

●    Assisting in continuously improving QA processes, tools, and best practices to enhance software testing efficiency and effectiveness.

●    Supporting user acceptance testing (UAT) and assisting clients with product validation.

●    Mentoring junior and mid-level engineers, providing technical guidance and conducting architectural reviews.

Required Experience

●    A Bachelor’s degree in Computer Science, Information Technology, Computer Engineering, or a related field.

●    8-10 years of proven experience in quality engineering, specifically within network cybersecurity, Identity Providers, or AI-integrated platforms.

●    Expertise in manual and automated testing.

●    Deep domain expertise in complex system validation and advanced automation practices at scale.

●    Proficiency in programming languages like Python to build and run automated test scripts.

●    "Strong knowledge of software testing methodologies, performance testing tools (e.g., JMeter, k6), and security traffic generation/simulation tools (e.g., Ixia BreakingPoint, Scapy, or Snort/Suricata traffic generators)."

●    Understanding of continuous integration/continuous deployment (CI/CD) pipelines and version control systems like Git.

●    Strong communication skills for documenting test results and interacting with cross-functional teams.

●    Excellent analytical skills, attention to detail, and problem-solving ability.

●    Ability to work independently as well as collaboratively in a team environment.

●    A curious mindset with a willingness to quickly learn new technologies and testing tools.

Required Skills & Qualifications

●    Familiarity with cloud-based testing environments (GCP, AWS, Azure).

●    Experience with cybersecurity products or cloud services or IDP or Web UI

The Mindset

●    Problem Solver: You thrive on complex, ambiguous challenges and engineer elegant solutions.

●    Ownership‑Driven: You take initiative, move fast, and deliver outcomes without hand‑holding.

●    Continuous Learner: You stay ahead of the curve in AI, ML, and emerging technologies.

●    Startup DNA: You excel in fast‑moving environments where priorities evolve and impact is immediate.

What you'll own

The quality charter. Define and evolve the testing strategy — web application, dashboards, APIs, webhooks, and customer integrations. Establish release criteria, coverage expectations, and quality gates that engineers actually respect because they're fast and they catch real bugs. Automation at scale. Build and maintain end-to-end and API automation suites that run on every PR and give a trustworthy signal. Own the flake rate. Design for parallel execution, deterministic test data, and fast feedback — a suite nobody trusts is worse than no suite. Performance and reliability. Design and run load, stress, and soak tests against APIs and the platform. Profile results, isolate bottlenecks with the engineers who own the service, and set up the monitoring that tells us about degradation before a customer does. Leverage through tooling, including AI. Use whatever gets coverage up and cycle time down: LLM-assisted test case generation from specs, AI-assisted authoring and maintenance of automation, synthetic test data generation, log and failure triage. We expect you to be opinionated about where these tools genuinely help and where they produce impressive-looking noise. Raising the bar. Partner with product and engineering from spec through release so quality risk is surfaced during design, not after code freeze. Mentor the QA team and build the practices — bug triage discipline, root cause analysis, prevention over detection — that outlast any single release.

What we're looking for

Must have

3–6 years in QA roles, with meaningful ownership of automation frameworks rather than only executing existing suites.

Strong test automation skills with a modern framework — Playwright, Cypress, or Selenium — and solid coding ability in Python, JavaScript/TypeScript, or Java. You should be comfortable reading application code, not just testing around it.

API testing experience: REST, auth, webhooks and asynchronous event flows, contract testing, and tooling such as Postman, REST Assured, or Pact.

Hands-on CI/CD integration — GitHub Actions, GitLab CI, Jenkins, or equivalent — including making suites fast and reliable enough to gate merges.

Working knowledge of SQL for validating data correctness, and comfort with logs, traces, and observability tooling for debugging.

Rigorous, systematic thinking about edge cases and failure modes, and the communication skills to make quality risk legible to product and leadership.

Nice to have

Experience testing AI, ML, or LLM-based products, and comfort with the reality that outputs are probabilistic rather than deterministic. Familiarity with Docker, Kubernetes, or cloud infrastructure for standing up test environments.

Performance and load testing with k6, JMeter, Gatling, or Locust, plus the analytical ability to turn results into a specific, actionable bottleneck. Exposure to enterprise SaaS with real integration surface area, or to security and compliance testing expectations of enterprise buyers. - Experience as the first or most senior quality voice on a team.

What we offer

Competitive compensation with equity. Direct ownership of a charter that materially affects how enterprise customers experience our product.

Proximity to frontier AI systems in production, and the freedom to use modern tooling to do your job well.

A fast-moving team where the distance between an idea and a shipped change is short.

Principal Software Engineer

Location: Bangalore India

Department: AI Engineering / Quality Assurance

Experience Level: Mid to Senior

We are looking for a highly motivated Senior AI Test / Automation Engineer to design and scale automated validation frameworks for AI/ML models, LLM-based applications, and agentic systems. This role is critical to ensure that AI solutions meet enterprise standards for quality, reliability, safety, and compliance before and after production deployment.

Key Responsibilities

• Build and maintain AI test automation frameworks for pre-qualification and continuous validation of models and agent workflows
• Develop comprehensive test suites, including:
• Unit, integration, and end-to-end (E2E)
• Functional, regression, performance, and safety testing
• Validate AI system behavior, including:
• Non-deterministic LLM outputs
• Hallucinations and edge cases
• Multi-step agent decision-making
• Design and manage evaluation systems:
• Golden datasets
• Benchmarking pipelines (accuracy, latency, reliability)
• Automate testing within CI/CD pipelines for model updates, prompt changes, and tool integrations
• Implement observability and telemetry to enable traceability, monitoring, and audit readiness
• Collaborate cross-functionally with ML, MLOps, Product, and Security teams to define quality gates and release criteria
• Track and report quality KPIs, including test coverage, defect leakage, and system reliability
• Drive root-cause analysis and continuous improvement across the AI testing lifecycle

Required Skills

Core Engineering

• Strong programming skills in Python; familiarity with Bash, TypeScript, or Go
• Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress
• Proficiency in CI/CD tools (GitHub Actions, Jenkins, GitLab CI)
• Experience with cloud platforms (AWS, Azure, GCP) and containers (Docker, Kubernetes)

AI / ML & Agentic Systems

• Hands-on experience with LLM ecosystems (OpenAI, Anthropic, Bedrock)
• Familiarity with:
• RAG architectures and vector databases (Pinecone, Weaviate)
• Agent frameworks (LangChain, LlamaIndex, AutoGen)

AI Testing Techniques

• Experience with non-deterministic testing approaches (statistical assertions, tolerance thresholds)
• Knowledge of evaluation methods:
• LLM-as-a-judge
• BLEU, ROUGE, semantic similarity scoring
• Experience with prompt and agent regression testing
• Understanding of AI safety testing, including adversarial testing, bias/fairness validation, and jailbreak detection

Tooling (Preferred)

• AI testing & observability tools: LangSmith, TruLens, Arize, Weights & Biases
• Evaluation tools: DeepEval, Ragas, PromptFoo, Giskard
• Monitoring: Prometheus, Grafana, OpenTelemetry

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication and cross-functional collaboration
• Data-driven mindset with focus on quality KPIs
• Detail-oriented with a strong bias toward automation and scalability

Experience Requirements

• 7+ years in QA, SDET, or test automation engineering
• Proven experience building and scaling automation frameworks
• Hands-on experience with AI/ML systems or LLM-based applications
• Experience testing RAG pipelines or agentic workflows
• Owned end-to-end AI test strategy and architecture
• Defined quality metrics and release gates
• Delivered scalable validation pipelines for production AI systems
• Supported audit and compliance readiness

Preferred

• Experience in enterprise or regulated environments (SOC2, ISO 27001, etc.)
• Exposure to:
• Shift-left testing practices
• Production observability and monitoring
• Chaos or resilience testing

Senior-Level Differentiators

Education

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field

Nice-to-have:

• ISTQB certification
• Cloud/ML certifications (AWS, Azure, GCP)
• AI testing certifications

What Success Looks Like

• AI systems that are accurate, reliable, and safe
• Fully automated test pipelines integrated into CI/CD
• Measurable improvements in defect leakage and model quality
• Strong observability and auditability across AI systems
• Scalable validation frameworks supporting rapid AI innovation

We are currently looking for a QA Engineer (3+ years of experience) to join our team. The ideal candidate should have strong experience in Python-based testing and mobile automation for Android and iOS using Appium. In this role, you will be responsible for ensuring product quality through automated and manual testing, identifying bugs, and collaborating with cross-functional teams to deliver high-quality applications.

Key Responsibilities:

• Design, develop, and execute test cases for web and mobile applications
• Perform automation testing using Python and PyTest
• Develop and maintain mobile automation scripts for Android and iOS using Appium
• Identify, log, and track bugs, and work closely with developers for resolution
• Perform regression, functional, and integration testing
• Participate in requirement analysis and test planning
• Improve test frameworks and automation coverage
• Ensure timely delivery of high-quality releases

Required Skills:

• 3+ years of experience in QA/testing
• Strong experience in Python testing
• Hands-on experience with PyTest
• Experience in Android & iOS automation testing
• Hands-on experience with Appium
• Understanding of SDLC/STLC and Agile methodology
• Good debugging and analytical skills
• Strong communication and teamwork skills

Preferred Skills:

• Experience with CI/CD tools
• API testing knowledge
• Experience with test management and bug tracking tools

We are seeking a forward-thinking Mid-Level QA Automation Engineer specializing in RPA, financial systems validation, and AI engineering. You will build and scale end-to-end automation workflows using UiPath for our financial services platform. Additionally, you will pioneer our testing evolution by leveraging prompt engineering, creating custom AI agents, and implementing Model Context Protocol (MCP) servers to bridge AI models with our testing infrastructure.

Key Responsibilities and Duties

   RPA Automation:

·    Design, develop, and maintain robust automated testing workflows using UiPath Studio.

 AI Agent Engineering:

·   Architect, build, and orchestrate custom AI agents to autonomously generate,execute, and self-heal test scripts.

Prompt Engineering:

·   Design, optimize, and manage advanced prompt templates to drive deterministic,high-quality code and test case generation from LLMs.

 MCP Integration:

·    Implement and configure Model Context Protocol (MCP) ecosystems to securely connect AI agents with local data, development tools, and testing environments. 

 Financial System Testing:

·   Validate end-to-end trade life cycles, market data processing, and capital market clearing workflows. 

Regression Orchestration

·   Manage continuous regression schedules via UiPath Orchestrator to ensure financial system stability. 

Mandatory skills

Project Mandatory Skills

Project Desired Skills

UiPath Studio and Orchestrator (UI/API automation activities)

Selenium or Playwright for web-based application testing

Building, deploying, and scripting autonomous AI agents for complex engineering workflows

Calypso treasury and capital markets platform

Prompt engineering — context injection, few-shot prompting, and guiding LLMs to exact test requirements

FitNesse for acceptance testing and collaborative documentation

Model Context Protocol (MCP) — using or developing MCP servers/clients to connect AI tools to external data sources and developer environments

TypeScript, Python, or Java to support agent tool calling and framework extensions

 Qualifications

• 3+ years of professional software QA automation or RPA development experience.
•  Strong understanding of capital markets, trading lifecycles, treasury workflows, or core banking architectures.
• Strong communication and collaboration skills to work across engineering, QA, and business teams.