11+ Model-based testing Jobs in Mumbai | Model-based testing Job openings in Mumbai

Position: QA Engineer – Machine Learning Systems (5 - 7 years)

Location: Remote (Company in Mumbai)

Company: Big Rattle Technologies Private Limited

Immediate Joiners only.

Summary:

The QA Engineer will own quality assurance across the ML lifecycle—from raw data validation through feature engineering checks, model training/evaluation verification, batch prediction/optimization validation, and end-to-end (E2E) workflow testing. The role is hands-on with Python automation, data profiling, and pipeline test harnesses in Azure ML and Azure DevOps. Success means probably correct data, models, and outputs at production scale and cadence.

Key Responsibilities:

Test Strategy & Governance

• ○ Define an ML-specific Test Strategy covering data quality KPIs, feature consistency
• checks, model acceptance gates (metrics + guardrails), and E2E run acceptance
• (timeliness, completeness, integrity).
• ○ Establish versioned test datasets & golden baselines for repeatable regression of
• features, models, and optimizers.

Data Quality & Transformation

• Validate raw data extracts and landed data lake data: schema/contract checks, null/outlier thresholds, time-window completeness, duplicate detection, site/material coverage.
• Validate transformed/feature datasets: deterministic feature generation, leakage detection, drift vs. historical distributions, feature parity across runs (hash or statistical similarity tests).
• Implement automated data quality checks (e.g., Great Expectations/pytest + Pandas/SQL) executed in CI and AML pipelines.

Model Training & Evaluation

• Verify training inputs (splits, windowing, target leakage prevention) and hyperparameter configs per site/cluster.
• Automate metric verification (e.g., MAPE/MAE/RMSE, uplift vs. last model, stability tests) with acceptance thresholds and champion/challenger logic.
• Validate feature importance stability and sensitivity/elasticity sanity checks (price/volume monotonicity where applicable).
• Gate model registration/promotion in AML based on signed test artifacts and reproducible metrics.

Predictions, Optimization & Guardrails

• Validate batch predictions: result shapes, coverage, latency, and failure handling.
• Test model optimization outputs and enforced guardrails: detect violations and prove idempotent writes to DB.
• Verify API push to third party system (idempotency keys, retry/backoff, delivery receipts).

Pipelines & E2E

• Build pipeline test harnesses for AML pipelines (data-gen nightly, training weekly,
• prediction/optimization) including orchestrated synthetic runs and fault injection
• (missing slice, late competitor data, SB backlog).
• Run E2E tests from raw data store -> ADLS -> AML -> RDBMS -> APIM/Frontend, assert
• freshness SLOs and audit event completeness (Event Hubs -> ADLS immutable).

Automation & Tooling

• Develop Python-based automated tests (pytest) for data checks, model metrics, and API contracts; integrate with Azure DevOps (pipelines, badges, gates).
• Implement data-driven test runners (parameterized by site/material/model-version) and store signed test artifacts alongside models in AML Registry.
• Create synthetic test data generators and golden fixtures to cover edge cases (price gaps, competitor shocks, cold starts).

Reporting & Quality Ops

• Publish weekly test reports and go/no-go recommendations for promotions; maintain a defect taxonomy (data vs. model vs. serving vs. optimization).
• Contribute to SLI/SLO dashboards (prediction timeliness, queue/DLQ, push success, data drift) used for release gates.

Required Skills (hands-on experience in the following):

• Python automation (pytest, pandas, NumPy), SQL (PostgreSQL/Snowflake), and CI/CD (Azure
• DevOps) for fully automated ML QA.
• Strong grasp of ML validation: leakage checks, proper splits, metric selection
• (MAE/MAPE/RMSE), drift detection, sensitivity/elasticity sanity checks.
• Experience testing AML pipelines (pipelines/jobs/components), and message-driven integrations
• (Service Bus/Event Hubs).
• API test skills (FastAPI/OpenAPI, contract tests, Postman/pytest-httpx) + idempotency and retry
• patterns.
• Familiar with feature stores/feature engineering concepts and reproducibility.
• Solid understanding of observability (App Insights/Log Analytics) and auditability requirements.

Required Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
• 5–7+ years in QA with 3+ years focused on ML/Data systems (data pipelines + model validation).
• Certification in Azure Data or ML Engineer Associate is a plus.

Why should you join Big Rattle?

Big Rattle Technologies specializes in AI/ ML Products and Solutions as well as Mobile and Web Application Development. Our clients include Fortune 500 companies. Over the past 13 years, we have delivered multiple projects for international and Indian clients from various industries like FMCG, Banking and Finance, Automobiles, Ecommerce, etc. We also specialise in Product Development for our clients.

Big Rattle Technologies Private Limited is ISO 27001:2022 certified and CyberGRX certified.

What We Offer:

• Opportunity to work on diverse projects for Fortune 500 clients.
• Competitive salary and performance-based growth.
• Dynamic, collaborative, and growth-oriented work environment.
• Direct impact on product quality and client satisfaction.
• 5-day hybrid work week.
• Certification reimbursement.
• Healthcare coverage.

How to Apply:

Interested candidates are invited to submit their resume detailing their experience. Please detail out your work experience and the kind of projects you have worked on. Ensure you highlight your contributions and accomplishments to the projects.

Hiring: Full-Time Project Manager

Location: Hiranandani Estate, Thane

Timing: 10:30 AM to 6:30 PM (2nd & 4th Saturday Off)

CTC: Up to ₹3 LPA

We are seeking an enthusiastic and detail-oriented Project Manager who can drive our design, marketing, and tech projects with passion and precision. The ideal candidate will have the ability to manage multiple teams, ensure project deadlines are met, and maintain high standards of execution and quality.

Role Responsibilities:

Manage and oversee a variety of projects focused on design, marketing campaigns, branding, websites, and tech development.

Plan project timelines, allocate tasks, monitor progress, and ensure timely delivery.

Act as the key point of contact between clients and internal teams (designers, developers, marketers, social media managers).

Coordinate team efforts to maintain high standards of quality, creativity, and performance.

Identify project risks, suggest solutions, and proactively manage any roadblocks.

Maintain clear project documentation and provide regular status updates to management.

Ensure that all projects stay within budget, timeline, and defined scope.

Ideal Candidate:

1–2 years of project management experience (preferably in a digital marketing or creative agency environment).

Hands-on experience managing design, marketing, and tech projects.

Excellent communication, coordination, and leadership skills.

Basic understanding of digital marketing strategies, branding processes, UI/UX design, and website development.

Familiarity with project management tools like Trello, ClickUp, or Notion is a plus.

Proactive, solution-oriented, and able to maintain a positive approach under pressure.

Your future role within QRT

• Your core objective is to create high quality predictive signals.
• By leveraging access to large and diversified datasets you will identify statistical patterns and opportunities.
• Share and discuss research results, methodology, data sets and processes with other researchers.
• Implement the signals and relevant datasets within the global execution platform.
• Monitor signal behaviour and model performance over time.

Your present skillset

• Advanced degree in a quantitative field such as data science, statistics, mathematics, physics or engineering.
• Strong knowledge in statistics, machine learning, NLP or AI techniques is a plus.
• Capacity to multi-task in a fast paced environment while keeping strong attention to detail.
• Coding skills required in at least one leading programing language (Python, R, Matlab and /or C++, C#).
• Experience in exploring large datasets across multiple time frames is a plus.
• Intellectual curiosity to explore new data sets, solve complex problems, drive innovative processes and connect the dots between multiple fields.
• Capacity to work with autonomy within a collegial and collaborative environment.

• Minimum 1 years of relevant experience, in PySpark (mandatory)
• Hands on experience in development, test, deploy, maintain and improving data integration pipeline in AWS cloud environment is added plus 
• Ability to play lead role and independently manage 3-5 member of Pyspark development team 
• EMR ,Python and PYspark mandate.
• Knowledge and awareness working with AWS Cloud technologies like Apache Spark, , Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS

We are building a global content marketplace that brings companies and content

creators together to scale up content creation processes across 50+ content verticals and 150+ industries. Over the past 2.5 years, we’ve worked with companies like India Today, Amazon India, Adobe, Swiggy, Dunzo, Businessworld, Paisabazaar, IndiGo Airlines, Apollo Hospitals, Infoedge, Times Group, Digit, BookMyShow, UpGrad, Yulu, YourStory, and 350+ other brands.
Our mission is to become the world’s largest content creation and distribution platform for all kinds of content creators and brands.

Our Team

We are a 25+ member company and is scaling up rapidly in both team size and our ambition.

If we were to define the kind of people and the culture we have, it would be -

a) Individuals with an Extreme Sense of Passion About Work

b) Individuals with Strong Customer and Creator Obsession

c) Individuals with Extraordinary Hustle, Perseverance & Ambition

We are on the lookout for individuals who are always open to going the extra mile and thrive in a fast-paced environment. We are strong believers in building a great, enduring

a company that can outlast its builders and create a massive impact on the lives of our

employees, creators, and customers alike.

Our Investors

We are fortunate to be backed by some of the industry’s most prolific angel investors - Kunal Bahl and Rohit Bansal (Snapdeal founders), YourStory Media. (Shradha Sharma); Dr. Saurabh Srivastava, Co-founder of IAN and NASSCOM; Slideshare co-founder Amit Ranjan; Indifi's Co-founder and CEO Alok Mittal; Sidharth Rao, Chairman of Dentsu Webchutney; Ritesh Malik, Co-founder and CEO of Innov8; Sanjay Tripathy, former CMO, HDFC Life, and CEO of Agilio Labs; Manan Maheshwari, Co-founder of WYSH; and Hemanshu Jain, Co-founder of Diabeto.