9 Benchmarking Jobs in India

LLM/AI Model Engineer – Training, Evaluation & Benchmarking

Location: New Delhi, India (On-site)

Employment Type: Full-time

Experience: 3–5 years

Industry: Enterprise AI / Generative AI / Machine Learning / NLP

About QubeLabs

QubeLabs is building the next generation of Enterprise AI Systems that transform workforce operations and intelligence for the financial services industry.

Our platform combines Conversational AI, Agentic AI, workflow automation, proprietary language models and enterprise intelligence. Our flagship product, QubeLabs Workmate, serves banks, NBFCs, MFIs, wealth management firms, insurance companies and fintechs across India and Europe.

At the core of our AI platform is the Vectro Series, our proprietary family of language models designed for enterprise and financial-services use cases.

We are looking for an LLM/AI Model Engineer to build the training pipelines, datasets and evaluation infrastructure required to continuously improve the Vectro Series.

Role Overview

You will work closely with the Head of AI, Research Partners and engineering teams to implement and operationalize LLM training approaches and build a reliable model improvement lifecycle:

Data Preparation → Training → Evaluation → Benchmarking → Error Analysis → Model Improvement

The role combines hands-on LLM fine-tuning, dataset engineering, tokenization, evaluation framework development and model quality management.

Key Responsibilities

1. LLM Training & Fine-tuning

Build and maintain pipelines for training and fine-tuning open-source foundation models for the Vectro Series.

Implement supervised fine-tuning (SFT), parameter-efficient fine-tuning (PEFT), LoRA/QLoRA and domain adaptation techniques; support continued pre-training where required.

Manage training configurations, checkpoints, model versions and experiment tracking.

Optimize training workflows for computational efficiency, reproducibility and reliability.

Translate model development approaches defined with the Head of AI and Research Partners into practical, scalable training pipelines.

2. Training Data & Tokenization

Build data pipelines to collect, clean, filter, normalize, deduplicate and validate training data.

Develop instruction-tuning, SFT and domain-specific datasets for financial-services and enterprise use cases.

Implement tokenization workflows, tokenizer configuration, vocabulary management, sequence packing, truncation and padding.

Address data representation challenges involving financial terminology, numerical values and multilingual content.

Maintain dataset versioning, lineage and quality controls, incorporating identified data gaps and relevant model failures into future training data.

3. Golden Dataset & Model Evaluation

Build and maintain QubeLabs' Golden Dataset to evaluate key model capabilities and target use cases.

Develop automated evaluation pipelines and reusable evaluation harnesses, combining automated metrics with human assessment where appropriate.

Evaluate accuracy, reasoning, factuality, instruction following, safety, multilingual performance and financial-domain capabilities.

Establish consistent evaluation methodology and regression tests to measure changes across model versions.

4. Benchmarking & Error Analysis

Evaluate Vectro against relevant public benchmarks and proprietary QubeLabs BFSI/enterprise benchmarks.

Compare results with relevant open-source and commercial models using consistent evaluation conditions.

Analyze failures, identify root causes across data, training and model behavior, and translate findings into actionable improvements.

Maintain reproducible benchmark results and reports to track model strengths, limitations and progress.

5. Model Quality & Release

Define model release quality gates and ensure each major release meets agreed evaluation and benchmark criteria.

Support red-teaming, robustness and safety testing.

Coordinate with the Head of AI, Research Partners and engineering teams to ensure validated model improvements are suitable for integration into production systems.

Experience

3–5 years in Machine Learning, Deep Learning, NLP, Generative AI or related fields.

Hands-on experience training or fine-tuning LLMs using open-source foundation models.

Practical experience building training pipelines, preparing datasets and evaluating model performance.

Experience with GPU-based training; distributed training is an advantage.

Financial-services, multilingual AI or enterprise AI experience is preferred.

Required Skills

LLM & Machine Learning: Large Language Models, Transformer architectures, Generative AI, NLP, Deep Learning, SFT, PEFT, LoRA/QLoRA, fine-tuning and continued pre-training.

Frameworks & Infrastructure: Python, PyTorch, Hugging Face Transformers, Hugging Face Tokenizers, GPU computing, experiment tracking and ML pipelines.

Data & Tokenization: Tokenization, tokenizer configuration, text preprocessing, sequence packing, dataset construction, data quality, versioning and lineage.

Evaluation & Benchmarking: Golden datasets, LLM evaluation, evaluation harnesses, automated and human evaluation, LLM-as-a-Judge, public and domain-specific benchmarks, regression testing and error analysis.

The role requires the ability to build both the model training pipeline and the measurement system that determines whether the model has actually improved.

What Success Looks Like

Reliable and reproducible training and fine-tuning pipelines for the Vectro Series.

High-quality training datasets, tokenization workflows and a comprehensive Golden Dataset.

Automated evaluation and benchmarking infrastructure with measurable model quality gates.

Clear, reproducible evidence of model performance against public, proprietary BFSI and relevant competing-model benchmarks.

A continuous improvement loop that converts evaluation findings into measurable gains across successive Vectro releases.

Key Performance Indicators

Training pipeline reliability and experiment throughput.

Training data quality and coverage.

Golden Dataset and evaluation coverage.

Benchmark reproducibility and model performance improvement.

Accuracy of error diagnosis and effectiveness of regression detection.

Time required to complete training, evaluation and benchmarking cycles.

Compliance with model release quality criteria.

Why Join QubeLabs?

Build the proprietary Vectro Series of enterprise and financial-services language models.

Develop model training, evaluation and benchmarking infrastructure from the ground up.

Work closely with the Head of AI and Research Partners.

Solve real-world AI challenges across financial services.

Develop proprietary datasets, benchmarks and model improvement systems.

Contribute directly to production AI systems serving customers across India and Europe.

Preferred Candidate Profile

B.Tech, M.Tech, MS or PhD in Computer Science, AI, ML, Mathematics or a related field.

3–5 years of relevant experience with strong practical skills in Python, PyTorch, LLM fine-tuning and training data pipelines.

Working knowledge of tokenization, evaluation datasets, benchmarking and regression testing.

Experience with open-source models such as Llama, Qwen, Mistral or similar is preferred.

Ability to independently build reliable model training and evaluation systems from scratch.

QubeLabs is an equal opportunity employer and values diversity and inclusion. We encourage applications from qualified candidates regardless of background or identity.

Job Summary

The Compensation Analyst is responsible for supporting the design, implementation, administration, and analysis of compensation programs that attract, retain, and motivate employees while ensuring market competitiveness and internal equity. This role partners with HR, Finance, and business leaders to provide compensation insights, conduct market analyses, and support salary planning processes.

Key Responsibilities

Compensation Analysis & Benchmarking

• Conduct market pricing of positions using compensation surveys and external benchmarking data.
• Analyze internal and external compensation trends to ensure competitive pay practices.
• Evaluate job descriptions and support job levelling and salary grade assignments.
• Recommend salary structures, pay ranges, and compensation adjustments.

Annual Compensation Processes

• Support annual merit, bonus, and promotion cycles.
• Prepare compensation guidelines, budget analysis, and management reports.
• Conduct pay equity reviews and identify areas requiring corrective action.
• Validate compensation data and ensure accuracy of employee records.

Data Management & Reporting

• Develop and maintain compensation dashboards, reports, and analytics.
• Analyze compensation metrics including compa-ratio, range penetration, salary progression, and market positioning.
• Prepare executive summaries and presentations for leadership review.
• Ensure data integrity within HRIS and compensation systems.

Program Administration

• Administer salary structures, incentive plans, and recognition programs.
• Provide consultation to HR Business Partners, managers and recruiting on compensation-related matters.
• Support offer evaluations, promotion recommendations, and retention proposals.
• Assist in implementation of new compensation programs and policies.

Compliance & Governance

• Ensure compliance with company compensation policies and applicable local labor regulations, especially recent labor code changes.
• Support compensation audits and documentation requirements.
• Maintain confidentiality of employee compensation information.

Required Qualifications

• Bachelor's degree in Human Resources, Business Administration, Finance, Economics, Statistics, or related field.
• 10+ years of experience in compensation, rewards, HR analytics, or related HR functions, with a background in finance, economics, business administration, or human resources.
• Strong analytical and quantitative skills.
• Advanced proficiency in Microsoft Excel
• Proficiency using Payfactors.
• Excellent communication skills, with confidence managing stakeholder feedback and expectations.
• Experience working with HRIS systems such as Workday, SAP SuccessFactors, Oracle HCM, or similar.
• Experience designing and administering incentive plans.
• Strong attention to detail and data accuracy.

Important: Protect Yourself from Recruitment Scams

Bad actors or scammers may try to impersonate Deltek and send fake job offers to people. Messages from Deltek about employment opportunities will be from an atdeltek.com account or Enterpriseattrm.brassring.com, never from free services like Gmail or Yahoo. Please look carefully at the email address that provides any job offer, as some fake accounts are created to look like a legitimate domain name or email address. We will also never ask you to pay money at any point in the hiring process, whether for training, equipment, background checks, or anything else. If you receive a suspicious offer claiming to be from Deltek, do not share personal or financial information. Report any suspicious communication to Secureatdeltek.com and consider reporting it to law enforcement.

!! We are Hiring !!

Role: Bench Sales Recruiters/HR Executive

Exp: 1 - 8 years

Work Location: Uppal(Onsite)

MOI: F2F

Srinivas

Victoire Systems LLC

Job Type: Full-time

Salary: Up to ₹80,000.00 per month

Benefits:

• Health insurance
• Paid sick time
• Provident Fund

Schedule:

• Night shift
• US shift

Supplemental pay types:

• Performance bonus
• Shift allowance

Ability to commute/relocate:

• Hyderabad, Telangana: Reliably commute or planning to relocate before starting work (Required)

Education:

• Bachelor's (Preferred)

Language:

• English (Preferred)

Work Location: In person

About Debox:

Debox is a business solutions firm that focuses on problem-solving through the use of data, logical reasoning and people-centred approach. We have a stellar record of delivering high-impact projects that have helped scale many startups, family-managed businesses and midsize corporates.

From redesigning and successful deployment of the Pan India Supply Chain process for Nippo Batteries to designing and developing InRadius - India's first true hyperlocal Job and Talent search platform, Debox has come a long way in solving business and real-life problems through their logical reasoning and innovative thinking.

Role: Client Servicing Specialist

Location: Thane (Mumbai)

Work Experience: 3-4 Years

Qualification: Masters or equivalent with a minimum of 3 years prior experience working in a Digital Media firm and having managed client communication.

Role Objective: The Client Servicing specialist will be the person responsible for driving the business objectives of the client to success through collaboration and coordination with the internal teams at Debox.

Job Description:

• Continuous benchmarking of clients competitors and industry practices
• Continuous benchmarking of media channels and keeping abreast with the latest updates, changes and modifications to various platforms that can be leveraged
• Research on industry trends, promotions, business cycles and retention rates
• Ideation for new promotions, content mix, communication channels to drive higher engagement and customer stickiness
• Monitor Google Analytics, Facebook Pixel and Sales data to derive actionable insights to course correct or drive business success
• Continuously monitor key business parameters and evaluate the success of marketing efforts, engagement rate, open rate and conversion
• Deep dive and evaluate the user flows and journeys (Google Analytics) and identify drop-off points and build strategy to improve conversions
• Collaborate with the Content and Design team to build an engaging monthly bank, campaigns and content to drive higher engagement and conversion driving monthly business targets
• Ensure timely planning and communication with clients to avoid last-minute requests or changes in the plan and campaigns
• Ensure internal checks, dummy testing and overall review is done before going live with all campaigns
• Ensure all campaign success parameters are tracked and are in line with expectations, immediately raise escalation in case of any deviation to plan vs actuals
• Create monthly reports and conduct monthly review meetings with clients to showcase key parameters and achievements for the month
• Create quarterly plans with the client and drive the finalized plans to success, create quarterly plan vs actuals reports and conduct client review meetings 

Required Skills:

• Excellent communication and presentation skills 
• Ability to manage various types of clients
• Good analytical skills and data orientation (should be good with excel and data analysis)
• Excellent in Time Management
• Ability to manage and get work done through the teams

As a Manager in our Performance Marketing team, you will be responsible for driving growth of APAC & North America regions through digital marketing. This team is a key driver of growth for Headout and is responsible for acquiring customers and generating demand for our vast collection of experiences and destinations. This is a key role for our overall company growth and profitability. 

As Performance Marketing Manager, you will: 

1. 1. Manage all paid user acquisition channels for APAC & North America. This includes: Responsible to deliver ROI & GBV targets for your regions and growing these each quarter
2. Actively manage and grow SEM campaigns for new and existing categories &  markets
3. Identify opportunities to scale existing channels of user acquisiton including SEM, OLA and Facebook d. Develop new channels of user acquisition to drive new growth and diversify channel-mix 
4. Improve the efficiency of the team through process improvements, decision making frameworks and automation 
5. Improve reporting and data analysis to uncover new opportunities or Coordinate with other teams such as product, data, operations & supply to participate in & lead company-wide growth projects by leveraging various levers like: 

• Identifying new market opportunities 
• Product enhancements 
• Pricing strategy 
• Competitive benchmarking 
• Building stronger engagement loops and cross-channel campaign strategies 

What we're looking for: 

• 4+ years of experience in managing SEM campaigns at scale 
• 2+ years of experience in managing a team 
• Detail-oriented and analytical 
• Strong learning curve 
• Data-driven decision making 
• Able to effectively organize, prioritize & manage their time 
• Experience with Automations is a big plus