Cutshort logo
MeltPlan logo
AI System QA Engineer – Large Language Models (Evaluation Testing)
AI System QA Engineer – Large Language Models (Evaluation Testing)
MeltPlan's logo

AI System QA Engineer – Large Language Models (Evaluation Testing)

Gaurav Bhardwaj's profile picture
Posted by Gaurav Bhardwaj
5 - 7 yrs
Best in industry
Bengaluru (Bangalore), Mumbai, Delhi, Gurugram, Noida, Ghaziabad, Faridabad, Pune, Hyderabad
Skills
Data-driven testing
Playwright
DeepEval
Ragas
LLM Evaluation Frameworks
MLOps

MeltPlan is building the “planning engine” for the $14 Tn construction industry, an AI system designed specifically to optimize decisions before construction begins. While design software optimizes use and aesthetics and construction software optimizes execution and control, MeltPlan is building the missing layer - software that optimizes decisions and tradeoffs upstream, before scope is locked, procurement begins, and change orders become inevitable. MeltPlan’s long-term goal is to help teams make construction “boring” by making planning more intense: surfacing constraints and tradeoffs early, aligning stakeholders before plans are frozen, and reducing the need for late-stage redlines, rework, and change orders.


MeltPlan is founded by operators who have built at scale. Kanav previously co-founded Innovaccer, a $3Bn healthtech company focused on making US healthcare more affordable and accessible. He’s now applying that systems-level thinking to construction.He’s joined by Tanmaya Kala, former Project Executive at DPR Construction, who led large commercial, healthcare, and life sciences projects. We combine deep tech scale with real construction execution.


What This Role Really is :


We are seeking a detail-oriented and technically strong AI QA Engineer to ensure the quality, reliability, and performance of Large Language Model (LLM)-based systems. In this role, you will be responsible for designing and executing test strategies, validating model outputs, and building evaluation frameworks to enhance the accuracy, safety, and overall performance of AI-driven applications.We would particularly value candidates who have hands-on experience in developing evaluation frameworks (evals) for AI systems, along with strong expertise in comprehensive system testing and quality assurance practices.You are responsible for making MeltPlan work in the real world.


What You'll Do:


  • Design, develop, and execute evaluation frameworks (Evals) for Large Language Models (LLMs) and AI systems.
  • Perform end-to-end system testing, regression testing, and performance testing for AI-driven applications.
  • Validate model outputs for accuracy, consistency, safety, hallucination detection, and edge cases.
  • Build automated test pipelines and quality benchmarks for AI systems.
  • Collaborate closely with AI/ML engineers, product teams, and platform engineers to improve system reliability.
  • Analyze failures, identify root causes, and provide actionable feedback to improve model behavior.
  • Develop datasets, prompts, and testing scenarios to measure model performance across multiple use cases.
  • Monitor production performance and continuously improve evaluation metrics and testing standards.
  • Ensure compliance with responsible AI and quality assurance best practices.


What We're looking for: 


  • Bachelor’s degree in Computer Science, Engineering, or related field
  • 5–7 years of experience in QA/testing, preferably in AI/ML or data-driven systems
  • Strong experience in AI/LLM evaluation frameworks and system testing.
  • Hands-on experience with automated testing methodologies and QA processes.
  • Familiarity with prompt engineering, AI benchmarking, and model validation techniques.
  • Experience working with Python and testing frameworks.
  • Understanding of LLM behaviors, hallucinations, prompt injection risks, and AI safety concepts.
  • Exposure to tools/frameworks such as OpenAI Evals, LangSmith, DeepEval, Promptfoo, or similar platforms is preferred.
  • Strong analytical and debugging skills with attention to detail.
  • Excellent collaboration and communication skills.
  • Familiarity with Large Language Models and Generative AI concepts
  • Experience with API testing tools (e.g., Postman) and automation frameworks
  • Understanding of NLP concepts such as tokenization, embeddings, and text generation
  • Strong analytical and problem-solving skills
  • Experience testing AI/ML models or data pipelines
  • Experience with prompt engineering and prompt testing
  • Familiarity with cloud platforms (AWS, GCP, or Azure)
  • Exposure to AI safety, bias detection, and model governance


Bonus if you have:


  • Have worked in construction or on project sites
  • Have startup experience
  • Experience working with Generative AI or conversational AI products.
  • Knowledge of CI/CD pipelines and automation workflows.
  • Prior experience in performance testing and monitoring distributed systems.
  • Understanding of AI product lifecycle and production deployment environments.


We’re not looking for someone who waits for clean requirements.We’re looking for someone who thrives in the mess and turns it into systems.


Why meltplan

  • Massive industry, real-world impact
  • High ownership from day one
  • Small team, zero bureaucracy
  • Competitive comp + meaningful equity
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About MeltPlan

Founded :
2024
Type :
Product
Size :
20-100
Stage :
Raised funding

About

MeltPlan is the AI-native planning software for construction – automate takeoffs with AI-assisted service, ensure building code compliance, and streamline cost planning with transparent, expert AI that shows its work. Designed for architects, engineers, contractors, and inspectors.
Read more

Company social profiles

bloglinkedin

Similar jobs

Gurugram
8 - 15 yrs
₹30L - ₹40L / yr
Recruitment/Talent Acquisition
talent acquisition head
NON-IT hiring
Retail

Position: Talent Acquisition Head – Retail Brand (Non-IT Hiring)

Location: Gurgaon

Experience: 10+ years in retail brand

Industry: Retail (Fashion, Apparel, FMCG, Luxury, or related)

Department: Human Resources (HR)

Job Summary:

We are looking for a dynamic and strategic Talent Acquisition Head to lead the recruitment function for our retail brand. The ideal candidate will have extensive experience in Non-IT hiring, particularly in retail, FMCG, or related industries. This role involves managing end-to-end recruitment, building talent pipelines, and ensuring the right talent acquisition strategies to support business growth.

Key Responsibilities:1. Talent Acquisition Strategy & Execution:

  • Develop and execute talent acquisition strategies aligned with business objectives.
  • Oversee bulk hiring and leadership hiring for retail stores, corporate functions, and supply chain roles.
  • Optimize sourcing channels (job portals, employee referrals, social media, and recruitment agencies).

2. Recruitment Operations & Stakeholder Management:

  • Lead end-to-end recruitment for store-level, regional, and corporate roles.
  • Partner with business heads and HRBPs to identify hiring needs and workforce planning.
  • Manage vendor relationships and negotiate contracts with recruitment agencies.

3. Employer Branding & Talent Pipeline Development:

  • Drive employer branding initiatives to attract top talent.
  • Build a strong talent pipeline for future hiring needs.
  • Engage in campus hiring and lateral hiring strategies.

4. Recruitment Analytics & Compliance:

  • Track recruitment metrics (TAT, cost-per-hire, quality-of-hire, etc.) for continuous improvement.
  • Ensure compliance with labor laws and company policies in hiring practices.
  • Implement and manage Applicant Tracking Systems (ATS) for seamless recruitment.

Key Requirements:

·         10+ years of experience in Non-IT hiring, preferably in Retail, FMCG, Apparel, Luxury, or QSR sectors.

·         Proven expertise in bulk hiring, corporate hiring, and leadership hiring.

·         Strong experience in headhunting, passive talent search, and market mapping.

·         Excellent communication, negotiation, and stakeholder management skills.

·         Hands-on experience with ATS, LinkedIn Talent Solutions, and recruitment tools.

·         Ability to thrive in a fast-paced, high-growth retail environment.

 

mail updated resume with current ctc and notice period-

email: jobs[at}glansolutions[dot]com

 

website: www.glansolutions.com

Read more
Zenfra
at Zenfra
2 candid answers
Mirudula Varsha
Posted by Mirudula Varsha
Coimbatore
6 - 12 yrs
₹6L - ₹15L / yr
Operations

Finance & Operations Manager – IT Company

Role Overview

The Finance & Operations Manager drives financial health and operational efficiency, ensuring IT projects are delivered on time, within budget, and aligned with strategic goals. This role blends financial expertise with operational oversight to optimize resources, control costs, and enable sustainable growth in a fast-paced technology environment.


Key Responsibilities

Financial Management

  • Develop annual budgets, forecasts, and financial models.
  • Monitor project profitability, billing cycles, and cash flow.
  • software licensing, and vendor expenses.
  • Prepare monthly/quarterly financial reports for leadership.
  • Ensure compliance with tax, audit, and accounting standards.

Operations Management

  • Allocate resources effectively across projects and teams.
  • Streamline processes using ITIL/Agile best practices.
  • Manage vendor relationships and negotiate contracts.
  • Track KPIs: utilization rates, SLA adherence, delivery timelines.


Skills & Tools

  • Finance: Budgeting, cost analysis, CAPEX/OPEX management.
  • Tech Awareness: IT project lifecycle
  • Analytical: KPI tracking(Excel, Power BI).
  • Tools: Jira, Confluence, MS Excel.
  • Soft Skills: Leadership, problem-solving, vendor negotiation.


Key Performance Indicators (KPIs)

  • Project profit margin
  • OPEX vs CAPEX ratio
  • Resource utilization rate
  • Vendor performance score
  • Invoice-to-payment cycle time


Impact Statement:

A Finance & Operations Manager ensures that every dollar spent drives value, every resource is utilized optimally, and every project contributes to the company’s bottom line—keeping IT operations efficient, scalable, and profitable.

Read more
XTEP
Sachin Shastry
Posted by Sachin Shastry
Bengaluru (Bangalore)
3 - 5 yrs
₹5L - ₹8L / yr
Social media management
Brand Management
Social Media Marketing (SMM)
Digital Marketing

We are seeking a creative and results-driven Digital Marketing Executive to join our team at Xtep India. The ideal candidate will lead efforts to enhance our brand's visibility, drive engagement across social media platforms, and increase traffic and reach for our Direct-to-Consumer (D2C) website. This role involves planning, executing, and analyzing various digital marketing strategies to build brand awareness and connect with our target audience effectively in the digital world.


Key Responsibilities:

1.   Social Media Management:

o Develop and implement a comprehensive social media strategy to increase brand visibility and engagement across platforms (e.g., Instagram, Facebook, Twitter, LinkedIn, YouTube).

o Create, schedule, and manage compelling content, including text, images, videos, and infographics.

o Monitor social media trends, audience behavior, and competitive activities to refine strategies.

o Respond to comments, messages, and queries promptly to foster community engagement.

2.   Brand Awareness & Building:

o Plan and execute campaigns that position Xtep India as a leading brand in the sportswear industry.

o Collaborate with creative teams to design impactful visual and written content that aligns with brand guidelines.

o Partner with influencers and bloggers to amplify the brand’s reach.

o Ensure consistent brand messaging across all digital platforms.

3.   D2C Website Marketing:

o Develop strategies to drive organic and paid traffic to the D2C website.

o Optimize the website for SEO, including keyword research, meta tags, and on-page optimization.

o Execute email marketing campaigns to retain and engage customers.

o Monitor website analytics and make data-driven recommendations to improve conversion rates.

4.   Content Marketing:

o Collaborate with content creators to produce blogs, videos, and other materials that resonate with the target audience.

o Ensure content aligns with SEO best practices to increase discoverability.

5.   Analytics & Reporting:

o Monitor and report on key performance metrics across all digital channels.

o Provide insights and recommendations to improve future campaigns.

o Stay updated on industry trends, emerging platforms, and new digital tools.

6.   Cross-Functional Collaboration:

o Work closely with the marketing, sales, and product teams to ensure cohesive brand messaging and integrated campaigns.

Requirements:

·    Bachelor’s degree in marketing, Digital Media, Communications, or a related field.

·    Proven experience in digital marketing, preferably in the retail or sportswear industry.

·    Strong understanding of social media platforms, SEO, and performance marketing.

·    Proficiency in tools such as Google Analytics, Google Ads, Facebook Ads Manager, and social media scheduling tools.

·    Excellent communication, creative thinking, and problem-solving skills.

·    Ability to work in a fast-paced environment and manage multiple projects simultaneously.

·    Knowledge of the sportswear market and passion for fitness or sports is a plus.


Read more
E Health Source
Kavita Gehani
Posted by Kavita Gehani
1st Floor, Heritage Annexe, Bank Ln, behind Vijyalaxmi, Besides Gujarat Vidhyapith, Usmanpura, Ahmedabad
6 - 10 yrs
₹8L - ₹10L / yr
User Experience (UX) Design
User Interface (UI) Design

Senior UI/UX Designer

Job Summary: We are seeking a highly skilled Senior UI/UX Designer to lead the design of our company's digital products and services. The ideal candidate will have at least 5 years of experience in UI/UX design and a proven track record of creating elegant and effective user-centered designs. You will work closely with cross-functional teams including product managers, developers, and other designers to create innovative solutions that meet user needs and business goals.

Responsibilities:

·        Lead the design of our company's digital products and services, ensuring that they meet user needs and business goals.

·        Collaborate with cross-functional teams to gather requirements, define user personas, and create user flows and wireframes.

·        Create high-fidelity mockups and prototypes to demonstrate design concepts and interactions.

·        Conduct user research and usability testing to validate design decisions and identify areas for improvement.

·        Work with developers to ensure that designs are implemented to spec and delivered on time.

·        Stay up-to-date with the latest design trends and technologies, and share your knowledge with the team.

·        Mentor and train junior designers.

Requirements:

·        At least 5 years of experience in UI/UX design, with a strong portfolio of work that demonstrates your skills.

·        Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.

·        Strong problem-solving skills, with the ability to think creatively and come up with innovative solutions.

·        Proficiency with design tools such as Sketch, Adobe Creative Suite, Figma, or similar.

·        Knowledge of HTML/CSS and front-end development is a plus.

·        Bachelor's degree in design, human-computer interaction, or related field is preferred.

If you are a self-motivated, detail-oriented, and experienced UI/UX designer who is passionate about creating innovative digital products and services, we encourage you to apply for this position.

 

Read more
Remote only
0 - 1 yrs
₹1000 - ₹15000 / mo
Communication Skills

About us :

Our Digital Marketing Agency identifies your goals, performs competitors bench-marking and evaluates your customers’ online behavior. Using data and analytics, we develop custom social media, brand management and paid advertising strategies tailored to your business.


Our mission is to empower clients to use the internet to its full potential by providing affordable, effective, custom design and marketing solutions


We are looking for a Business Development Intern.


Duration: 2 months


Work type: Remote


Stipend: Performance Based

[Earn lot's of commission, rewards, gifts and many more]


Business Development Intern roles and responsibilities:-


◆Prospecting potential customers by phone, by email, physically

Selling products or services to those clients.

◆Managing a client's portfolio.

◆Managing sales process.

◆Identifying new development channels.

◆Follow-up of sales.

◆Promoting the company and its product/service.


Skills require

-Communication skill


Perks :

-Offer letter

-Letter of recommendation (based on performance)

  • -Completion certificate
Read more
Vitara Analytics
at Vitara Analytics
1 recruiter
Srinivas Pratapa
Posted by Srinivas Pratapa
Remote only
1 - 3 yrs
₹8L - ₹10L / yr
skill iconHTML/CSS
skill iconBootstrap
Material Design
skill iconSASS
LESS
+4 more

As a senior engineer you will be working independently in designing and implementing new visuals and also helping improve code quality. A lot of emphasis is on usability; so any specific experience is ux is a big plus. Any experience with backend implementation is also a plus

You will work with a mix of front end technologies, primarily JavaScript, D3, HTML 5, SVG, CSS (and related tech like SaSS/LESS). You'd also work with Git, WebPack. You don't already need to know all of these technologies.The key requirement is a keen interest in UI/UX work, solid front-end programming skills (thorough understanding of JavaScript, CSS and browser rendering workflow) and critical thinking.

Remote working is allowed

Read more
Datawrkz
Thallapalli Chandrakala
Posted by Thallapalli Chandrakala
Bengaluru (Bangalore)
3 - 6 yrs
₹7L - ₹10L / yr
Affiliate marketing
Marketing
Marketing Strategy
Digital Marketing
Roles and Responsibilities:
  • Grow business month on month - Identify and map prospective clients, forecasting revenue monthly/quarterly and negotiating the terms of agreement.
  • Focus on growing new relationships with key partners, negotiate contracts and pursue ways to secure long-term relationships.
  • Primary responsibility will be to understand client requirements & on-board quality Affiliates / Publishers / Ad networks to deliver required quality & numbers through research, referrals, networking, cold calling, data feeds and emails.
  • Media buying (both mobile and PC web traffic of CPA, CPC, CPL, CPI, CPS, CPT, CPR, CPV, CPA traffic.
  • Close coordination with BD & Client Servicing Team.
  • Negotiate and manage commission structures for affiliates
  • Thorough knowledge of attribution platforms Appsflyer, Branch, Singular, Adjust, Kochava etc.
  • Troubleshooting tracking issues at publisher end. Ensure timely resolution of publisher queries through proper escalation.
  • Develop business strategies to improve affiliate program.
  • Maintain open communication with all affiliates on the day-to-day issues. Communicate with new and existing affiliates regarding upcoming promotions and optimization opportunities.
  • Allocate targets to affiliates and ensure that targets for the respective campaigns are met within specified timelines.
  • Stay current with trends in affiliate marketing and identify new opportunities.
  • Interacting with advertisers along with the demand team to understand campaign performance & to further optimize campaigns.

Qualifications and Skills:
  • 3-6 years professional experience in Affiliate Marketing.
  • Working knowledge of HasOffers, Affise, Trackier, and other Tracking Platforms.
  • Bachelors or above from an accredited university or college
  • Candidates should have very good connection with Affiliates / Publishers / Ad networks. 
  • Core Affiliate Industry/Performance Marketing Experience.
  • Strong data analytical skills.
  • Team management skills as well as ability to work independently.
  • Thorough understanding of the types of fraud in Web or Mobile.
  • Must have existing relationship with wide array of Affiliates / Publishers / Ad Networks.
  • Strong understanding of attribution mechanisms, cookies, pixels, tracking platforms & MMPs


      Other Benefits:

  •  Insurance - Medical Insurance Coverage of 3,00,000 for individual, spouse and kids. *Option of including parents and higher coverage amount of 5,00,000 
  • Professional Development - Reimbursement towards relevant courses that can help you grow your career and build your knowledge level. 
  • Food - Lunch, Tea and Snacks. All you can eat! 
  • Work Culture- Flexible Result Oriented Work Environment 
  • Dress - We aren’t buttoned down. Wear anything you’re comfortable in. Wear any comfortable dress you like to the office – and be as comfortable at work 
  • Foosball - Whether you want to pick up the game or master your skills, you’ll find plenty of people to play with 
  • TGIF - We’ll find something to do at the end of each week. 
  • Work hard, party harder!

 

Read more
Efficacious Solutions P Ltd
Simran Anand
Posted by Simran Anand
Delhi
0 - 1 yrs
₹1.5L - ₹2L / yr
skill icon.NET
skill iconC#
ASP.NET
DOT
ASP.NET MVC

Should be able to take responsibility and work independently.

- Professional experience with C#, .NET & OOPS

- Skill for writing reusable C# libraries

Read more
Magikle Media
at Magikle Media
1 recruiter
Adithya Shreshti
Posted by Adithya Shreshti
Bengaluru (Bangalore)
0 - 2 yrs
₹3L - ₹5L / yr
Sales
Business Development
Telesales
Client Servicing
Sales Operations

We are looking for an ambitious and energetic Business Development Manager to help us expand our clientele. You will be the front of the company and will have the dedication to create and apply an effective sales strategy.

The goal is to drive sustainable financial growth through boosting sales and forging strong relationships with clients.

 

Responsibilities

  • Develop a growth strategy focused both on financial gain and customer satisfaction
  • Conduct research to identify new markets and customer needs
  • Arrange business meetings with prospective clients
  • Promote the company’s products/services addressing or predicting clients’ objectives
  • Prepare sales contracts ensuring adherence to law-established rules and guidelines
  • Keep records of sales, revenue, invoices etc.
  • Provide trustworthy feedback and after-sales support
  • Build long-term relationships with new and existing customers
  • Develop entry level staff into valuable salespeople 

 

Requirements

  • Proven working experience as a business development manager, sales executive or a relevant role
  • Proven sales track record
  • Experience in customer support is a plus
  • Proficiency in MS Office and CRM software (e.g. Salesforce)
  • Proficiency in English
  • Market knowledge
  • Communication and negotiation skills
  • Ability to build rapport
  • Time management and planning skills
  • BSc/BA in business administration, sales or relevant field
Read more
company logo
Agency job
via nVipani Technology Solution by Sreedhar Reddy G
Bengaluru (Bangalore)
2 - 8 yrs
₹4L - ₹16L / yr
skill iconAngularJS (1.x)
skill iconJavascript
AJAX
JSON
MEAN stack
+3 more
3+ years of experience in building successful Angular JS applications. Excellent programming knowledge and experience in AngularJS. Strong work experience in HTML5, CSS3, JavaScript, AJAX and JSON. Solid understanding of the latest MVC frameworks and UI toolkits. Experience with GruntJS and GIT. Writing extensive unit tests using frameworks and test runners like Jasmine, Mocha, Karma, etc. Experience in handling Cross-browser compatibility issues and ways to work around them. Ability to create clean, light, scalable code with clear documentation. Ability to write appropriate unit tests, integration tests and deployment scripts. Ability to estimate, plan, and execute complex technical projects. Experience influencing product direction in creative ways. Experience in Angular Material Designs. Experience in bootstrap styles.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos