Cutshort logo
MeltPlan logo
AI System QA Engineer – Large Language Models (Evaluation Testing)
AI System QA Engineer – Large Language Models (Evaluation Testing)
MeltPlan's logo

AI System QA Engineer – Large Language Models (Evaluation Testing)

Gaurav Bhardwaj's profile picture
Posted by Gaurav Bhardwaj
5 - 7 yrs
Best in industry
Bengaluru (Bangalore), Mumbai, Delhi, Gurugram, Noida, Ghaziabad, Faridabad, Pune, Hyderabad
Skills
Data-driven testing
Playwright
DeepEval
Ragas
LLM Evaluation Frameworks
MLOps

MeltPlan is building the “planning engine” for the $14 Tn construction industry, an AI system designed specifically to optimize decisions before construction begins. While design software optimizes use and aesthetics and construction software optimizes execution and control, MeltPlan is building the missing layer - software that optimizes decisions and tradeoffs upstream, before scope is locked, procurement begins, and change orders become inevitable. MeltPlan’s long-term goal is to help teams make construction “boring” by making planning more intense: surfacing constraints and tradeoffs early, aligning stakeholders before plans are frozen, and reducing the need for late-stage redlines, rework, and change orders.


MeltPlan is founded by operators who have built at scale. Kanav previously co-founded Innovaccer, a $3Bn healthtech company focused on making US healthcare more affordable and accessible. He’s now applying that systems-level thinking to construction.He’s joined by Tanmaya Kala, former Project Executive at DPR Construction, who led large commercial, healthcare, and life sciences projects. We combine deep tech scale with real construction execution.


What This Role Really is :


We are seeking a detail-oriented and technically strong AI QA Engineer to ensure the quality, reliability, and performance of Large Language Model (LLM)-based systems. In this role, you will be responsible for designing and executing test strategies, validating model outputs, and building evaluation frameworks to enhance the accuracy, safety, and overall performance of AI-driven applications.We would particularly value candidates who have hands-on experience in developing evaluation frameworks (evals) for AI systems, along with strong expertise in comprehensive system testing and quality assurance practices.You are responsible for making MeltPlan work in the real world.


What You'll Do:


  • Design, develop, and execute evaluation frameworks (Evals) for Large Language Models (LLMs) and AI systems.
  • Perform end-to-end system testing, regression testing, and performance testing for AI-driven applications.
  • Validate model outputs for accuracy, consistency, safety, hallucination detection, and edge cases.
  • Build automated test pipelines and quality benchmarks for AI systems.
  • Collaborate closely with AI/ML engineers, product teams, and platform engineers to improve system reliability.
  • Analyze failures, identify root causes, and provide actionable feedback to improve model behavior.
  • Develop datasets, prompts, and testing scenarios to measure model performance across multiple use cases.
  • Monitor production performance and continuously improve evaluation metrics and testing standards.
  • Ensure compliance with responsible AI and quality assurance best practices.


What We're looking for: 


  • Bachelor’s degree in Computer Science, Engineering, or related field
  • 5–7 years of experience in QA/testing, preferably in AI/ML or data-driven systems
  • Strong experience in AI/LLM evaluation frameworks and system testing.
  • Hands-on experience with automated testing methodologies and QA processes.
  • Familiarity with prompt engineering, AI benchmarking, and model validation techniques.
  • Experience working with Python and testing frameworks.
  • Understanding of LLM behaviors, hallucinations, prompt injection risks, and AI safety concepts.
  • Exposure to tools/frameworks such as OpenAI Evals, LangSmith, DeepEval, Promptfoo, or similar platforms is preferred.
  • Strong analytical and debugging skills with attention to detail.
  • Excellent collaboration and communication skills.
  • Familiarity with Large Language Models and Generative AI concepts
  • Experience with API testing tools (e.g., Postman) and automation frameworks
  • Understanding of NLP concepts such as tokenization, embeddings, and text generation
  • Strong analytical and problem-solving skills
  • Experience testing AI/ML models or data pipelines
  • Experience with prompt engineering and prompt testing
  • Familiarity with cloud platforms (AWS, GCP, or Azure)
  • Exposure to AI safety, bias detection, and model governance


Bonus if you have:


  • Have worked in construction or on project sites
  • Have startup experience
  • Experience working with Generative AI or conversational AI products.
  • Knowledge of CI/CD pipelines and automation workflows.
  • Prior experience in performance testing and monitoring distributed systems.
  • Understanding of AI product lifecycle and production deployment environments.


We’re not looking for someone who waits for clean requirements.We’re looking for someone who thrives in the mess and turns it into systems.


Why meltplan

  • Massive industry, real-world impact
  • High ownership from day one
  • Small team, zero bureaucracy
  • Competitive comp + meaningful equity
Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About MeltPlan

Founded :
2024
Type :
Product
Size :
20-100
Stage :
Raised funding

About

MeltPlan is the AI-native planning software for construction – automate takeoffs with AI-assisted service, ensure building code compliance, and streamline cost planning with transparent, expert AI that shows its work. Designed for architects, engineers, contractors, and inspectors.
Read more

Company social profiles

blog

Similar jobs

CometChat
Rachel Angelin
Posted by Rachel Angelin
Mumbai
2.5 - 7 yrs
₹14L - ₹20L / yr
skill iconiOS App Development
skill iconSwift
UIKit

CometChat Overview

CometChat is a full-stack conversational platform built to unify every layer of interaction - bringing together real-time conversations (chat, messaging, voice, and video), AI Agents, moderation, notifications, and analytics in one modular, developer-first solution.


We were also recognized on Forbes' 2026 America's Top 500 Startup Employers List - a reflection of the team and culture we've been deliberate about building.


We believe the interface of the future is conversation - not clicks. Every app will soon have an AI layer that's as native as text messaging today. That's why we're building the infrastructure for the world's AI-powered conversations - from human-to-human, to human-to-agent, to multi-party collaboration with AI in the mix.


From AI onboarding assistants that get users productive in minutes, to copilots that perform complex workflows in-app, to intelligent moderators that protect and guide communities in real time - our AI Agent platform makes it all possible.


With CometChat's ready-to-use UI kits, powerful SDKs, and our Full Stack AI Agent Platform, product teams across startups and enterprises can launch safe, scalable, and smart in-app interactions faster than ever.


Why Join Us Now

We're at the tipping point where AI becomes a native part of every conversation. At CometChat, you'll help shape a future where users can talk to their apps as naturally as to a friend - where agents think, reason, act, and collaborate with humans in real time.


You won't just be joining a product team - you'll be building the standard for AI interaction layers: agent-aware UI, intelligent guardrails, rich actions libraries, and multi-party collaboration between people and AI.

If you want to help define how the next billion users will communicate - and push the boundaries of what's possible in real-time, AI-powered engagement - we'd love to work with you.


What we mean by AI-native

AI-native here means you build with agents by default. We expect 100% of the code to be generated via agentic tools (Cursor, Claude Code, and similar), while you own everything that actually matters: problem framing, architecture, tradeoffs, review quality, tests, performance, and security. This is not "vibe coding." The code is generated. The thinking is yours. You should be great at steering agents, validating output, catching subtle issues early, and debugging code you did not manually write.


Why AI-native

Software development is changing permanently, and we are leaning all the way in. You will work in a team that ships fast with agents, where your leverage comes from judgment and systems thinking rather than typing speed. You will get access to whatever AI tools you need (Cursor, Claude Code, or anything else that makes you effective), and the freedom to use them aggressively. If you want to operate at the edge of how modern engineering gets done and level up your output, this role is built for that.

Join us to build where AI meets human connection.


Position Overview & Priorities


We are looking for versatile and experienced technical additions to our development team. The position offers an extensive amount of ownership and influence over our development process as we scale the team. We’re looking for people who enjoy solving meaningful problems and love seeing the things they build in the hands of real users.


Primary responsibility would be:


  • Designing and building applications/SDK for the iOS platform
  • Ensuring the performance, quality, and responsiveness of applications
  • Collaborating with a team to define, design, and ship new features
  • Defining correct architecture and following the right design principles.
  • Helping maintain code quality, organization, and automatization


Work Location


Chembur, Mumbai


Prioritized Experiences and Capabilities


  • Minimum 2-5 years of experience in iOS app development
  • Leverage AI-native development workflows, including agentic IDEs and coding assistants (e.g., Cursor, Claude Code), to accelerate delivery while maintaining code quality, security, and engineering standards.
  • Proficiency with Objective-C and Swift.
  • Experience in iOS frameworks such as Core Data, Core Animation, etc.
  • Experience with offline storage, threading, and performance tuning
  • Familiarity with RESTful APIs to connect iOS applications to back-end services
  • Knowledge of other web technologies and UI/UX standards
  • Understanding of Apple’s design principles and interface guidelines
  • Knowledge of low-level C-based libraries is preferred
  • Familiarity with cloud message APIs and push notifications
  • Knack for benchmarking and optimization
  • Proficient understanding of code versioning tools
  • Familiarity with continuous integration


Read more
PeopleX Ventures
at PeopleX Ventures
2 candid answers
Tanisha Sanyal
Posted by Tanisha Sanyal
Bengaluru (Bangalore)
1 - 5 yrs
₹6L - ₹7L / yr
Presales
Communication Skills
Lead Generation

Job description:

Location: Bangalore

Work Days: Either Tuesday to Sunday or Wednesday to Monday

Work Hours: 10:00 AM – 7:00 PM

Language Requirement: English communication is mandatory

Experience: Minimum 1+ years in a calling role

Cab Facility: Provided after 7:30 PM (for women only)

Role Summary

We are seeking dynamic and motivated individuals to join our Sales & Marketing team as Property Specialists. In this role, you will be the first point of contact for potential clients, creating a strong first impression and helping them navigate the buying and selling process. This position offers a mix of office and on-site work, giving you the opportunity to collaborate closely with the Sales team and support client visits.

Key Responsibilities

  • Call and engage with a specific number of qualified leads per month for property buying and selling.
  • Communicate with property owners to understand their requirements.
  • Virtually showcase apartments to prospective buyers and assist the Sales team in closing deals.
  • Maintain accurate and timely data entry in the CRM system.
  • Generate dashboards and reports from CRM for stakeholders.
  • Build and maintain a database of potential and current sellers/buyers.
  • Coordinate with Operations team for home inspections.
  • Explain and demonstrate property features to clients.
  • Stay updated on competing products and services in the market.
  • Work towards achieving monthly sales targets with the Sales and BD team.

What We’re Looking For

  • 1–4 years of experience in a calling or customer-facing role (real estate experience preferred but not mandatory).
  • Strong communication skills in English (knowledge of an additional Indian language is a plus).
  • Organisational and multitasking skills with attention to detail.
  • Enthusiasm to learn and adapt to CRM systems for reporting and record maintenance.
  • Ability to thrive in a fast-paced, target-driven environment.
  • Willingness to work on weekends (as they are peak days in real estate).
  • Positive, collaborative, and motivated mindset.

Why Join Us?

  • Exciting opportunity in a fast-growing real estate sector.
  • Exposure to both office and on-site sales activities.
  • Supportive work environment with learning opportunities.
  • Cab service available for women employees post 7:30 PM.

Job Type: Full-time

Work Schedule: 6 days a week (weekends working, 1 weekday off)

Job Types: Full-time, Permanent

Application Question(s):

  • Are you comfotable to work on week ends( one day in a week off)

Experience:

  • Pre-sales: 2 years (Required)

Language:

  • English (Required)

Location:

  • Bengaluru, Karnataka (Required)

Willingness to travel:

  • 75% (Required)

Work Location: In person

Read more
Blend InfoTech
Minal Ahluwalia
Posted by Minal Ahluwalia
Pune
5 - 7 yrs
₹8L - ₹11L / yr
skill iconNodeJS (Node.js)
skill iconJavascript
TypeScript

Job Title: Senior Backend Developer

Location: Mali mahajan road, somwar peth, Pune.

Experience: 5-7 years


Responsibilities:


Backend Development:

  • Design, develop, and maintain scalable and high-performance backend systems.
  1. Utilize AWS Serverless skills (API Gateway, Lambda, DynamoDB, SQS, Event bridge, CloudWatch, permissions, accounts, multi accounts, streams


Technology Stack:

  • Proficiency in JavaScript, Node.js, and Typescript for backend development.
  • Leverage AWS services to build serverless applications with a focus on efficiency, security, and reliability.


E-commerce Expertise:

  • Bring experience in e-commerce projects, with additional consideration for knowledge of Commerce tools.


Communication Skills:

  • Communicate effectively with stakeholders across the US, UK, and Australia.
  • Collaborate with cross-functional teams to understand requirements and provide technical insights.


Project Collaboration:

  • Work closely with front-end developers, QA engineers, and other team members to deliver end-to-end solutions.


Requirements:


  1. Bachelor’s degree in Computer Science or a related field.
  2. 5-7 years of backend development experience.
  3. Strong proficiency in AWS Serverless technologies.
  4. Expertise in JavaScript, Node.js, and Typescript.
  5. Experience with e-commerce projects, particularly Commercetools, is an advantage.
  6. Excellent communication skills to engage with international stakeholders.


Read more
DAZN
at DAZN
Shivani Sharma
Posted by Shivani Sharma
Hyderabad
4 - 8 yrs
Best in industry
skill iconData Analytics
Data Visualization
PowerBI
Tableau
Qlikview
+7 more

Is your next career move to work in a team which uses data, reporting and analytical skills to help answer business questions to make DAZN a data-driven company?

 

DAZN is a tech-first sport streaming platform that reaches millions of users every week. We are challenging a traditional industry and giving power back to the fans. Our new Hyderabad tech hub will be the engine that drives us forward to the future. We’re pushing boundaries and doing things no-one has done before. Here, you have the opportunity to make your mark and the power to make change happen - to make a difference for our customers. When you join DAZN you will work on projects that impact millions of lives thanks to your critical contributions to our global products

 

This is the perfect place to work if you are passionate about technology and want an opportunity to use your creativity to help grow and scale a global range of IT systems, Infrastructure, and IT Services. Our cutting-edge technology allows us to stream sports content to millions of concurrent viewers globally across multiple platforms and devices. DAZN’s Cloud based architecture unifies a range of technologies in order to deliver a seamless user experience and support a global user base and company infrastructure.

 

This role will be based in our brand-new Hyderabad office. Join us in India’s beautiful “City of Pearls” and bring your ambition to life.

 

Responsibilities:

 

  • Communicate with different stakeholders such as Ad Tech Engineers and Product Owners
  • Should be able to extensively work in Google Analytics and strong SQL knowledge is expected.
  • Strong analytical skills

 

Key Competencies:

 

  • 4-8 years of experience as Data Analyst
  • Advanced Microsoft Excel Skills
  • Strong command on Google Analytics
  • Reporting platform UI experience (Tableau, Looker, etc)
  • Experience with VAST tags, pixels trackers, etc.
  • Experience with DSPs & third-party ad platforms (GAM, YoSpace, etc)

 

At DAZN, we bring ambition to life. We are innovators, game-changers and pioneers. So, if you want to push boundaries and make an impact, DAZN is the place to be.

 

As part of our team, you'll have the opportunity to make your mark and the power to make change happen. We're doing things no-one has done before, giving fans and customers access to sport anytime, anywhere. We're using world-class technology to transform sports and revolutionise the industry and we're not going to stop.

 

 

Read more
MBA and Beyond
at MBA and Beyond
1 recruiter
Ritika Sharma
Posted by Ritika Sharma
Remote only
2 - 7 yrs
₹6L - ₹15L / yr
skill iconAngularJS (1.x)
skill iconAngular (2+)
skill iconReact.js
skill iconNodeJS (Node.js)
skill iconMongoDB
+4 more

We are looking for a Full Stack Developer, to be a core member of our Engineering Team, who is a great problem solver, can learn quickly, and communicate clearly. You like to work in a fast-paced environment, want to own the work, get recognized for it and therefore startup environment excites you.

 

Responsibilities

  • Work on end to end website development including frontend, backend and deployment. Build useful and handy tools in field of Admissions and consultancy
  • Work on the core platform for Higher Education Aspirants with respect to admissions, resume building, interview experiences and application for B-Schools
  • Build efficient, testable, and reusable modules and components
  • REST API development for integration with frontend web components which would be easy to manage and scale
  • Accurately understand and translate business and user needs into functional backend or frontend code to build robust features
  • Participate in a culture of code reviews, writing tech specs, and collaborating closely with other people
  • Writing standalone services with business logic to support automation workflows and integrations
  • Support Marketing and Operations team with small tools, scripts or automations across multiple tools

 

Criteria

Hunger for learning and getting out of comfort zone to build amazing web applications & websites would be enough. Otherwise

  • Basic understanding of backend, database and Server technologies
  • Knowledge of Frontend technologies (HTML, CSS, JS, ReactJS, VueJS)
  • Knowledge of atleast one Backend technologies (NodeJS, Python, ROR, GoLang etc)
  • Understanding and knowledge of Database technologies (MySQL, MongoDB)
  • Clear understanding of RESTful API development standards.
  • Creating database schemas that represent and support business processes. Integration of multiple data sources and databases into one system.

 

About Company

MBA & Beyond is a global admission consulting startup for applicants who dare to question their purpose with a global MBA. We help purpose-driven applicants make it to the top business schools.

Read more
SHRI RAJESHWARI INDUSTRIAL ESSENTIALS
preethi bk
Posted by preethi bk
Bengaluru (Bangalore)
8 - 12 yrs
₹10L - ₹14L / yr
BIM
Autodesk Revit

1

Good Knowledge about utilization of BIM format in implementation of the Architecture model

2

Working Knowledge of Structural & Architectural services modelling for larger and complex International projects.

3

Working Knowledge of coordinating, raising RFIs and submitting the final models with good presentation as per the demand.

4

Excellent Teamwork, Coordination ability with good communication.

5

Ability to generate various Architectural layouts, sections, elevations and details

6

Modelling the project understanding the LOD with tolerance

7

Strong knowledge in Building Architecture Domain

8

Works with Project Lead and project engineers to communicate, problem solve, and update models for coordination and design issues.

9

Develops accurate 2D & 3D drawings from project BIMs, including sheet management tasks, annotations, dimensions, notes and all visibility settings.

10

Knowledge of relevant industry standards and codes and the ability to prepare a full DA or CD submission understanding BCA and relevent Australian Standards

11

Quality of Drafting, Presnetation skills and technically sound

12

Performs both routine and non-routine, complex drafting assignments that require judgment in resolving issues or making recommendations.

13

Collaborates and communicates with other disciplines regarding coordination issues and common model development tasks.

14

Excellent REVIT skills in Modelling, Detailing and content creations

15

Ability to produce the work First Time Right & Time Management

Read more
For a builer, construction site
For a builer, construction site
Agency job
via Sira placement consultancy by Aarti Keshav
Pune
4 - 5 yrs
₹6L - ₹7L / yr
Digital Marketing
Google Adwords
Web Analytics
skill iconGoogle Analytics
Facebook Marketing
+1 more
5 Years of experience as Digital Marketer in Real Estate Industry is a must
Excellent understanding of digital marketing concepts and best practices
Experience with B2C social media, Google Adwords and email campaigns and SEO/SEM
Working knowledge of ad serving tools (e.g., DART, Atlas)
Perfect knowledge of web analytics tools (e.g. Google Analytics, NetInsight, WebTrends etc.)
Skills and experience in creative content writing
Analytical mindset and critical thinking
Excellent communication and interpersonal skills
Read more
education asia
at education asia
1 video
1 recruiter
HR Bijeta
Posted by HR Bijeta
Remote only
1 - 3 yrs
₹1L - ₹2L / yr
Reporting
Content Writing
Copy Writing
Interviewing
Technical editing

Responsibilities:

  • Interact & Interview Chairperson of college or university.
  • Featured & Research oriented articles about the Education System.
  • Interview of famous personalities from the educational industry.
  • Stories of success from leading professionals.

Requirements:

  • Proven working experience as a Reporter.
  • Portfolio of published articles.
  • Computer proficiency (MS Office, digital editing, web search, databases)
  • Excellent communication, lobbying, and active listening skills.
  • Good hold in education industry persons.
  • Bachelors degree in journalism or mass communications.

Contact Details:

Website : www.educationasia.in

Read more
Dhwani Rural Information Systems
at Dhwani Rural Information Systems
1 candid answer
3 recruiters
Sunandan Madan
Posted by Sunandan Madan
NCR (Delhi | Gurgaon | Noida)
1 - 2 yrs
₹2L - ₹4L / yr
skill iconAngularJS (1.x)
skill iconReact.js
skill iconJavascript
Create front-end applications on javascript, HTML and CSS. Integrate front-end with back-end through API's Create interactive dashboards Explore various front-end framework based on Angular and React Js. Be a part of the startup team. Work on social projects in rural areas.
Read more
TEJAS NETWORKS LIMITED
at TEJAS NETWORKS LIMITED
1 recruiter
Silpa Balakrishnan
Posted by Silpa Balakrishnan
Bengaluru (Bangalore), Mumbai, NCR (Delhi | Gurgaon | Noida)
3 - 12 yrs
₹0L - ₹30L / yr
c/c++
sdh
sonet
DWDM
optical
+3 more
Packet-Optical Group 3+ years of development experience in DWDM/OTN/SDH/EoS/EoOTN technology. Embedded Software (Bangalore, Gurgaon, Mumbai) o Development experience in embedded S/W design, programming and debugging in C/C++, in embedded Linux environment o Experience with carrier class system capabilities like Redundancy, High Availability, Hot Standby, Hit-less upgrade is highly desirable Carrier Ethernet Group 3+ years’ development experience in Carrier Ethernet and MPLS-TP technologies including Ethernet/MPLS-TP Protection, Ethernet/MPLS-TP OAM, Synchronous Ethernet, Circuit Emulation, PTP 1588. Embedded Software (Bangalore, Gurgaon, Mumbai) About Tejas Networks Tejas is a pioneer in developing cost effective, next-generation optical networking products that enable telecom carriers to converge traditional voice-based transmission networks with the new data-dominated networks. Tejas has successfully deployed over 350,000 of its systems in the networks of major telecom operators in more than 60 countries. We provide an end-to-end portfolio of optical and data networking products covering access, metro and long-haul networks. Our products are used to build communication networks that carry voice, data and video traffic from fixed line, mobile and broadband networks over optical fibre. We offer a diverse portfolio of products based on global technology standards, which enable telecommunications networks that are used to provide mobile, internet and broadband services, primarily over optical fibre.
Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos