
AI System QA Engineer – Large Language Models (Evaluation Testing)
at MeltPlan
MeltPlan is building the “planning engine” for the $14 Tn construction industry, an AI system designed specifically to optimize decisions before construction begins. While design software optimizes use and aesthetics and construction software optimizes execution and control, MeltPlan is building the missing layer - software that optimizes decisions and tradeoffs upstream, before scope is locked, procurement begins, and change orders become inevitable. MeltPlan’s long-term goal is to help teams make construction “boring” by making planning more intense: surfacing constraints and tradeoffs early, aligning stakeholders before plans are frozen, and reducing the need for late-stage redlines, rework, and change orders.
MeltPlan is founded by operators who have built at scale. Kanav previously co-founded Innovaccer, a $3Bn healthtech company focused on making US healthcare more affordable and accessible. He’s now applying that systems-level thinking to construction.He’s joined by Tanmaya Kala, former Project Executive at DPR Construction, who led large commercial, healthcare, and life sciences projects. We combine deep tech scale with real construction execution.
What This Role Really is :
We are seeking a detail-oriented and technically strong AI QA Engineer to ensure the quality, reliability, and performance of Large Language Model (LLM)-based systems. In this role, you will be responsible for designing and executing test strategies, validating model outputs, and building evaluation frameworks to enhance the accuracy, safety, and overall performance of AI-driven applications.We would particularly value candidates who have hands-on experience in developing evaluation frameworks (evals) for AI systems, along with strong expertise in comprehensive system testing and quality assurance practices.You are responsible for making MeltPlan work in the real world.
What You'll Do:
- Design, develop, and execute evaluation frameworks (Evals) for Large Language Models (LLMs) and AI systems.
- Perform end-to-end system testing, regression testing, and performance testing for AI-driven applications.
- Validate model outputs for accuracy, consistency, safety, hallucination detection, and edge cases.
- Build automated test pipelines and quality benchmarks for AI systems.
- Collaborate closely with AI/ML engineers, product teams, and platform engineers to improve system reliability.
- Analyze failures, identify root causes, and provide actionable feedback to improve model behavior.
- Develop datasets, prompts, and testing scenarios to measure model performance across multiple use cases.
- Monitor production performance and continuously improve evaluation metrics and testing standards.
- Ensure compliance with responsible AI and quality assurance best practices.
What We're looking for:
- Bachelor’s degree in Computer Science, Engineering, or related field
- 5–7 years of experience in QA/testing, preferably in AI/ML or data-driven systems
- Strong experience in AI/LLM evaluation frameworks and system testing.
- Hands-on experience with automated testing methodologies and QA processes.
- Familiarity with prompt engineering, AI benchmarking, and model validation techniques.
- Experience working with Python and testing frameworks.
- Understanding of LLM behaviors, hallucinations, prompt injection risks, and AI safety concepts.
- Exposure to tools/frameworks such as OpenAI Evals, LangSmith, DeepEval, Promptfoo, or similar platforms is preferred.
- Strong analytical and debugging skills with attention to detail.
- Excellent collaboration and communication skills.
- Familiarity with Large Language Models and Generative AI concepts
- Experience with API testing tools (e.g., Postman) and automation frameworks
- Understanding of NLP concepts such as tokenization, embeddings, and text generation
- Strong analytical and problem-solving skills
- Experience testing AI/ML models or data pipelines
- Experience with prompt engineering and prompt testing
- Familiarity with cloud platforms (AWS, GCP, or Azure)
- Exposure to AI safety, bias detection, and model governance
Bonus if you have:
- Have worked in construction or on project sites
- Have startup experience
- Experience working with Generative AI or conversational AI products.
- Knowledge of CI/CD pipelines and automation workflows.
- Prior experience in performance testing and monitoring distributed systems.
- Understanding of AI product lifecycle and production deployment environments.
We’re not looking for someone who waits for clean requirements.We’re looking for someone who thrives in the mess and turns it into systems.
Why meltplan
- Massive industry, real-world impact
- High ownership from day one
- Small team, zero bureaucracy
- Competitive comp + meaningful equity

About MeltPlan
About
Similar jobs
CometChat Overview
CometChat is a full-stack conversational platform built to unify every layer of interaction - bringing together real-time conversations (chat, messaging, voice, and video), AI Agents, moderation, notifications, and analytics in one modular, developer-first solution.
We were also recognized on Forbes' 2026 America's Top 500 Startup Employers List - a reflection of the team and culture we've been deliberate about building.
We believe the interface of the future is conversation - not clicks. Every app will soon have an AI layer that's as native as text messaging today. That's why we're building the infrastructure for the world's AI-powered conversations - from human-to-human, to human-to-agent, to multi-party collaboration with AI in the mix.
From AI onboarding assistants that get users productive in minutes, to copilots that perform complex workflows in-app, to intelligent moderators that protect and guide communities in real time - our AI Agent platform makes it all possible.
With CometChat's ready-to-use UI kits, powerful SDKs, and our Full Stack AI Agent Platform, product teams across startups and enterprises can launch safe, scalable, and smart in-app interactions faster than ever.
Why Join Us Now
We're at the tipping point where AI becomes a native part of every conversation. At CometChat, you'll help shape a future where users can talk to their apps as naturally as to a friend - where agents think, reason, act, and collaborate with humans in real time.
You won't just be joining a product team - you'll be building the standard for AI interaction layers: agent-aware UI, intelligent guardrails, rich actions libraries, and multi-party collaboration between people and AI.
If you want to help define how the next billion users will communicate - and push the boundaries of what's possible in real-time, AI-powered engagement - we'd love to work with you.
What we mean by AI-native
AI-native here means you build with agents by default. We expect 100% of the code to be generated via agentic tools (Cursor, Claude Code, and similar), while you own everything that actually matters: problem framing, architecture, tradeoffs, review quality, tests, performance, and security. This is not "vibe coding." The code is generated. The thinking is yours. You should be great at steering agents, validating output, catching subtle issues early, and debugging code you did not manually write.
Why AI-native
Software development is changing permanently, and we are leaning all the way in. You will work in a team that ships fast with agents, where your leverage comes from judgment and systems thinking rather than typing speed. You will get access to whatever AI tools you need (Cursor, Claude Code, or anything else that makes you effective), and the freedom to use them aggressively. If you want to operate at the edge of how modern engineering gets done and level up your output, this role is built for that.
Join us to build where AI meets human connection.
Position Overview & Priorities
We are looking for versatile and experienced technical additions to our development team. The position offers an extensive amount of ownership and influence over our development process as we scale the team. We’re looking for people who enjoy solving meaningful problems and love seeing the things they build in the hands of real users.
Primary responsibility would be:
- Designing and building applications/SDK for the iOS platform
- Ensuring the performance, quality, and responsiveness of applications
- Collaborating with a team to define, design, and ship new features
- Defining correct architecture and following the right design principles.
- Helping maintain code quality, organization, and automatization
Work Location
Chembur, Mumbai
Prioritized Experiences and Capabilities
- Minimum 2-5 years of experience in iOS app development
- Leverage AI-native development workflows, including agentic IDEs and coding assistants (e.g., Cursor, Claude Code), to accelerate delivery while maintaining code quality, security, and engineering standards.
- Proficiency with Objective-C and Swift.
- Experience in iOS frameworks such as Core Data, Core Animation, etc.
- Experience with offline storage, threading, and performance tuning
- Familiarity with RESTful APIs to connect iOS applications to back-end services
- Knowledge of other web technologies and UI/UX standards
- Understanding of Apple’s design principles and interface guidelines
- Knowledge of low-level C-based libraries is preferred
- Familiarity with cloud message APIs and push notifications
- Knack for benchmarking and optimization
- Proficient understanding of code versioning tools
- Familiarity with continuous integration
Job description:
Location: Bangalore
Work Days: Either Tuesday to Sunday or Wednesday to Monday
Work Hours: 10:00 AM – 7:00 PM
Language Requirement: English communication is mandatory
Experience: Minimum 1+ years in a calling role
Cab Facility: Provided after 7:30 PM (for women only)
Role Summary
We are seeking dynamic and motivated individuals to join our Sales & Marketing team as Property Specialists. In this role, you will be the first point of contact for potential clients, creating a strong first impression and helping them navigate the buying and selling process. This position offers a mix of office and on-site work, giving you the opportunity to collaborate closely with the Sales team and support client visits.
Key Responsibilities
- Call and engage with a specific number of qualified leads per month for property buying and selling.
- Communicate with property owners to understand their requirements.
- Virtually showcase apartments to prospective buyers and assist the Sales team in closing deals.
- Maintain accurate and timely data entry in the CRM system.
- Generate dashboards and reports from CRM for stakeholders.
- Build and maintain a database of potential and current sellers/buyers.
- Coordinate with Operations team for home inspections.
- Explain and demonstrate property features to clients.
- Stay updated on competing products and services in the market.
- Work towards achieving monthly sales targets with the Sales and BD team.
What We’re Looking For
- 1–4 years of experience in a calling or customer-facing role (real estate experience preferred but not mandatory).
- Strong communication skills in English (knowledge of an additional Indian language is a plus).
- Organisational and multitasking skills with attention to detail.
- Enthusiasm to learn and adapt to CRM systems for reporting and record maintenance.
- Ability to thrive in a fast-paced, target-driven environment.
- Willingness to work on weekends (as they are peak days in real estate).
- Positive, collaborative, and motivated mindset.
Why Join Us?
- Exciting opportunity in a fast-growing real estate sector.
- Exposure to both office and on-site sales activities.
- Supportive work environment with learning opportunities.
- Cab service available for women employees post 7:30 PM.
Job Type: Full-time
Work Schedule: 6 days a week (weekends working, 1 weekday off)
Job Types: Full-time, Permanent
Application Question(s):
- Are you comfotable to work on week ends( one day in a week off)
Experience:
- Pre-sales: 2 years (Required)
Language:
- English (Required)
Location:
- Bengaluru, Karnataka (Required)
Willingness to travel:
- 75% (Required)
Work Location: In person
Job Title: Senior Backend Developer
Location: Mali mahajan road, somwar peth, Pune.
Experience: 5-7 years
Responsibilities:
Backend Development:
- Design, develop, and maintain scalable and high-performance backend systems.
- Utilize AWS Serverless skills (API Gateway, Lambda, DynamoDB, SQS, Event bridge, CloudWatch, permissions, accounts, multi accounts, streams
Technology Stack:
- Proficiency in JavaScript, Node.js, and Typescript for backend development.
- Leverage AWS services to build serverless applications with a focus on efficiency, security, and reliability.
E-commerce Expertise:
- Bring experience in e-commerce projects, with additional consideration for knowledge of Commerce tools.
Communication Skills:
- Communicate effectively with stakeholders across the US, UK, and Australia.
- Collaborate with cross-functional teams to understand requirements and provide technical insights.
Project Collaboration:
- Work closely with front-end developers, QA engineers, and other team members to deliver end-to-end solutions.
Requirements:
- Bachelor’s degree in Computer Science or a related field.
- 5-7 years of backend development experience.
- Strong proficiency in AWS Serverless technologies.
- Expertise in JavaScript, Node.js, and Typescript.
- Experience with e-commerce projects, particularly Commercetools, is an advantage.
- Excellent communication skills to engage with international stakeholders.
Is your next career move to work in a team which uses data, reporting and analytical skills to help answer business questions to make DAZN a data-driven company?
DAZN is a tech-first sport streaming platform that reaches millions of users every week. We are challenging a traditional industry and giving power back to the fans. Our new Hyderabad tech hub will be the engine that drives us forward to the future. We’re pushing boundaries and doing things no-one has done before. Here, you have the opportunity to make your mark and the power to make change happen - to make a difference for our customers. When you join DAZN you will work on projects that impact millions of lives thanks to your critical contributions to our global products
This is the perfect place to work if you are passionate about technology and want an opportunity to use your creativity to help grow and scale a global range of IT systems, Infrastructure, and IT Services. Our cutting-edge technology allows us to stream sports content to millions of concurrent viewers globally across multiple platforms and devices. DAZN’s Cloud based architecture unifies a range of technologies in order to deliver a seamless user experience and support a global user base and company infrastructure.
This role will be based in our brand-new Hyderabad office. Join us in India’s beautiful “City of Pearls” and bring your ambition to life.
Responsibilities:
- Communicate with different stakeholders such as Ad Tech Engineers and Product Owners
- Should be able to extensively work in Google Analytics and strong SQL knowledge is expected.
- Strong analytical skills
Key Competencies:
- 4-8 years of experience as Data Analyst
- Advanced Microsoft Excel Skills
- Strong command on Google Analytics
- Reporting platform UI experience (Tableau, Looker, etc)
- Experience with VAST tags, pixels trackers, etc.
- Experience with DSPs & third-party ad platforms (GAM, YoSpace, etc)
At DAZN, we bring ambition to life. We are innovators, game-changers and pioneers. So, if you want to push boundaries and make an impact, DAZN is the place to be.
As part of our team, you'll have the opportunity to make your mark and the power to make change happen. We're doing things no-one has done before, giving fans and customers access to sport anytime, anywhere. We're using world-class technology to transform sports and revolutionise the industry and we're not going to stop.
We are looking for a Full Stack Developer, to be a core member of our Engineering Team, who is a great problem solver, can learn quickly, and communicate clearly. You like to work in a fast-paced environment, want to own the work, get recognized for it and therefore startup environment excites you.
Responsibilities
- Work on end to end website development including frontend, backend and deployment. Build useful and handy tools in field of Admissions and consultancy
- Work on the core platform for Higher Education Aspirants with respect to admissions, resume building, interview experiences and application for B-Schools
- Build efficient, testable, and reusable modules and components
- REST API development for integration with frontend web components which would be easy to manage and scale
- Accurately understand and translate business and user needs into functional backend or frontend code to build robust features
- Participate in a culture of code reviews, writing tech specs, and collaborating closely with other people
- Writing standalone services with business logic to support automation workflows and integrations
- Support Marketing and Operations team with small tools, scripts or automations across multiple tools
Criteria
Hunger for learning and getting out of comfort zone to build amazing web applications & websites would be enough. Otherwise
- Basic understanding of backend, database and Server technologies
- Knowledge of Frontend technologies (HTML, CSS, JS, ReactJS, VueJS)
- Knowledge of atleast one Backend technologies (NodeJS, Python, ROR, GoLang etc)
- Understanding and knowledge of Database technologies (MySQL, MongoDB)
- Clear understanding of RESTful API development standards.
- Creating database schemas that represent and support business processes. Integration of multiple data sources and databases into one system.
About Company
MBA & Beyond is a global admission consulting startup for applicants who dare to question their purpose with a global MBA. We help purpose-driven applicants make it to the top business schools.
|
1 |
Good Knowledge about utilization of BIM format in implementation of the Architecture model |
|
2 |
Working Knowledge of Structural & Architectural services modelling for larger and complex International projects. |
|
3 |
Working Knowledge of coordinating, raising RFIs and submitting the final models with good presentation as per the demand. |
|
4 |
Excellent Teamwork, Coordination ability with good communication. |
|
5 |
Ability to generate various Architectural layouts, sections, elevations and details |
|
6 |
Modelling the project understanding the LOD with tolerance |
|
7 |
Strong knowledge in Building Architecture Domain |
|
8 |
Works with Project Lead and project engineers to communicate, problem solve, and update models for coordination and design issues. |
|
9 |
Develops accurate 2D & 3D drawings from project BIMs, including sheet management tasks, annotations, dimensions, notes and all visibility settings. |
|
10 |
Knowledge of relevant industry standards and codes and the ability to prepare a full DA or CD submission understanding BCA and relevent Australian Standards |
|
11 |
Quality of Drafting, Presnetation skills and technically sound |
|
12 |
Performs both routine and non-routine, complex drafting assignments that require judgment in resolving issues or making recommendations. |
|
13 |
Collaborates and communicates with other disciplines regarding coordination issues and common model development tasks. |
|
14 |
Excellent REVIT skills in Modelling, Detailing and content creations |
|
15 |
Ability to produce the work First Time Right & Time Management |
Excellent understanding of digital marketing concepts and best practices
Experience with B2C social media, Google Adwords and email campaigns and SEO/SEM
Working knowledge of ad serving tools (e.g., DART, Atlas)
Perfect knowledge of web analytics tools (e.g. Google Analytics, NetInsight, WebTrends etc.)
Skills and experience in creative content writing
Analytical mindset and critical thinking
Excellent communication and interpersonal skills
Responsibilities:
- Interact & Interview Chairperson of college or university.
- Featured & Research oriented articles about the Education System.
- Interview of famous personalities from the educational industry.
- Stories of success from leading professionals.
Requirements:
- Proven working experience as a Reporter.
- Portfolio of published articles.
- Computer proficiency (MS Office, digital editing, web search, databases)
- Excellent communication, lobbying, and active listening skills.
- Good hold in education industry persons.
- Bachelors degree in journalism or mass communications.
Contact Details:
Website : www.educationasia.in









