
AI System QA Engineer – Large Language Models (Evaluation Testing)
at MeltPlan
MeltPlan is building the “planning engine” for the $14 Tn construction industry, an AI system designed specifically to optimize decisions before construction begins. While design software optimizes use and aesthetics and construction software optimizes execution and control, MeltPlan is building the missing layer - software that optimizes decisions and tradeoffs upstream, before scope is locked, procurement begins, and change orders become inevitable. MeltPlan’s long-term goal is to help teams make construction “boring” by making planning more intense: surfacing constraints and tradeoffs early, aligning stakeholders before plans are frozen, and reducing the need for late-stage redlines, rework, and change orders.
MeltPlan is founded by operators who have built at scale. Kanav previously co-founded Innovaccer, a $3Bn healthtech company focused on making US healthcare more affordable and accessible. He’s now applying that systems-level thinking to construction.He’s joined by Tanmaya Kala, former Project Executive at DPR Construction, who led large commercial, healthcare, and life sciences projects. We combine deep tech scale with real construction execution.
What This Role Really is :
We are seeking a detail-oriented and technically strong AI QA Engineer to ensure the quality, reliability, and performance of Large Language Model (LLM)-based systems. In this role, you will be responsible for designing and executing test strategies, validating model outputs, and building evaluation frameworks to enhance the accuracy, safety, and overall performance of AI-driven applications.We would particularly value candidates who have hands-on experience in developing evaluation frameworks (evals) for AI systems, along with strong expertise in comprehensive system testing and quality assurance practices.You are responsible for making MeltPlan work in the real world.
What You'll Do:
- Design, develop, and execute evaluation frameworks (Evals) for Large Language Models (LLMs) and AI systems.
- Perform end-to-end system testing, regression testing, and performance testing for AI-driven applications.
- Validate model outputs for accuracy, consistency, safety, hallucination detection, and edge cases.
- Build automated test pipelines and quality benchmarks for AI systems.
- Collaborate closely with AI/ML engineers, product teams, and platform engineers to improve system reliability.
- Analyze failures, identify root causes, and provide actionable feedback to improve model behavior.
- Develop datasets, prompts, and testing scenarios to measure model performance across multiple use cases.
- Monitor production performance and continuously improve evaluation metrics and testing standards.
- Ensure compliance with responsible AI and quality assurance best practices.
What We're looking for:
- Bachelor’s degree in Computer Science, Engineering, or related field
- 5–7 years of experience in QA/testing, preferably in AI/ML or data-driven systems
- Strong experience in AI/LLM evaluation frameworks and system testing.
- Hands-on experience with automated testing methodologies and QA processes.
- Familiarity with prompt engineering, AI benchmarking, and model validation techniques.
- Experience working with Python and testing frameworks.
- Understanding of LLM behaviors, hallucinations, prompt injection risks, and AI safety concepts.
- Exposure to tools/frameworks such as OpenAI Evals, LangSmith, DeepEval, Promptfoo, or similar platforms is preferred.
- Strong analytical and debugging skills with attention to detail.
- Excellent collaboration and communication skills.
- Familiarity with Large Language Models and Generative AI concepts
- Experience with API testing tools (e.g., Postman) and automation frameworks
- Understanding of NLP concepts such as tokenization, embeddings, and text generation
- Strong analytical and problem-solving skills
- Experience testing AI/ML models or data pipelines
- Experience with prompt engineering and prompt testing
- Familiarity with cloud platforms (AWS, GCP, or Azure)
- Exposure to AI safety, bias detection, and model governance
Bonus if you have:
- Have worked in construction or on project sites
- Have startup experience
- Experience working with Generative AI or conversational AI products.
- Knowledge of CI/CD pipelines and automation workflows.
- Prior experience in performance testing and monitoring distributed systems.
- Understanding of AI product lifecycle and production deployment environments.
We’re not looking for someone who waits for clean requirements.We’re looking for someone who thrives in the mess and turns it into systems.
Why meltplan
- Massive industry, real-world impact
- High ownership from day one
- Small team, zero bureaucracy
- Competitive comp + meaningful equity

About MeltPlan
About
Similar jobs
Video Editor
Company Overview
We are a full-service marketing agency that helps ambitious brands show up sharper, grow faster and stand out in crowded markets. From content and social to performance marketing, design, video and PR, we build and execute brand strategies end-to-end for a diverse roster of clients across industries. We're a team that moves fast, takes creative risks, and holds itself to agency-grade standards on every single deliverable. If you want your work to ship, get seen, and actually move numbers for real brands rather than sit in a deck, you'll fit right in.
Role Summary
Transform raw footage into high-impact, brand-consistent video content across long-form and short-form formats.
Video Editor
Company Overview
We are a full-service marketing agency that helps ambitious brands show up sharper, grow faster and stand out in crowded markets. From content and social to performance marketing, design, video and PR, we build and execute brand strategies end-to-end for a diverse roster of clients across industries. We're a team that moves fast, takes creative risks, and holds itself to agency-grade standards on every single deliverable. If you want your work to ship, get seen, and actually move numbers for real brands rather than sit in a deck, you'll fit right in.
Role Summary
Transform raw footage into high-impact, brand-consistent video content across long-form and short-form formats.
Job Details
- Experience: 2-5 years
- Work Model: On-site
- Work Location: ATS Bouquet, Sector 132, Noida
- Work Week: Monday-Saturday, 9:30 AM - 6:30 PM
- Reports To: Creative Lead / Head of Content
- Education: Bachelor's degree in Film, Mass Communication or related field (or equivalent professional portfolio)
Key Responsibilities
- Edit long-form and short-form video content for social, marketing and brand
- Apply motion graphics, sound design, color grading and transitions
- Collaborate closely with content and social teams on story structure and pacing
- Organize and manage footage, project files and version control
- Deliver polished edits within tight, fast-paced turnarounds
- Repurpose long-form content into multiple platform-native short cuts
- Stay current with editing trends, transitions and sound design styles
- Maintain consistent visual quality and brand style across all output
- Troubleshoot technical issues with footage, audio and export settings
- Suggest creative improvements to pacing, structure and storytelling
Mandatory Skills & Experience
- 2-5 years of professional video editing experience
- Strong command of Premiere Pro, Final Cut Pro or DaVinci Resolve
- Proven sense of pacing, sound design and visual storytelling
- Demonstrated ability to deliver under tight, recurring deadlines
Preferred Skills (Good to Have)
- Motion graphics or basic animation skills (After Effects)
- Experience editing for a brand with high-volume content output
- Color grading expertise
Tools & Technologies
- Adobe Premiere Pro / Final Cut Pro / DaVinci Resolve
- Adobe After Effects (motion graphics)
- Cloud storage and version control systems
- Audio editing tools (Audition)
KPIs / Success Metrics
- Number of edits delivered vs. target
- Average turnaround time per video
- Watch-through/completion rate of published edits
- Content repurposing rate (long-form to short-form)
Benefits & Perks
- Competitive compensation with performance-linked growth
- High-ownership role with direct access to leadership
- Fast-tracked career growth in a high-growth environment
- Learning and development support
- A culture that rewards speed, ownership and craft
Job Title : Data Engineer – GCP + Spark + DBT
Location : Bengaluru (On-site at Client Location | 3 Days WFO)
Experience : 8 to 12 Years
Level : Associate Architect
Type : Full-time
Job Overview :
We are looking for a seasoned Data Engineer to join the Data Platform Engineering team supporting a Unified Data Platform (UDP). This role requires hands-on expertise in DBT, GCP, BigQuery, and PySpark, with a solid foundation in CI/CD, data pipeline optimization, and agile delivery.
Mandatory Skills : GCP, DBT, Google Dataform, BigQuery, PySpark/Spark SQL, Advanced SQL, CI/CD, Git, Agile Methodologies.
Key Responsibilities :
- Design, build, and optimize scalable data pipelines using BigQuery, DBT, and PySpark.
- Leverage GCP-native services like Cloud Storage, Pub/Sub, Dataproc, Cloud Functions, and Composer for ETL/ELT workflows.
- Implement and maintain CI/CD for data engineering projects with Git-based version control.
- Collaborate with cross-functional teams including Infra, Security, and DataOps for reliable, secure, and high-quality data delivery.
- Lead code reviews, mentor junior engineers, and enforce best practices in data engineering.
- Participate in Agile sprints, backlog grooming, and Jira-based project tracking.
Must-Have Skills :
- Strong experience with DBT, Google Dataform, and BigQuery
- Hands-on expertise with PySpark/Spark SQL
- Proficient in GCP for data engineering workflows
- Solid knowledge of SQL optimization, Git, and CI/CD pipelines
- Agile team experience and strong problem-solving abilities
Nice-to-Have Skills :
- Familiarity with Databricks, Delta Lake, or Kafka
- Exposure to data observability and quality frameworks (e.g., Great Expectations, Soda)
- Knowledge of MDM patterns, Terraform, or IaC is a plus
-
Key Skills / Keywords
E-commerce website, E learning Application,
Good in creating:
Functional specification document
Use Cases
Data flow diagrams
SalarySalary is not a constrain
Job Description / Responsibilities
Understanding requirements and creation of scope of work document
Efficient in work ethics and discipline
-
Validate project viability
-
Discuss the brief with the client and define SMART objectives
-
Ask for requirements, budget, deadline & milestones
-
Discuss the project with Delivery Leads
-
Provide a ballpark estimate & timeline
-
Share and discuss with the client
-
Get Client approval on ballpark budget, timeline & requirements
-
Create a project tracker spreadsheet
-
Timelines
-
Milestones and Deliverables
-
Create an activity tracker for the project
-
Setup meeting with team members (Design, Development, Testing and Support)
-
Project Understanding to team
-
User Journey
-
Functionality
-
Verify requirements and get more clarity
-
Risk Identification
-
Review
-
Client Meeting after meeting with the team to validate requirements and get more clarity
-
Update document
-
Change request and approval or parking of the new requirement
-
Risk mitigation strategy
-
Create a project communication plan
-
Identify team members and send weekly update meetings to client
-
Constant communication with the client for clarity on the requirements or features
-
Create weekly status tracker
-
Daily Meeting internal
-
Activity updates
-
Weekly Meeting with client
-
Take regular Project Updates
Qualifications and Preferred Skills
Good communication skills
Good presentation skills
Must Have
FSD Creation, Discipline, Ecommerce experience
Joining
One month
-
We are looking to hire a business development executive with 1-3 years of IT experience as well as good client-handling skills.
Experience: 1-3 Years (Relevant Experience Only)
Work Mode: WFO
Location: Seoni, Madhya Pradesh
Priority: Immediate /Maximum 15-Day Notice Period Acceptable
Budget: 15,000–25,000 Monthly (Professional Tax & Insurance Deduction Exclusive)
Interview Rounds: 3 (HR, Technical, or Practical)
Working Days: 5 Days a Week (Monday - Friday) | 9:30 AM–6:30 PM
Leave Policy: 8 Paid Leaves *Terms Applied
Notice Period: 1 Month (Mandatory) - If Bypassed, No Relieving/Experience Letter Allotted
Provisional Period: 90 Days
No salary will be provided if the candidate leaves the company within 15 days
Commitment Period (Bond): Provisional Period + 365 Days (1 Year)
Responsibility:
- Handling Cold calls and freelancing platforms.
- Generating the leads, Negotiation with clients Convert leads into clients.
- Understand client requirements and shape the project structure.
- Good knowledge about how to create SOW, build data architecture, and other necessary documents.
- strategies to identify and engage potential clients through various channels.
- Conduct research to identify new leads and opportunities within target industries.
- Gather a deep understanding of prospective clients' needs, pain points, and goals.
- Present our products/services and make plans for their updating with time.
- Handling Dispute and Finding solution.
- Maintain regular follow-ups to keep prospects engaged throughout the sales cycle.
- Collaborate with Development teams and give information about client needs.
- Do Price negotiations, discussing terms, pricing, and contractual agreements with clients.
- Prepare and present regular reports on sales activities, lead generation, and performance, and share them with the MD.
- Good knowledge of market standards and trends.
- Knowledge about every type of product and service.
Qualifications:
- Bachelor's degree in Business Administration, Marketing, or a relevant field.
- Handson is experienced in IT, with at least 2 years of business development or sales roles.
- Exceptional communication, negotiation, and convention skills
- Ability to build and maintain client relations
- Knowledge about how to present a demo
- Focus on achieving and exceeding targets.
- Self-motivated, proactive, and capable of working independently.
Pre-Requirements:
Must read the complete job description and acknowledge self-fit for this position.
- Good internet connection/ WiFi.
- Diary and a pen to solve some questions.
- Working earphones and camera.
- Must Join with your own Laptop/Computer.
- Your previous work.
- Must Visit our website Defenzelite.Com to know about the company.
WFH Note:
We do not provide laptops. The candidate has to arrange before joining.
The candidate has to make sure they have good internet connectivity. (Call such expenses covered by the candidates themselves.)
We encourage you to apply if you think you are a good fit, can handle the Business Development Executive environment, and have a proven ability to generate leads and convert them into clients. Join us in driving our business to new heights and significantly impacting our industry.
Position description:
As a Sourcer, you will be responsible for sourcing active candidates from on-line databases, contact lists, and internal databases as well as sourcing passive candidates through networking, cold calling, complex internet searches, and research. This is a great opportunity for anyone in Staffing to become a part of a truly global sourcing effort.
Responsibilities:
1. Proactively conduct research and investigate new ideas to create innovative sourcing strategies. Strong focus on sourcing passive candidates through networking, cold calling, complex internet searches, and research.
2. Source active candidates from on-line databases, contact lists, internal databases, and employee referrals.
3. Act as a search expert for a functional area(s) by understanding functional hiring needs, position specifications, and search requirements.
4. Partner with recruiters to generate ideas, leverage resources, and share information that facilitates an effective search process.
5. BA/BS degree preferred or equivalent experience
6. 3-5 years of recruitment experience in an agency
7. Be a market intel expert for the respective skill sets you hire for and drive insightful discussions around talent availability, forecasting, competitor landscape with recruiters and clients
What is the role?
You will be responsible for architecting and developing features on the application end to end. You will work closely with the Product Managers and the Technical architect to develop high-quality, resilient, and highly performant applications.
Key Responsibilities
- Design and develop SaaS-based applications based on microservices architecture.
- Work on the core backend and ensure it meets the performance benchmarks.
- Work on the front end with ReactJS.
- Designing and developing APIs for the front end to consume.
- Constantly improve the architecture of the application by clearing the technical backlog.
- Meeting both technical and consumer needs.
- Staying abreast of developments in web applications and programming languages.
What are we looking for?
An enthusiastic individual with the following skills. Please do not hesitate to apply if you do not match all of them. We are open to promising candidates who are passionate about their work and are team players.
- Education - BE/MCA or equivalent.
- Overall 4+ years of experience.
- Agnostic/Polyglot with multiple tech stacks.
- Worked on open source technologies – NodeJS, ReactJS, MySQL, NoSQL, MongoDB, DynamoDB.
- Good experience with Front-end technologies like JavaScript frameworks, ReactJS, and AngularJS.
- Backend exposure – good knowledge of building API.
- Worked on serverless technologies.
- Efficient in building microservices in combining server & front-end.
- Knowledge of cloud architecture.
- Should have sound working experience with relational and columnar DB.
- Should be innovative and communicative in approach.
- Will be responsible for functional/technical track of a project.
Who will you work with?
You will closely work with the engineering team and support the Product Team of Plum.
What can you look for?
A wholesome opportunity in a fast-paced environment that will enable you to juggle between concepts, yet maintain the quality of content, interact, and share your ideas and have loads of learning while at work. Work with a team of highly talented young professionals and enjoy the benefits of being at this company
We are
It is a rapidly growing fintech SaaS firm that propels business growth while focusing on human motivation. Backed by Giift and Apis Partners Growth Fund II, Company offers a suite of three products - Plum, Empuls, and Compass. works with more than 2000 clients across 10+ countries and over 2.5 million users. Headquartered in Bengaluru, It is a 300+ strong team with four global offices in San Francisco, Dublin, Singapore, New Delhi.
Way forward
We look forward to connecting with you. As you may take time to review this opportunity, we will wait for a reasonable time of around 3-5 days before we screen the collected applications and start lining up job discussions with the hiring manager. We however assure you that we will attempt to maintain a reasonable time window for successfully closing this requirement. The candidates will be kept informed and updated on the feedback and application status.
- Developing new user-facing features using React.js.
- Build reusable components and front-end libraries for future use.
- Translate designs and wireframes into high-quality code.
- Optimize components for maximum performance across a vast array of web-capable devices and browsers.
- Design a modern highly responsive web-based user interface.
Skills and Competencies;
- Bachelor's degree in computer science preferred.
- 3-5 years of demonstrated experience.
- Good knowledge of HTML5, CSS3.
- Strong proficiency in Javascript.
- Strong working knowledge of React JS and web apps.
- Good working experience using API in UI development.
- Should be able to do responsive design and cross-browser compatibility.
- Willingness to experiment and take complete responsibility for projects.
- Work independently and remote, with excellent follow-up, time management, and organization skills.
- Any additional knowledge of/familiarity with programming, in general, would be a great added bonus.
- Knowledge of Jira/confluence.
- Familiarity with code versioning tools such as Git.
- A team player with a problem-solving aptitude.
- Fluent in a second language preferred, good level of verbal English required.










