
AI System QA Engineer – Large Language Models (Evaluation Testing)
at MeltPlan
MeltPlan is building the “planning engine” for the $14 Tn construction industry, an AI system designed specifically to optimize decisions before construction begins. While design software optimizes use and aesthetics and construction software optimizes execution and control, MeltPlan is building the missing layer - software that optimizes decisions and tradeoffs upstream, before scope is locked, procurement begins, and change orders become inevitable. MeltPlan’s long-term goal is to help teams make construction “boring” by making planning more intense: surfacing constraints and tradeoffs early, aligning stakeholders before plans are frozen, and reducing the need for late-stage redlines, rework, and change orders.
MeltPlan is founded by operators who have built at scale. Kanav previously co-founded Innovaccer, a $3Bn healthtech company focused on making US healthcare more affordable and accessible. He’s now applying that systems-level thinking to construction.He’s joined by Tanmaya Kala, former Project Executive at DPR Construction, who led large commercial, healthcare, and life sciences projects. We combine deep tech scale with real construction execution.
What This Role Really is :
We are seeking a detail-oriented and technically strong AI QA Engineer to ensure the quality, reliability, and performance of Large Language Model (LLM)-based systems. In this role, you will be responsible for designing and executing test strategies, validating model outputs, and building evaluation frameworks to enhance the accuracy, safety, and overall performance of AI-driven applications.We would particularly value candidates who have hands-on experience in developing evaluation frameworks (evals) for AI systems, along with strong expertise in comprehensive system testing and quality assurance practices.You are responsible for making MeltPlan work in the real world.
What You'll Do:
- Design, develop, and execute evaluation frameworks (Evals) for Large Language Models (LLMs) and AI systems.
- Perform end-to-end system testing, regression testing, and performance testing for AI-driven applications.
- Validate model outputs for accuracy, consistency, safety, hallucination detection, and edge cases.
- Build automated test pipelines and quality benchmarks for AI systems.
- Collaborate closely with AI/ML engineers, product teams, and platform engineers to improve system reliability.
- Analyze failures, identify root causes, and provide actionable feedback to improve model behavior.
- Develop datasets, prompts, and testing scenarios to measure model performance across multiple use cases.
- Monitor production performance and continuously improve evaluation metrics and testing standards.
- Ensure compliance with responsible AI and quality assurance best practices.
What We're looking for:
- Bachelor’s degree in Computer Science, Engineering, or related field
- 5–9 years of experience in QA/testing, preferably in AI/ML or data-driven systems
- Strong experience in AI/LLM evaluation frameworks and system testing.
- Hands-on experience with automated testing methodologies and QA processes.
- Familiarity with prompt engineering, AI benchmarking, and model validation techniques.
- Experience working with Python and testing frameworks.
- Understanding of LLM behaviors, hallucinations, prompt injection risks, and AI safety concepts.
- Exposure to tools/frameworks such as OpenAI Evals, LangSmith, DeepEval, Promptfoo, or similar platforms is preferred.
- Strong analytical and debugging skills with attention to detail.
- Excellent collaboration and communication skills.
- Familiarity with Large Language Models and Generative AI concepts
- Experience with API testing tools (e.g., Postman) and automation frameworks
- Understanding of NLP concepts such as tokenization, embeddings, and text generation
- Strong analytical and problem-solving skills
- Experience testing AI/ML models or data pipelines
- Experience with prompt engineering and prompt testing
- Familiarity with cloud platforms (AWS, GCP, or Azure)
- Exposure to AI safety, bias detection, and model governance
Bonus if you have:
- Have worked in construction or on project sites
- Have startup experience
- Experience working with Generative AI or conversational AI products.
- Knowledge of CI/CD pipelines and automation workflows.
- Prior experience in performance testing and monitoring distributed systems.
- Understanding of AI product lifecycle and production deployment environments.
We’re not looking for someone who waits for clean requirements.We’re looking for someone who thrives in the mess and turns it into systems.
Why meltplan
- Massive industry, real-world impact
- High ownership from day one
- Small team, zero bureaucracy
- Competitive comp + meaningful equity

About MeltPlan
About
Similar jobs
Notice Period - 0-15 days Max
Apply only who are currently in Karnataka
F2F interview
Interview - 4 rounds
Job Title: AI Specialist
Company Overview: We are the Technology Center of Excellence for Long Arc Capital
which provides growth capital to businesses with a sustainable competitive advantage and
a strong management team with whom we can partner to build a category leader. We focus
on North American and European companies where technology is transforming traditional
business models in the Financial Services, Business Services, Technology, Media and
Telecommunications sectors.
As part of our mission to leverage AI for business innovation, we are establishing AI COE to
develop Generative AI (GenAI) and Agentic AI solutions that enhance decision-making,
automation, and user experiences.
Job Overview: We are seeking dynamic and talented individuals to join our AI COE. This
team will focus on developing advanced AI models, integrating them into our cloud-based
platform, and delivering impactful solutions that drive efficiency, innovation, and customer
value.
Key Responsibilities:
• As a Full Stack AI Engineer, research, design, and develop AI solutions for text,
image, audio, and video generation
• Build and deploy Agentic AI systems for autonomous decision-making across
business outcomes and enhancing associate productivity.
• Work with domain experts to design and fine-tune AI solutions tailored to portfoliospecific challenges.
• Partner with data engineers across portfolio companies to –
o Preprocess large datasets and ensure high-quality input for training AI
models.
o Develop scalable and efficient AI pipelines using frameworks like
TensorFlow, PyTorch, and Hugging Face.
• Implement MLOps best practices for AI model deployment, versioning, and
monitoring using tools like MLflow and Kubernetes.
• Ensure AI solutions adhere to ethical standards, comply with regulations (e.g.,
GDPR, CCPA), and mitigate biases.
• Design intuitive and user-friendly interfaces for AI-driven applications, collaborating
with UX designers and frontend developers.
Internal Use Only
• Stay up to date with the latest AI research and tools and evaluate their applicability
to our business needs.
Key Qualifications:
Technical Expertise:
• Proficiency in full stack application development (specifically using Angular, React).
• Expertise in backend technologies (Django, Flask) and cloud platforms (AWS
SageMaker/Azure AI Studio).
• Proficiency in deep learning frameworks (TensorFlow, PyTorch, JAX).
• Proficiency with Large Language Models (LLMs) and generative AI tools (e.g., OpenAI
APIs, LangChain, Stable Diffusion).
• Solid understanding of data engineering workflows, including ETL processes and
distributed computing tools (Apache Spark, Kafka).
• Experience with data pipelines, big data processing, and database management
(SQL, NoSQL).
• Knowledge of containerization (Docker) and orchestration (Kubernetes) for scalable
AI deployment.
• Familiarity with CI/CD pipelines and automation tools (Terraform, Jenkins).
• Good understanding of AI ethics, bias mitigation, and compliance standards.
• Excellent problem-solving abilities and innovative thinking.
• Strong collaboration and communication skills, with the ability to work in crossfunctional teams.
• Proven ability to work in a fast-paced and dynamic environment.
Preferred Qualifications:
• Advanced studies in Artificial Intelligence, or a related field.
• Experience with reinforcement learning, multi-agent systems, or autonomous
decision-making
JD :
React. Js Developer
Skill Sets: HTML, CSS, JS , Typescript, Nextjs
Location : Bangalore , Complete WFO
Address:
Embassy Tech Village Rd, Devarabisanahalli, Bellandur, Bengaluru, Karnataka 560103, India.
Candidate Persona:
Need Product based Background Folks only or someone who has worked on product projects.
Resume is extremely crucial here, if roles and responsibilities, Education details and company names are not mentioned properly in utmost details then they will not be considered further.
Communication skills- Good
Btech graduated ONLY
Skills that should be mentioned in the CV - HTML, css, js, typescript, Data structures and algorithm.
Candidates should have a linkedin ID.
Candidates should have some leet code or hackerrank links (preferred)
Note : candidates who are ready for all the rounds can apply.
- Strong analytical and problem-solving skills
- Ability to work independently, learn quickly and be proactive
- 10-14 years of hands-on experience working on Web Full Stack technologies, with at least 4-6 years of experience developing applications with/on React/NextJS, NoSQL, REST APIs
- Proficiency in JavaScript/TypeDcript (ES6), NodeJS, HTML5, CSS3, CSS Preprocessors, Webpack, Gulp
- Client-side scripting and JavaScript frameworks – jQUery, ReactJS, Redux, Babel, JSX
- Experience in designing high-performance REST APIs and associated data structures
- Familiarity with developing microservices using containerization technologies such as docker, Kubernetes, etc.
- Working knowledge of git and using branches for development
As part of our Developer team you will get high quality experience in a young, high energy, friendly, fun and professional environment where people work together as a family. Our employees are constantly learning, upgrading their skills, developing their competences and finding ways to fully use their potential.
Aura Global Developers professionals work with innovative technologies, build world-class products and cooperate with global customers. Being part of Aura Global means working in an independent, well-organized and collaborative team.
SKILLS NEEDED - React Native, React JS, GraphQL, Postgres SQL
-3 to 7 years of experience is required.
-Should be able to handel a team.
-Designing and developing user interfaces using React JS best practices.
-Adapting interface for modern internet applications using the latest front-end technologies.
-Writing JavaScript, CSS, and HTML.
-Developing product analysis tasks.
-Developing application codes and unit tests in React Native and GraphQL Services
-Consulting with the design team.
-Ensuring high performance of applications and providing support.
-Ensuring optimal performance of the central database and responsiveness to front-end requests
-Collaborating with Back-end developers on the integration of elements
-Designing customer-facing UI
-Developing high-performance applications by writing testable, reusable, and efficient code
-Implementing effective security protocols, data protection measures, and storage solutions
-Running diagnostic tests, repairing defects, and providing technical support
-Technical Experience: React JS, React Native, GraphQL.
-Good Analytical skill to analyze Product Requirements, Agile Methodology and Process Knowledge of Banking Payment Gateway Domain is added advantage Actively participate in Agile Daily stand up and Jira
-Professional Attributes Able to work independently
-Educational Qualification: BE
Contact: SEVEN THREE FIVE EIGHT THREE THREE SEVEN FOUR NINE FIVE.
We are looking for a passionate, highly driven, intrinsically motivated Associate Product Manager who wants to join a high-growth startup, learn something new everyday and join of one the most energetic and speedy Product Teams in health-tech!
Responsibilities
- Building and executing new initiatives and roadmaps for retention and increasing customer lifetime value.
- Understand the healthcare market, customers and build business cases for new product opportunities.
- Listen to the users on a regular basis and figure out the opportunities to solve their problems.
- Manage product lifecycle from ideation to launch and beyond which would include liaising with multiple stake-holders.
- Work closely with partners and clients from 4 continents to localize the product builds as per their need and ensure adoption of the new features.
- Use data, creativity, and experimentation to constantly improve the product experience
Requirements
- Have 1-4 years of experience in building and managing large-scale enterprise products.
- Have prior experience in the development team and understand how modern development frameworks function.
- Are passionate about translating customer needs to usable design and process flows for growth levers and optimization.
- Have Good understanding of funnels, agile & sprints, and wireframes.
- Display the attitude to be comfortable with ambiguity and have the skill to transform broad ideas into action plans and display Empathy towards users and also your colleagues.
- Understand how the application functions and the technicalities around it.
- Have familiarity with tools like Google data studio, Firebase, Google Analytics, Product Analytics tools, and Figma.
- Love analytics and very frequent experimentation
Our client is looking for a Senior/Lead Full Stack .Net Developer for their headquarter office which is located in San Francisco, California. Looking for top-notch developers who are passionate and well-versed with modern technologies like .Net Core, MVC, SQL, and JS frameworks.
This position gives you an opportunity to lead exciting projects, work directly with teams in the US and with super smart people that you can learn from, and contribute to their knowledge.
Requirement:
- 7-15 years of experience in software development.
- Hands-on design and coding are required.
- Expertise in React.JS.
- Good knowledge of database concepts and Microsoft SQL Server.
- Highly proficient in modern JavaScript, HTML, CSS, ReactJs, and in one or more libraries for state management (e.g. Redux)
- Solid foundation in computer science, with strong competencies in data structures, design patterns, concurrency, algorithms, and software design
- Research and evaluate new software, frameworks, and techniques to provide recommendations to the division.
- Design and develop robust and scalable software components.
- Strong analytical and troubleshooting skills.
- Bachelor's degree in Computer Science or a technical field.
Perks & Benefits:
- Friendly, talented, collaborative, and entrepreneurial team
- Competitive and comprehensive benefits and perks
- Generous holiday and PTO policies
- Training and development opportunities and allowance
- Fun and inclusive digital, and (in the future) in-person events
- Employee groups - DEI committee, fun committee, wellness group, and more
- Flexible remote work
About company
The company is the industry-leading collection of online destinations (including Astrology.com, Horoscope.com, Keen, and PsychicCenter) providing spiritual guidance on love, relationships, career, health, and life overall. They are passionate about connecting people with the world's best advisors (including psychics, tarot card readers, life coaches, and more) and content to empower everyone to live happier lives.
A team of over 100 employees is powered by their diverse perspectives and company core values:
- They are humble. They believe the best result is achieved by leveraging others perspectives
- They think like owners. They make decisions that optimize for the greater good of the organization
- They challenge limiting beliefs. They are at their best when they identify and shatter the status quo expectations.
The company is an equal opportunity workplace and is an affirmative action employer. They are committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, or Veteran status.
Job Title – Sr. Database Developer and DBA MS SQL Server
Job Location – Bangalore
Experience – 8 to 10 years
CTC – 10-12 LPA
Job Description
- 8-10 years of industry experience with an appropriate University technology degree (BE,BTech,MCA)
- Should have extensive Hands-on experience in managing RDBMS Production and application DBA for MS-SQL Database.
- Must be proficient in MS-SQL programming and can handle complex logic.
- Knowledge of data structure and algorithms is a plus.
- Must have good experience in SSIS and SSRS using MSSQL Sever 2016 or later
- Monitoring, maintaining, Query tuning, performance tuning, cluster management for all RDBMS Production and application (MS-SQL).
- Database server administration with focus on security, automation, tuning, optimization, and standardization of new and deployed systems.
- Be able to work as a DBA lead and individual contributor for RDBMS platform.
- Will be responsible for database server planning with regards to scalability, redundancy, and data preservation of backend systems.
- Will develop High Availability Clustered server/network RDBMS topologies.
- Write comprehensive documentation, help develop tools to monitor systems performance, and work on optimization and tuning of various systems.
- Working with Production support teams on troubleshooting Incidents on a 24x7 basis. Looking for Sr. Sql Server DBA Who Would Provide Technical Support In Analysis, Design, Testing, And Deployment Of All Database Platform.
- Strong object oriented concepts. Experience in structure object oriented modelling with preferred expertise in using tools like Enterprise Architect or similar. Experience in working with TOGAF standards is preferred.
- Experience in architecting the technical scale and scope of high volume, scalable enterprise software solutions including logical and physical landscape requirements with specific attention to design, development, and deployment strategies
- Capability to adapt, learn and work with multiple technology platforms.
- Knowledge in Application Security including Information security principles & realization, OWASP & PCI DSS Compliance ( Security Design & Technology Skills )
- In depth knowledge and experience in large scale database management, data modelling and database design in RDBMS and NoSQL.
- Experience in recommending and implementing DevOps tools for enterprise projects.
- Capability to evaluate tools, technologies and processes, including assessing their strategic benefit in the solution.
- Willingness to work hands-on with engineers to review, troubleshoot coding problems quickly and efficiently.
- Expertise in following technologies – ASP.Net MVC, Web API, ASP.Net Core, Entity Framework, Entity Framework Core, ASP.Net Identity, REST
- Experience in implementing various application deployment models and monitoring the server infrastructure using industry standard tools.
- Experience in docker based deployment models.
- Experience in architecting, developing and deploying cloud based (One or more among AWS, Azure, Google Cloud) enterprise solutions.
- Experience in designing and developing micro-services based applications.
- Experience in designing and developing solutions with TDD (Test Driven Development)










