Cutshort logo
For Employers
Thomasmount Consulting logo
Data Annotation

Data Annotation at Thomasmount Consulting · Kochi (Cochin) · 5 - 9 years · ₹20L - ₹22L / yr · Profitable · Posted 14 Oct 2026

Thomasmount Consulting's logo

Data Annotation

Shirin Shahana's profile picture
Posted by Shirin Shahana
5 - 9 yrs
₹20L - ₹22L / yr
Kochi (Cochin)
Skills
Data annotation
data labelling
Artificial Intelligence (AI)

Role Overview

We are looking for an experienced professional to evaluate AI-generated responses for conversational AI agents used in travel-related products. The role focuses on assessing response accuracy, relevance, completeness, and quality to improve AI performance.

Key Responsibilities

  • Evaluate AI-generated responses against predefined guidelines and quality standards.
  • Perform data annotation, labeling, classification, and quality reviews.
  • Identify incorrect, misleading, irrelevant, or incomplete responses and potential AI hallucinations.
  • Analyze evaluation data to identify recurring errors, quality gaps, and improvement opportunities.
  • Track quality metrics, document findings, and recommend improvements.
  • Use Excel/Google Sheets and basic SQL, where applicable, to analyze evaluation data.
  • Collaborate with AI and Machine Learning teams to improve conversational AI performance.

Required Skills

  • 5–8 years of experience in Data Annotation, AI Evaluation, Content Quality, Search Relevance, or Quality Assurance.
  • Strong analytical, problem-solving, and attention-to-detail skills.
  • Experience applying evaluation guidelines and quality standards consistently.
  • Proficiency in Excel or Google Sheets.
  • Good written communication and documentation skills.
  • Understanding of quality metrics and data analysis.

Good to Have

  • Exposure to Generative AI, LLMs, conversational AI, or AI response evaluation.
  • Experience with Human-in-the-Loop (HITL), AI training data validation, or hallucination detection.
  • Basic SQL knowledge.
  • Experience in travel, hospitality, or search quality evaluation.


Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

About Thomasmount Consulting

Founded :
2008
Type :
Products & Services
Size
Stage :
Profitable

About

ThomasMount is an Information Technology (IT) and Resource management company providing consulting solutions and highly skilled IT consultants to organizations involved in in-house or outsourced IT engagements. We are organized into three primary business divisions. Staffing Services is our flag ship division, offering IT staff augmentation to our clients. Our Solutions division offer clients one-stop answers to IT requirements, Project teams, Outsourcing capacity, Custom software development and Application hosting and infrastructure alternatives. Our Human Resource services division offers services in the areas of Recruitment Strategy Development The company is a led by value driven leaders who focus on delivering value to all the stake holders- Customers, Investors, Employees and the society. ThomasMount has a rapidly increasing workforce of dedicated cutting edge professionals who constantly upgrade themselves on technical, managerial and behavioral skills.
Read more

Company social profiles

linkedin

Similar jobs (10)

Neosapien
Neosapien
Agency job
via Recro by Nehlata Pandey
Bengaluru (Bangalore)
3 - 7 yrs
₹15L - ₹40L / yr (ESOP available)
skill iconPython
Large Language Models (LLM) tuning
Agentic AI
Retrieval Augmented Generation (RAG)
skill iconMachine Learning (ML)
+2 more

The Role

You own AI systems end to end. From the speech-to-text models that turn audio into text, to the diarization that separates and identifies speakers, to the agentic layer that turns conversation into memory and action, to the observability and evaluation that keep all of it honest in production. This is a wide role by design. You will own model selection, serving, and production reliability. If you want to tune one model and ignore the system around it, this is not the role.

What You Will Own

•     Speech-to-text. Evaluate, integrate, and optimize STT models across cloud and self-hosted. Drive accuracy and cost trade-offs with ground-truth metrics.

•     Speaker diarization and identification. Push accuracy on hard, real-world, multi-speaker audio.

•     Agentic AI. Build the memory and retrieval pipeline, LLM orchestration, and the agent workflows that sit on top of captured conversation.

•     Model serving and infrastructure. Stand up and optimize self-hosted serving (vLLM, Triton class). Own latency, throughput, and cost per user.

Observability

An always-on wearable means models run in production every second, on messy real-world audio. You own the visibility into that.

•     Instrument the full audio-to-memory pipeline: STT, diarization, retrieval, and LLM calls.

•     Define and track model-quality SLOs in production: transcription drift, diarization error over time, retrieval relevance, latency, throughput, and cost per user.

•     Build dashboards and alerting so model degradation is caught before users feel it.

•     Trace failures across a distributed, always-on system using metrics, logs, and traces.

•     Close the loop. Production signals feed back into evaluation and model selection.

Evaluation

We do not ship what we cannot measure. You own the systems that prove a model is actually better, not just newer.

•     Build and own ground-truth evaluation harnesses for every model in the stack.

•     Measure with real metrics: WER for transcription, DER for diarization, Recall and F1 for retrieval and speaker identification.

•     Build and maintain labeled benchmark datasets that reflect real, messy, multi-speaker audio.

•     Run regression and A/B evaluations on every model swap, prompt change, or pipeline update. Nothing ships on a vibe.

•     Reject anecdotal proxies, single confidence scores, and cherry-picked examples as evidence of quality.

What We Are Looking For

•     3 to 5 years as an AI/ML engineer with production systems behind you. Engineering and production experience is non-negotiable.

•     Depth across the modern AI stack: LLMs, speech models, vector retrieval, model serving.

•     Strong software engineering. You write code that ships and survives contact with real users.

•     Fluency in Python and the production ML ecosystem.

•     Comfort with cloud infrastructure (GCP a plus) and containerized deployment on Kubernetes.

•     A working command of observability and evaluation. You measure first and trust metrics over intuition.

•     First-principles reasoning and metric discipline.

Nice to Have

•     Research background or publications. A strong signal, not a substitute for production work.

•     Audio and speech ML experience (STT, diarization, voice).

•     Experience self-hosting and optimizing open models.

•     Experience with LLM gateway and agent orchestration patterns.

•     Experience building eval harnesses or production model-monitoring systems.


Requirements

Agentic work is must. Audio is good to have

. Self hosting models is a must

 Experience with LLM gateway and agent orchestration is a must have

Read more
Leadsquared
Leadsquared
Agency job
via Right Hire by Vrishali Mishra
icon

The recruiter has not been active on this job recently. You may apply but please expect a delayed response.

Bengaluru (Bangalore)
2 - 4 yrs
₹15L - ₹35L / yr
Test Automation (QA)
API
skill iconPython
skill iconJavascript
Regression Testing
+2 more

About Us

We’re building the next generation of AI-powered business software, and we’re looking for people who want to shape that future with us. With Lumen, we’re reimagining how users interact with CRM — moving beyond screens, menus and dashboards to an intelligent interface where users can simply ask AI to take actions, retrieve knowledge, generate insights and get work done. With Agent Studio, we’re enabling businesses to build, test and deploy their own AI agents for real-world workflows. And with Invorto, we’re bringing AI to voice, allowing businesses to create intelligent voice agents tailored to their customer and operational use cases.

What makes this especially exciting is the stage and scale of the opportunity. You’ll get to work on genuinely hard problems across LLMs, agents, reasoning, orchestration, voice AI, evaluation, reliability and enterprise security — not as isolated experiments, but as products used in real business workflows. You’ll have the opportunity to build zero-to-one, own meaningful parts of the product end-to-end, work closely with customers, experiment rapidly, and see your work reach production at scale.

Why join now? Because the playbook for enterprise AI is still being written. You won’t just be implementing someone else’s roadmap — you’ll help define the product, architecture and experiences that become that playbook. Expect high ownership, fast iteration, hard technical and product problems, direct customer impact, and the chance to build AI systems that have to work reliably in the real world — not just in a demo.

About the Role

We are looking for a QA Engineer who specializes in testing agentic AI platforms. You will design and automate quality processes for systems that involve LLMs, autonomous agents, tool use and orchestration across Lumen and Agent Studio — ensuring that AI-driven workflows behave reliably, safely and predictably in production, not just in a demo.

What You’ll Do

  • Design and build automated test suites and evaluation frameworks for agentic AI workflows, including multi-step and tool-calling behaviors.
  • Use AI/LLM-based QA tools and evaluation frameworks to test model outputs, agent decisions and end-to-end task completion at scale.
  • Define quality metrics and benchmarks for agent reliability, correctness, latency and safety, and track them over releases.
  • Identify edge cases, failure modes and regressions specific to non-deterministic AI systems, and build automated checks to catch them early.
  • Integrate automated agent/LLM testing into CI/CD pipelines to support fast, reliable iteration.
  • Partner closely with AI/ML and backend engineers to reproduce issues, root-cause failures and validate fixes.
  • Work with customers and customer-facing teams to understand real-world usage patterns and translate them into test scenarios.

What We’re Looking For

  • 2–4 years of QA/test automation experience, including hands-on work testing agentic AI or LLM-based platforms.
  • Practical experience using AI-focused QA/evaluation tools to test agent behavior, prompts and model outputs.
  • Strong scripting/automation skills (Python preferred) to build and maintain test frameworks.
  • Understanding of how LLM-based agents work — tool calling, orchestration, memory, reasoning chains — well enough to design meaningful test cases.
  • Comfort working with non-deterministic systems and designing evaluation approaches beyond traditional pass/fail testing.
  • Strong communication skills — this is a customer-facing role, and you will be expected to clearly articulate technical concepts, decisions and trade-offs to both technical and non-technical stakeholders, including customers.

Good to Have

  • Experience testing voice AI or real-time conversational systems.
  • Familiarity with CRM or enterprise SaaS platforms.
  • Exposure to enterprise security or compliance testing for AI systems.


Read more
LeadSquared
at LeadSquared
8 recruiters
Agency job
via Right Hire by Vrishali Mishra
Bengaluru (Bangalore)
4 - 6 yrs
₹20L - ₹40L / yr
skill iconPython
Speech-to-Text (STT)
ASR
Text-to-Speech (TTS)
Large Language Models (LLM)
+3 more

About Us

Invorto is our Voice AI product, bringing intelligent voice agents to real-world customer and operational use cases. Our voice pipeline is built in Python, running an STT → LLM → TTS architecture on top of the Pipecat framework.

This is a chance to work on hard problems in voice AI — latency, accuracy, naturalness, and reliability — building zero-to-one, owning your area end-to-end, and shipping to production at scale.

Note: This is a customer-facing role, and strong communication skills are essential.

About the Role

We're looking for a Voice AI Research Engineer to join the Invorto team and help build and continuously improve the voice AI systems that power our intelligent voice agents. This role is focused on the specialized craft of voice AI — designing evaluation and automation frameworks that ensure our STT, LLM, and TTS pipeline performs reliably in real-world, production conditions.

 

What You'll Do

  • Design and build automated testing and quality frameworks for our STT → LLM → TTS voice pipeline, built on Pipecat
  • Evaluate and benchmark STT, LLM, and TTS/ASR components on accuracy, latency, naturalness, and robustness across accents, languages, and real-world audio conditions
  • Work hands-on with STT, TTS, and ASR models — fine-tuning, evaluating, and improving them for production use cases
  • Identify failure modes and edge cases across the pipeline (background noise, accents, interruptions, turn-taking, latency, pipeline-stage handoffs) and build systems to catch them before production
  • Collaborate closely with engineering to integrate quality checks and automation into the voice agent development lifecycle within the Pipecat-based architecture
  • Research and stay current with advances in voice AI, and bring in new techniques, models, and tools to improve pipeline performance
  • Work directly with customers to understand real-world voice use cases and translate them into evaluation criteria and quality benchmarks
  • Partner with product and engineering to define what "production-grade quality" means for voice agents and drive the team toward it

 

What We're Looking For

  • 4–6 years of experience, with a specialization in voice AI systems and automated quality evaluation
  • Hands-on experience with STT (Speech-to-Text), TTS (Text-to-Speech), and ASR (Automatic Speech Recognition) models
  • Experience designing and building automated testing/evaluation frameworks for voice or speech systems
  • Strong understanding of what drives voice AI quality — accuracy, latency, naturalness, and robustness to real-world variability
  • Strong programming skills in Python; familiarity with Pipecat or similar voice pipeline/orchestration frameworks is a plus
  • Understanding of STT → LLM → TTS pipeline architectures and the trade-offs involved at each stage
  • Research mindset — comfortable exploring new models, techniques, and tools and translating them into practical improvements
  • Excellent communication skills — this is a customer-facing role, and you'll regularly engage directly with customers to understand needs and validate quality expectations


Read more
LetsIntern
at LetsIntern
1 candid answer
Ashish Singh
Posted by Ashish Singh
Remote only
0 - 1 yrs
₹1L - ₹2L / yr
Artificial Intelligence (AI)
skill iconPython

About the Job :


We are looking for a passionate and driven AI Intern to join our dynamic team. As an intern, you will have the opportunity to work on real-world projects, develop AI models, and collaborate with experienced professionals in the field. This internship is designed to provide hands-on experience in AI and machine learning, offering you the chance to contribute to impactful projects while enhancing your skills.


Job Description:

We are seeking a talented Artificial Intelligence Specialist to join our dynamic team. As an AI Specialist, you will be responsible for developing, implementing, and optimizing AI models and algorithms. You will collaborate closely with cross-functional teams to integrate AI capabilities into our products and services. The ideal candidate should have a strong background in machine learning, deep learning, and natural language processing, with a passion for applying AI to real-world problems.


Responsibilities:


  • Design, develop, and deploy AI models and algorithms.
  • Conduct data analysis and pre-processing to prepare data for modeling.
  • Implement and optimize machine learning algorithms.
  • Collaborate with software engineers to integrate AI models into production systems.
  • Evaluate and improve the performance of existing AI models.
  • Stay updated with the latest advancements in AI research and apply them to enhance our products.
  • Provide technical guidance and mentorship to junior team members.



Requirements:


  • Any Graduate / Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field; Master's degree preferred.
  • Proven experience in developing and implementing machine learning models and algorithms.
  • Strong programming skills in languages such as Python, R, or Java.


Benefits :


  • Internship Certificate
  • Letter of Recommendation
  • Performance-Based Stipend
  • Part-time work from home (2-3 hours per day)
  • 5 days a week, fully flexible shift


Read more
NA
NA
Agency job
via NAM Info Pvt Ltd by Ramya Munirathnam
Hyderabad, Pune
10 - 14 yrs
₹6L - ₹14L / yr
Generative AI
skill iconC#
skill icon.NET
AI Agents

AI Engineer


We are seeking an AI Engineering specialist focused on AI evaluation, and continuous quality improvement for Ezra MetLife's employee-facing AI platform. This role will establish and scale the testing strategy for enterprise AI agents, ensuring high response quality, reliability, and production readiness. The engineer will build automated regression testing framework (preferred Playwright ), define AI evaluation methodologies, analyze AI performance metrics, and partner with engineering teams to continuously improve answer quality, grounding accuracy, and customer experience. This position is critical to enabling confidence as Ezra expands its AI agent portfolio and employee-facing capabilities.


Required Skills & Experience


• C# and .NET development experience


• Experience with at least one AI evaluation framework (e.g., prompt evaluation, RAG evaluation, LLM quality assessment)


• Microsoft Agent Framework (preferred) or similar enterprise agent frameworks


• Experience with Azure OpenAI / Azure AI Foundry


• Microsoft 365 Agent SDK


• Azure AI Search, RAG pipelines, and retrieval quality testing


• Infrastructure as Code using Terraform


• Experience building automated testing and AI quality validation processes


• Familiarity with telemetry analysis, AI observability, and performance measurement


• Strong analytical skills with a passion for improving AI response quality and reliability


Skills: AI Agents~Core .NET Technologies~C# 5.0

Experience Required: 10 & Above

Location: Hyderabad :5+ relevant exp in AI + .NET

Read more
Deqode
at Deqode
1 recruiter
Apoorva Jain
Posted by Apoorva Jain
Bengaluru (Bangalore)
4 - 9 yrs
₹12L - ₹22L / yr
Software Testing (QA)
Automation
skill iconPython
Large Language Models (LLM)
Retrieval Augmented Generation (RAG)
+1 more

Required Skills:-

  • 4+ years of experience in Software Testing (Manual & Automation).
  • Strong experience with Selenium or Playwright.
  • Hands-on experience in API Testing (Postman, REST Assured, Swagger, etc.).
  • Experience with CI/CD pipelines (Jenkins, GitLab CI, Azure DevOps, GitHub Actions).
  • Strong understanding of Functional, Regression, and Non-functional Testing.
  • Excellent analytical and debugging skills.

AI-Specific Skills

  • Hands-on experience in GenAI / AI Testing.
  • Experience testing LLM-based applications.
  • Strong understanding of prompt engineering and prompt validation.
  • Knowledge of LLM behavior, model variability, and non-deterministic outputs.
  • Experience validating AI outputs for accuracy, hallucinations, bias, and safety.
  • Test data management for AI applications.
  • Understanding of Responsible AI testing concepts.


Read more
QuantumProp AI
at QuantumProp AI
1 candid answer
Athrava Shukla
Posted by Athrava Shukla
Remote only
0 - 0 yrs
₹1L - ₹4L / yr
Analytical Skills
Technical analysis

Role Overview:


We are looking for AI Market Evaluators to join our growing team. In this role, you will review daily market indicators prepared by our AI systems and provide simple directional input ("Up" or "Down"). No trading or financial experience is required—we provide all the training and support you need.


Key Responsibilities:

  • Review daily market charts and indicators sent to your inbox
  • Provide directional input ("Up" or "Down") to help refine our AI trading models
  • Maintain consistent daily participation (5-10 minutes)
  • Communicate with our support team if any issues arise


Read more
Remote only
0 - 3 yrs
₹0 / mo
Microsoft Excel
skill iconData Analytics
Problem solving

As an intern for the Artificial Intelligence business automation team, you will be focusing on researching and developing AI tools that automate some parts of our business automation. As business automation is important throughout the firm, you will have the opportunity to collaborate with teams across Springer. 

Read more
The supreme consultancy
Pramila Ranjane
Posted by Pramila Ranjane
Pune
1 - 4 yrs
₹12L - ₹25L / yr
GEO
AEO
SGE initiatives
Generative AI
Fine-tuning LLMs
+5 more

🔹 Key Responsibilities


• Design, develop, and deploy production-grade AI/ML and Generative AI solutions

• Work on GEO, AEO, and SGE initiatives to improve visibility across AI-driven search platforms

• Optimize content and digital experiences for conversational queries and LLM-based search

• Develop solutions using LLMs, NLP, embeddings, semantic search, RAG, and vector databases

• Analyze search intent, AI-generated responses, citations, retrieval patterns, and content discoverability

• Build frameworks to measure GEO/AEO strategies and AI-search performance

• Collaborate with Product, Engineering, Content, SEO, Marketing, and Business teams

• Improve solution accuracy, relevance, latency, and user experience


🔹 Mandatory Requirements


✅ 1–4 years of professional experience

✅ Minimum 1 year of hands-on experience in GEO, AEO, or SGE

✅ Experience with prompt engineering, embeddings, vector search, or RAG systems

✅ Understanding of semantic search and entity-based optimization

✅ Exposure to ChatGPT, Google Gemini, or similar LLM platforms

✅ Knowledge of schema, context building, content structuring, and knowledge representation


🎓 Preferred Education


B.Tech, M.Tech, Integrated M.Sc., or MS from a Tier-1 engineering institute such as IIT, NIT, BITS, VIT, DTU, or NSUT.

Read more
E2M Solutions Pvt. Ltd.
Zankhan Kukadiya
Posted by Zankhan Kukadiya
Ahmedabad
2 - 6 yrs
₹10L - ₹15L / yr
skill iconMachine Learning (ML)
skill iconDeep Learning
Artificial Intelligence (AI)
Neural networks
Artificial Neural Network (ANN)
+6 more

About E2M:

E2M Solutions works as a trusted white-label partner for digital agencies. We support agencies with consistent and reliable delivery through services such as website design, web development, eCommerce, SEO, AI SEO, PPC, AI automation, and content writing .Founded on strong business ethics, we are an equal opportunity organization powered by 300+ experienced professionals, partnering with 400+ digital agencies across the US, UK, Canada, Europe, and Australia. At E2M, we value ownership, consistency, and people who are committed to doing meaningful work and growing together .If you’re someone who dreams big and has the gumption to make them come true, E2M has a place for you.


Role Overview:

We are seeking a highly skilled and client-centric AI Consultant/AI Adoption Specialist to join our growing team. In this pivotal role, you'll serve as a vital link between our clients' strategic objectives and the transformative power of AI. You'll primarily focus on understanding their needs, scoping opportunities, and architecting actionable AI roadmaps.

 

Key Responsibilities:

  • Collaborate closely with clients to understand their challenges and identify opportunities to apply AI.
  • Assess client requirements and prepare solution strategies using AI tools and methodologies.
  • Work with internal teams to design, propose, and help execute AI-powered solutions.
  • Provide AI-based recommendations that align with the client’s business objectives.
  • Communicate technical possibilities in a business-friendly manner to decision-makers.
  • Take ownership of the client journey from discovery to implementation and support.
  • Stay updated with AI trends, tools, and real-world use cases that can benefit clients.

 

Required Skills & Qualifications:

  • Minimum 2+ Years of hands on experience into Custom AI Development.
  • Minimum 3+ years of experience in roles like Project Manager, Customer Success Manager, or Account Manager, preferably in a service-based company or digital agency.
  • Strong understanding of AI concepts, trends, and tools (e.g., NLP, ML, Chatbots, Automation, native cloud technologies).
  •  Some hands-on experience in AI projects – either through execution, coordination, or implementation.
  • Ability to manage multiple client engagements and communicate effectively with both technical and non-technical stakeholders.
  •  Strong problem-solving mind set with the ability to translate business needs into AI opportunities.
  • Flexible to work with international clients, especially in the US time zone as needed.


Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos