AI/ML Test Engineer at Improving · Mumbai · 2 - 4 years · ₹6L - ₹15L / yr · Raised funding · Posted 18 Sep 2026
PythonTitle: AI/ML Test Engineer – GenAI
Location - Mumbai
Technical Skills
• Strong experience in Generative AI, LLMs, and Agentic AI systems
• Hands-on expertise with AI evaluation frameworks (RAGAS, DeepEval, TruLens, LangSmith, Promptfoo, etc.)
• Proficiency in Python and AI/ML development libraries
• Knowledge of Prompt Engineering, prompt testing, and optimization
Ability to define and track evaluation metrics such as accuracy, relevance, groundedness, hallucination rate, latency, and user satisfaction
• Experience in creating automated evaluation pipelines and benchmarking frameworks
• Strong understanding of AI safety, guardrails, bias testing, and responsible AI practices
• Familiarity with REST APIs, JSON, vector databases, and knowledge retrieval systems
• Experience in A/B testing, human-in-the-loop evaluation, and red teaming
• Strong experience in Manual Testing of AI/GenAI applications, including functional, exploratory, UAT, regression, and end-to-end testing
• Expertise in validating Agent Reasoning, Tool Calling, Workflow Execution, and Response Quality
• Hands-on experience in Automation Testing using Python frameworks
Key Responsibilities
- Design, execute, and automate evaluation strategies for Agentic AI applications.
- Develop evaluation datasets, test cases, and benchmark suites.
- Measure and improve agent performance, reasoning quality, tool usage, and workflow effectiveness.
- Analyze model outputs and identify hallucinations, biases, safety risks, and failure patterns.
- Collaborate with AI Engineers, Product Teams, and Domain Experts to improve agent quality and reliability.
- Generate evaluation reports, dashboards, and actionable recommendations.
Shubham Vishwakarma
Full Stack Developer - Averlon
About Improving
About
Improving is a leading IT professional services firm committed to helping companies achieve lasting success through modern technology. With core expertise in AI, Data, and Applications, we specialize in transforming legacy systems, building cloud-native platforms, and delivering intelligent, future-ready solutions for today’s complex business needs. Improving’s leaders are equally committed to fostering a great place to work that is inclusive and purpose-centered, empowering Improvers to bring their whole selves to work. Our team is known for its collaborative approach and long-term partnerships that prioritize measurable outcomes. By combining technical excellence with strategic insight, Improving enables all stakeholders to grow, adapt, and lead in an ever-evolving digital landscape.
Tech stack
Go Programming (Golang)
Java
Kubernetes
DockerCompany social profiles
Similar jobs (10)
Express
NodeJS (Node.js)
MongoDB
React.js
React NativeWe’re hiring a hands-on Full-Stack Developer to join as our first technical hire and help turn our operational edge into scalable software. You’ll work directly with the founding team to:
* Build internal and guest-facing tools in React
* Develop a Node.js backend to power core workflows
* Integrate with property management software APIs (e.g., Guesty, Hostaway, Airbnb)
* Implement and extend functionality using AI APIs like OpenAI (required)
Our Stack
* Frontend: React
* Backend: Node.js (Express or similar)
* Database: MongoDB or Postgres
* APIs: REST/GraphQL, PMS integrations
* AI: OpenAI API and related services
Must-Have Skills
* 2+ years of full-stack development experience
* Strong proficiency in React and Node.js
* Demonstrated experience integrating external APIs (especially SaaS or PMS platforms)
* Familiarity working with OpenAI or other AI APIs
* Ability to work independently and ship quickly
Nice to Have
* Background in hospitality, rentals, or operational software
* Experience building dashboards or guest lifecycle tools
* Bonus: Prior experience with Guesty, Hostaway, or PMS APIs
.NETResponsibilities
- Manage and drive a team of Data Analysts and Sr. Data Analysts to provide logistics and supply chain solutions.
- Conduct meetings with Clients to gather the requirements and understand the scope.
- Conduct meetings with internal stake holders to walk them through the solution and handover the analysis.
- Define business problems, identify solutions, provide analysis and insights from the client's data.
- 5 Conduct scheduled progress reviews on all projects and interact with onsite team daily.
- Ensure solutions are delivered error free and submitted on time.
- Implement ETL processes using Pentaho Data Integration (Pentaho ETL)Design and implement data models in Hadoop.
- Provide end-user training and technical assistance to maximize utilization of tools.
- Deliver technical guidance to team, including hands-on development as necessary; oversee standards, change controls and documentation library for training and reuse.
Requirements
- Bachelor's degree in Engineering.
- 16+ years of experience in Supply Chain and logistics or related industry and Analytics experience.
- 3 years of experience in team handling(8+People) and interacting with the executive leadership teams.
- Strong project and time management skills with ability to multitask and prioritize workload.
- Solid expertise with MS Excel, SQL, any visualization tools like Tableau/Power BI, any ETL tools.
- Proficiency in Hadoop / Hive.
- Experience Pentaho ETL, Pentaho Visualization API, Tableau.
- Hands on experience of working with Big data sets (Data sets with millions of records).
- Strong technical and Management experience.
Desired Skills and Experience
- NET,ASP.NET
- Sales Consultants:
- Responsibilities:
- Interact with clients to understand their travel requirements and preferences.
- Present and promote our holiday packages and services effectively.
- Close sales and achieve monthly targets.
- Build and maintain strong client relationships to ensure repeat business.
- Requirements:
- Excellent communication and negotiation skills.
- Proven track record in sales or customer service.
- Passion for travel and knowledge of tourist destinations.
- Ability to work in a fast-paced environment.
You would be leading the Post-sales management and will closely coordinate with the Sales team in US and with Customers.
You may also be helping us in lead generation via LinkedIn and other media.
NodeJS (Node.js)
Angular (2+)
Javascript- Hands-on experience with Node JS and Angular (2+ years).
- Experience with popular frameworks like Express / Loopback.
- Creating and Integrating backend REST APIs
- Hands-on experience with any RDBMS systems.
- Passion about complex, interactive applications with a thoughtful UX/UI.
- Should be able to understand project requirements thoroughly.
- A deep understanding of the importance of building maintainable, efficient, clean code, while balancing the urgency of business needs.
- Collaborative attitude and experience working with cross-functional teams.
Responsibilities
- Responsible for driving & delivering web applications.
- Responsible for architecting, solutioning of the project in the requirement phase;
- Responsible for working with the team & enhancing the collaboration, adding value to the project with critical thinking on project design.
- Responsible for creating and integrating the REST APIs
Good to have
- Experience with AWS / Azure.
- Experience with CI/CD tools.
- Experience with any testing and automation frameworks.
Django
FlaskPython
Elastic Search
Amazon Web Services (AWS)Company Description
SteelEye is a fast growing FinTech company based in London and has offices in Bangalore and Paris, that offers a data platform to help financial institutions such as Investment Banks, Hedge Funds, Brokerage Firms, Asset Management Firms to comply with financial regulations in the European Union. Our clients can aggregate, search, surveillance and report on trade, communications and market data. SteelEye also enables customers to gain powerful insights from their data, helping them to trade with greater efficiency and profitability.
The company has a highly experienced management team and a strong board, who have decades of technology and management experience and worked in senior positions at many leading international financial businesses.
We are looking to hire a seasoned Sr API Developer to join us as we start on our next phase of growth. We have a culture of openness, collaboration, and the passion to get things done whilst appreciating the importance of a good work life balance.
Being part of a start-up can be equally exciting as it is challenging. You will be part of the SteelEye team not just because of your talent but also because of your entrepreneurial flare which we thrive on at SteelEye. This means we want you to be curious, contribute, ask questions and share ideas. We encourage you to get involved in helping shape our business.
SteelEye is a fast growing FinTech company based in London and has offices in Bangalore and Paris, that offers a data platform to help financial institutions such as Investment Banks, Hedge Funds, Brokerage Firms, Asset Management Firms to comply with financial regulations in the European Union. Our clients can aggregate, search, surveillance and report on trade, communications and market data. SteelEye also enables customers to gain powerful insights from their data, helping them to trade with greater efficiency and profitability.
The company has a highly experienced management team and a strong board, who have decades of technology and management experience and worked in senior positions at many leading international financial businesses.
We are looking to hire a seasoned Sr API Developer to join us as we start on our next phase of growth. We have a culture of openness, collaboration, and the passion to get things done whilst appreciating the importance of a good work life balance.
Being part of a start-up can be equally exciting as it is challenging. You will be part of the SteelEye team not just because of your talent but also because of your entrepreneurial flare which we thrive on at SteelEye. This means we want you to be curious, contribute, ask questions and share ideas. We encourage you to get involved in helping shape our business.
Job Description
What you’ll do
- Develop new features and functionality in SteelEye's best breed SaaS application
- Work on setup API Developer for our growing SaaS product
- Ensure the software is up-to-date with latest technologies
- Triage issues and enhance core features of our existing API's for rapidly growing client base
- Design and architech our API layer solving problems around scale and performance
- Be part of great team building a fantastic product!
What we’re looking for
- You are experienced Python Developer, with minimum 5 years in commercial Python experience
- You have experience building REST-based API's using Flask, Django or FastAPI in a production environment
- You have excellent knowledge of data structures, software design patterns and software architecture
- You are comfortable working in a fast-paced environment and can adapt to changing requirements
- You have experience with code versioning (Git) and CI/CD (Jenkins, Circle, etc..) tools
- Familiarity/Experience with Elasticsearch is a huge plus
- Familiarity/Experience with FastAPI nice to have
- You are experienced Python Developer, with minimum 5 years in commercial Python experience
- You have experience building REST-based API's using Flask, Django or FastAPI in a production environment
- You have excellent knowledge of data structures, software design patterns and software architecture
- You are comfortable working in a fast-paced environment and can adapt to changing requirements
- You have experience with code versioning (Git) and CI/CD (Jenkins, Circle, etc..) tools
- Familiarity/Experience with Elasticsearch is a huge plus
- Familiarity/Experience with FastAPI nice to have
Java
Spring BootWe are looking for a Java backend developer who would be working on the bleeding edge of technologies. We work primarily with a fully reactive stack powered by Spring Webflux and Reactive MongoDb Repository in AWS. Our services follow both event based approaches as well as workflow based depending on the use case.
Responsibilities
- Understand why a particular design was chosen and code accordingly
- Will have to deliver clean bug free unit tested code with minimal guidance
- Strive for continuous improvement by refactoring and applying best practices
- Learn and adapt to new technologies as necessary.
- Capable of working in both client side and server side technologies
Requirements
- Must have a Bachelor’s degree in computer science or equivalent
- Must have 2-3 years experience as a software developer
- Must be proficient in Core Java and Spring/Springboot. Knowledge of Webflux is not mandatory.
- Must have developed RESTful services
- Should understand git
Added Bonus
- Experience working with microservices
- Experience in a NoSql solution like MongoDB
- Experience working with Python
React Native
Android Development
iOS App DevelopmentExperience in IOS SWIFT Framework.
Proven experience in React Native components in native iOS and Android.
Good understanding of Android / iOS / Web design guidelines, SDK and excellent JavaScript skills. Knowledge of React and the Java script ecosystem.
Exposure to other hybrid mobile platforms like IONIC,Sencha, HTML5, PhoneGap, jQuery Mobile etc. Familiarity with RESTful APIs development.
Expertise with code testing best practices, including unit and integration testing to ensure quality of code. Familiarity with code versioning tools such as Git, SVN, Github.
Excellent verbal and written communication skills.
Preferred Skills & Knowledge: IOS Development (Swift), React Native.
Laravel- Excellent working knowledge of Web application development
- Advance coding Skills in PHP, HTML, CSS, JavaScript, and scripting languages desirable
- Excellent working knowledge of MySQL database
- Good understanding of database performance tuning and sql queries
- Experience working with a Laravel framework.
- Experience in both Front End / Back End Developer.
- Good Knowledge and understanding of CRM, CMS, SHOPPING-CART, PAYMENT GATEWAY & OTHER API INTEGRATION,
- Good communication skills9.Experience working in agile software development methodology
- Understanding of cross platform OS systems like Windows, Linux or UNIX with hands-on working experience
- Have worked on projects like e-Commerce, CRM, CMS, Individual Contributor role.
- Key skills: HTML4/5, CSS3, PHP, Apache, MySQL, Laravel, WORDPRESS , Bootstrap, jQuery, JavaScript, AJAX,
Responsibilities: The job requires PHP experience as well as expertise with Laravel or any other framework.
This Developer is expected to have strong technical skills, to work closely with the other team members in
the development and managing key projects.Ability to work on a small team with minimal supervisionTroubleshoot,
test and maintain the core product software and databases to ensure strong optimization and functionality.
Javascript
HTML/CSSReact.jsAngular (2+)-
REQUIREMENTS
-
Write hand-coded HTML5, JS, and CSS(SCSS) that is thoroughly standards-compliant.
-
Solid Javascript experience.
-WordPress Must needed
-Experience designing and developing responsive design websites -
Knowledge of modern authorization mechanisms, such as JSON Web Token
-
You know how to architect all components of a frontend project
-
You have experience reading API documentation and working with different endpoints to compose the desired effect
-
You know how to run projects through solid processes (things like code reviews, reviewing stories, planning)
-
Ability to understand business requirements and translate them into technical requirements
-
Picks up new technologies quickly and efficiently
-
Uses Git effectively
-
Know how to create tests and why it's important
-
Knowledge of OOP and software design principles
-
You know how to optimize frontend load times, debug rendering issues, and overall provide a smooth experience to the user.
-
Knowledge of responsive frontends
-
You don't have to be an expert in backend tech, but you'll need to understand how to integrate with the backend guys.
-
Understanding differences between multiple delivery platforms, such as mobile vs. desktop, and optimizing output to match the specific platform
-
-
WHAT do WE expect?
-
Someone who is comfortable taking on multiple projects at the same time, juggling tasks.
-
Experience of working in and collaborating with teams
-
Should take ownership of tasks.
-
Acumen. You must be able to think and judge multiple solutions for a problem and be able to reason between them.
GOOD TO KNOW
-
Experience in Linux and command-line interface.
-
JS/Webpack/Browserify.
-
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon