Lead I - Software Engineering - AI Solutions Analyst

Global Digital Transformation Solutions Provider

Lead I - Software Engineering - AI Solutions Analyst

Peak Hire Solutions

Company

Home

Lead I - Software Engineering - AI Solutions Analyst

at Global Digital Transformation Solutions Provider

Agency job

via Peak Hire Solutions

6 - 8 yrs

₹10L - ₹26L / yr

Bengaluru (Bangalore), Mumbai, Delhi, Gurugram, Noida, Ghaziabad, Faridabad, Pune, Hyderabad

Skills

Large Language Models (LLM)

Prompt engineering

Knowledge base

Large Language Models (LLM) tuning

Artificial Intelligence (AI)

Interaction Design

Integration

Data integration

Integration testing

databricks

Amazon Web Services (AWS)

ChatGPT

MUST-HAVES:

LLM, AI, Prompt Engineering LLM Integration & Prompt Engineering
Context & Knowledge Base Design.
Context & Knowledge Base Design.
Experience running LLM evals

NOTICE PERIOD: Immediate – 30 Days

SKILLS: LLM, AI, PROMPT ENGINEERING

NICE TO HAVES:

Data Literacy & Modelling Awareness Familiarity with Databricks, AWS, and ChatGPT Environments

ROLE PROFICIENCY:

Role Scope / Deliverables:

Scope of Role Serve as the link between business intelligence, data engineering, and AI application teams, ensuring the Large Language Model (LLM) interacts effectively with the modeled dataset.
Define and curate the context and knowledge base that enables GPT to provide accurate, relevant, and compliant business insights.
Collaborate with Data Analysts and System SMEs to identify, structure, and tag data elements that feed the LLM environment.
Design, test, and refine prompt strategies and context frameworks that align GPT outputs with business objectives.
Conduct evaluation and performance testing (evals) to validate LLM responses for accuracy, completeness, and relevance.
Partner with IT and governance stakeholders to ensure secure, ethical, and controlled AI behavior within enterprise boundaries.

KEY DELIVERABLES:

LLM Interaction Design Framework: Documentation of how GPT connects to the modeled dataset, including context injection, prompt templates, and retrieval logic.
Knowledge Base Configuration: Curated and structured domain knowledge to enable precise and useful GPT responses (e.g., commercial definitions, data context, business rules).
Evaluation Scripts & Test Results: Defined eval sets, scoring criteria, and output analysis to measure GPT accuracy and quality over time.
Prompt Library & Usage Guidelines: Standardized prompts and design patterns to ensure consistent business interactions and outcomes.
AI Performance Dashboard / Reporting: Visualizations or reports summarizing GPT response quality, usage trends, and continuous improvement metrics.
Governance & Compliance Documentation: Inputs to data security, bias prevention, and responsible AI practices in collaboration with IT and compliance teams.

KEY SKILLS:

Technical & Analytical Skills:

LLM Integration & Prompt Engineering – Understanding of how GPT models interact with structured and unstructured data to generate business-relevant insights.
Context & Knowledge Base Design – Skilled in curating, structuring, and managing contextual data to optimize GPT accuracy and reliability.
Evaluation & Testing Methods – Experience running LLM evals, defining scoring criteria, and assessing model quality across use cases.
Data Literacy & Modeling Awareness – Familiar with relational and analytical data models to ensure alignment between data structures and AI responses.
Familiarity with Databricks, AWS, and ChatGPT Environments – Capable of working in cloud-based analytics and AI environments for development, testing, and deployment.
Scripting & Query Skills (e.g., SQL, Python) – Ability to extract, transform, and validate data for model training and evaluation workflows.
Business & Collaboration Skills Cross-Functional Collaboration – Works effectively with business, data, and IT teams to align GPT capabilities with business objectives.
Analytical Thinking & Problem Solving – Evaluates LLM outputs critically, identifies improvement opportunities, and translates findings into actionable refinements.
Commercial Context Awareness – Understands how sales and marketing intelligence data should be represented and leveraged by GPT.
Governance & Responsible AI Mindset – Applies enterprise AI standards for data security, privacy, and ethical use.
Communication & Documentation – Clearly articulates AI logic, context structures, and testing results for both technical and non-technical audiences.

MUST-HAVES:

LLM, AI, Prompt Engineering LLM Integration & Prompt Engineering
Context & Knowledge Base Design.
Context & Knowledge Base Design.
Experience running LLM evals

NOTICE PERIOD: Immediate – 30 Days

SKILLS: LLM, AI, PROMPT ENGINEERING

NICE TO HAVES:

Data Literacy & Modelling Awareness Familiarity with Databricks, AWS, and ChatGPT Environments

ROLE PROFICIENCY:

Role Scope / Deliverables:

Scope of Role Serve as the link between business intelligence, data engineering, and AI application teams, ensuring the Large Language Model (LLM) interacts effectively with the modeled dataset.
Define and curate the context and knowledge base that enables GPT to provide accurate, relevant, and compliant business insights.
Collaborate with Data Analysts and System SMEs to identify, structure, and tag data elements that feed the LLM environment.
Design, test, and refine prompt strategies and context frameworks that align GPT outputs with business objectives.
Conduct evaluation and performance testing (evals) to validate LLM responses for accuracy, completeness, and relevance.
Partner with IT and governance stakeholders to ensure secure, ethical, and controlled AI behavior within enterprise boundaries.

KEY DELIVERABLES:

LLM Interaction Design Framework: Documentation of how GPT connects to the modeled dataset, including context injection, prompt templates, and retrieval logic.
Knowledge Base Configuration: Curated and structured domain knowledge to enable precise and useful GPT responses (e.g., commercial definitions, data context, business rules).
Evaluation Scripts & Test Results: Defined eval sets, scoring criteria, and output analysis to measure GPT accuracy and quality over time.
Prompt Library & Usage Guidelines: Standardized prompts and design patterns to ensure consistent business interactions and outcomes.
AI Performance Dashboard / Reporting: Visualizations or reports summarizing GPT response quality, usage trends, and continuous improvement metrics.
Governance & Compliance Documentation: Inputs to data security, bias prevention, and responsible AI practices in collaboration with IT and compliance teams.

KEY SKILLS:

Technical & Analytical Skills:

LLM Integration & Prompt Engineering – Understanding of how GPT models interact with structured and unstructured data to generate business-relevant insights.
Context & Knowledge Base Design – Skilled in curating, structuring, and managing contextual data to optimize GPT accuracy and reliability.
Evaluation & Testing Methods – Experience running LLM evals, defining scoring criteria, and assessing model quality across use cases.
Data Literacy & Modeling Awareness – Familiar with relational and analytical data models to ensure alignment between data structures and AI responses.
Familiarity with Databricks, AWS, and ChatGPT Environments – Capable of working in cloud-based analytics and AI environments for development, testing, and deployment.
Scripting & Query Skills (e.g., SQL, Python) – Ability to extract, transform, and validate data for model training and evaluation workflows.
Business & Collaboration Skills Cross-Functional Collaboration – Works effectively with business, data, and IT teams to align GPT capabilities with business objectives.
Analytical Thinking & Problem Solving – Evaluates LLM outputs critically, identifies improvement opportunities, and translates findings into actionable refinements.
Commercial Context Awareness – Understands how sales and marketing intelligence data should be represented and leveraged by GPT.
Governance & Responsible AI Mindset – Applies enterprise AI standards for data security, privacy, and ethical use.
Communication & Documentation – Clearly articulates AI logic, context structures, and testing results for both technical and non-technical audiences.

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma

Full Stack Developer - Averlon

I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.

Companies hiring on Cutshort

Similar jobs

Sr. QA Automation (Cypress)

at furation tech

Posted by akash yadav

Navi Mumbai, Mumbai

5 - 10 yrs

₹4L - ₹10L / yr

cypress

Job Title: Sr. Automation Tester

Location: CBD Belapur – Navi Mumbai

Experience: 5 to 10 years

Educational Qualification: B.E / B.Tech / B.Sc-IT / BCA / B.Sc / M.E / MCA / M.Sc-IT /

M.Tech

Role Overview

We are seeking an experienced Senior Automation Tester with a strong background in

designing, developing, and maintaining advanced automation frameworks — primarily

using Cypress and Cucumber. The ideal candidate will have hands-on expertise in test

automation for both UI and API layers, deep understanding of Agile practices, and a

proven ability to drive quality initiatives across large-scale enterprise systems.

Key Responsibilities

 Design, develop, and maintain robust, scalable, and reusable automation

frameworks using Cypress and Cucumber.

 Define and implement test automation strategies, standards, and best practices

across projects.

 Perform end-to-end automation for web and API testing, ensuring

comprehensive test coverage.

 Collaborate closely with developers, product owners, and business analysts to

define acceptance criteria and refine testing scope.

 Conduct code reviews, perform root cause analysis, and ensure timely defect

resolution.

 Integrate automation frameworks with CI/CD pipelines (Jenkins, GitLab, Azure

DevOps, etc.) to enable continuous testing.

 Evaluate and recommend new tools, frameworks, and technologies to improve

automation efficiency.

 Mentor and guide junior QA engineers, fostering a culture of quality and

continuous learning.

 Prepare accurate testing estimates, risk assessments, and provide regular status

reports to stakeholders.

Required Skills & Competencies

 Strong hands-on experience in Cypress and Cucumber (BDD approach).

 Solid understanding of JavaScript/TypeScript and test design principles.

 Experience in API automation using Postman, RestAssured, or similar tools.

 Working knowledge of Selenium WebDriver, TestNG, or Playwright is an added

advantage.

 Proficiency in CI/CD tools such as Jenkins, GitLab CI, or Azure DevOps.

 Experience with version control systems (Git, Bitbucket).

 Familiarity with test management tools like JIRA, Zephyr, or HP ALM.

 Strong understanding of STLC, Agile/Scrum, and DevOps practices.

 Excellent analytical, debugging, and problem-solving skills.

 Effective communication and collaboration skills, with the ability to lead

technical discussions.

Job Title: Sr. Automation Tester

Location: CBD Belapur – Navi Mumbai

Experience: 5 to 10 years

Educational Qualification: B.E / B.Tech / B.Sc-IT / BCA / B.Sc / M.E / MCA / M.Sc-IT /

M.Tech

Role Overview

We are seeking an experienced Senior Automation Tester with a strong background in

designing, developing, and maintaining advanced automation frameworks — primarily

using Cypress and Cucumber. The ideal candidate will have hands-on expertise in test

automation for both UI and API layers, deep understanding of Agile practices, and a

proven ability to drive quality initiatives across large-scale enterprise systems.

Key Responsibilities

 Design, develop, and maintain robust, scalable, and reusable automation

frameworks using Cypress and Cucumber.

 Define and implement test automation strategies, standards, and best practices

across projects.

 Perform end-to-end automation for web and API testing, ensuring

comprehensive test coverage.

 Collaborate closely with developers, product owners, and business analysts to

define acceptance criteria and refine testing scope.

 Conduct code reviews, perform root cause analysis, and ensure timely defect

resolution.

 Integrate automation frameworks with CI/CD pipelines (Jenkins, GitLab, Azure

DevOps, etc.) to enable continuous testing.

 Evaluate and recommend new tools, frameworks, and technologies to improve

automation efficiency.

 Mentor and guide junior QA engineers, fostering a culture of quality and

continuous learning.

 Prepare accurate testing estimates, risk assessments, and provide regular status

reports to stakeholders.

Required Skills & Competencies

 Strong hands-on experience in Cypress and Cucumber (BDD approach).

 Solid understanding of JavaScript/TypeScript and test design principles.

 Experience in API automation using Postman, RestAssured, or similar tools.

 Working knowledge of Selenium WebDriver, TestNG, or Playwright is an added

advantage.

 Proficiency in CI/CD tools such as Jenkins, GitLab CI, or Azure DevOps.

 Experience with version control systems (Git, Bitbucket).

 Familiarity with test management tools like JIRA, Zephyr, or HP ALM.

 Strong understanding of STLC, Agile/Scrum, and DevOps practices.

 Excellent analytical, debugging, and problem-solving skills.

 Effective communication and collaboration skills, with the ability to lead

technical discussions.

Manual Testing

at flywings enterprises

Posted by Rekha Gorle

Pune

4 - 4 yrs

₹4L - ₹7L / yr

Manual testing

Software Testing (QA)

Understand product requirements and translate them into effective test cases.

Perform manual functional, regression, UI, and integration testing.

Conduct thorough API testing using tools like Postman or Swagger.

Perform database validation using SQL queries to verify data integrity.

Create, manage, and execute detailed test plans and test cases.

Log and track defects using tools like JIRA, Bugzilla, or Azure DevOps.

Participate in Agile ceremonies (daily standups, sprint planning, retrospectives).

Collaborate closely with the development and product teams to reproduce issues and validate fixes