GCP Data Engineer at VY SYSTEMS PRIVATE LIMITED · Bengaluru (Bangalore), Hyderabad · 5 - 18 years · ₹4L - ₹35L / yr · Profitable · Posted 16 Sep 2026

Senior Data Engineer – Ab Initio | GCP | Spark | Agentic AI
Location: Bangalore
Experience: 5+ Years
Role: Senior Data Engineer
Work Mode: Bangalore
Job Summary
We are looking for an experienced Senior Data Engineer with strong expertise in Ab Initio, GCP, Apache Spark, and Agentic AI. The ideal candidate will have hands-on experience designing and developing scalable data engineering solutions, building data pipelines, and working with modern cloud and AI technologies.
The candidate should be comfortable working across traditional enterprise data platforms and emerging Generative AI / Agentic AI solutions.
Key Responsibilities
- Design, develop, and maintain scalable and high-performance data pipelines using Ab Initio, Spark, and GCP services.
- Develop and optimize complex ETL/ELT workflows using Ab Initio.
- Build and maintain data processing solutions using Apache Spark / PySpark.
- Develop cloud-based data solutions on Google Cloud Platform (GCP).
- Work with GCP data services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Pub/Sub, or equivalent services.
- Perform data integration, transformation, cleansing, and validation.
- Optimize data pipelines for performance, scalability, reliability, and cost.
- Collaborate with data architects,

About VY SYSTEMS PRIVATE LIMITED
About
Vy Systems is a Global Technology consulting, Solutions, and Managed Technology Services company. We service our customers with ‘RESPONSIVENESS’ as a key factor and we believe that timely response to any transaction increases the operational efficiency and accelerates the revenue and profitability to our customers.
The Company is founded and managed by a team of professionals having more than two+ decades of global experience in the business of Technology Consulting and Services.
Tech stack
Similar jobs (10)
*Key Responsibilities*
*Core Experience*
- Extensive functional experience in *SAP WM and SAP MM* modules
- Hands-on experience in full lifecycle SAP implementations, rollouts, upgrades, and enhancements in *Agile/Scrum* environment
*Technical & Functional Expertise*
- *SAP Modules*: SAP WM, SAP MM, SAP IM, SAP SD, SAP PP
- *Configuration*: Org master data, storage strategies, info types, dependency mapping
- *Development & Enhancements*: Requirement definition for User Exits, RICEF objects, custom FS
- *Integrations*: Experience integrating SAP with RF scanners and 3rd-party shipping systems like FedEx, UPS
*Soft Skills & Delivery*
- Strong stakeholder management and communication skills
- Process harmonization, fit/gap analysis, change management
- End-user training and documentation
Davis Index is a leading market intelligence platform and publication that specializes in providing accurate and up-to-date price benchmarks for ferrous and non-ferrous scrap, as well as primary metals. Our dedicated team of reporters, analysts, and data specialists publishes and processes over 1,400 proprietary price indexes, metals futures prices, and other reference data. In addition, we offer market intelligence, news, and analysis through an industry-leading technology platform. With a global presence across the Americas, Asia, Europe, and Africa, our team of over 50 professionals works tirelessly to deliver essential market insights to our valued clients.
Job Overview:
We are looking for an enthusiastic and energetic Full Stack Intern to join our development team. This is a fantastic opportunity for students or recent graduates who have a passion for coding and want to gain hands-on experience in developing web applications.
Key Responsibilities:
- Contribute to the development of web applications using the MERN stack (MongoDB, Express.js, React, Node.js).
- Collaborate with the team to design, develop, and test new features.
- Work with Python for backend integrations and scripting.
- Debug and troubleshoot issues in existing applications.
Required Skills and Qualifications:
- Proficiency in React and Express.js.
- Working knowledge of Python.
- Completed 1-2 small to mid-level MERN stack projects.
- Strong communication skills and a positive attitude.
Application Process:
- Candidates must submit their project portfolio showcasing the MERN stack projects they have worked on.
- Shortlisted candidates will be given a beginner-level technical test to assess their skills.
- Candidates should demonstrate strong communication skills and enthusiasm during the interview process.
Objectives
- Budget: Management of budget for Hardware Asset Management tools, services and staffing (permanent and temporary)
- Talent management: Permanent, contract, temporary and graduate trainees
- Matrix management: Management of staff with Hardware Asset Management responsibility in other business units
- Lead third-party Hardware Asset Management resources: Supervisory responsibility for third-party resources supporting Hardware Asset Management activity
- Financial: Oversight of spending on technology asset support, which includes hardware, software, cloud services, support, maintenance and hardware disposal
- Environment: Endpoints and edge devices, software and service asset vendors, on-premises data center assets, and cloud services
- Other: Liability management and remediation tracking
The IT asset manager will:
- Ensure that appropriate resources (people, tools, services) and processes are in place to gather and analyze data relating to technology asset life cycles and related processes.
- Promote and advocate the value of Hardware Asset Management to various leaders. This includes developing and promoting effective Hardware Asset Management strategies, policy, approaches and practices across the full range of digital technology assets used within the organization.
- Identify and report on breaches of Hardware Asset Management policy, and track remedial action.
- Report on (or develop reporting on) remediation of digital technology asset-related risks and progress of optimization activity.
- Ensure accurate and timely reporting, and that data completeness and quality issues are quickly identified and escalated to the responsible data or process owners.
- Gather data on and provide analysis of all activities that have an impact on the value, cost and risk of digital technology asset life cycles.
- Provide finance with the data to support the analysis of digital technology asset project and maintenance budgets and business cases.
- Assign responsibilities for sub disciplines, such as IT asset disposition (ITAD).
- Ensure that Hardware Asset Management -related tools and services are properly evaluated, selected, implemented, configured and maintained, with appropriate integration with other sources of organizational data.
- Provide access to technology asset data in support of essential activities required to support the effective running of the business.
- Manage the everyday functions of the core Hardware Asset Management team. This includes staffing, budgeting and other relevant management functions required to hold all IT and business stakeholders accountable for optimizing the cost, risk and benefit of digital technology assets throughout their life cycle.
- Managing the deployment and collection of devices and other hardware assets to the associates or offices.
The IT asset manager acts in a leadership role and must demonstrate the following leadership attributes:
- Strategic thinking — An understanding of strategic business objectives and the ability to develop a vision and strategy for Hardware Asset Management and execute it effectively in order to drive results toward those objectives
- Interpersonal skills — Proven ability to collaborate, build relationships and influence individuals at all levels in a matrix management environment (as well as external vendors and service providers) to ensure that segregation and overlapping roles are identified and coordinated
- Strategic relationship management — Includes working with vendors during compliance audits
- Strategic vision — Ability to sense emerging needs and drive change as Hardware Asset Management develops and matures to meet organizational and technology challenges
The Hardware asset manager must have the following skills:
- Strong communication skills with a proven ability to understand key business and technical concepts, and then effectively communicate these concepts with technical staff, business stakeholders and senior management
- Strong organizational skills, the ability to perform under pressure and to manage multiple priorities with competing demands for resources
- Strong analytical, data processing and problem-solving skills
- Proficiency in process formulation and improvement
- Ability to read and interrupt contracts to properly manage digital technology, which includes license and maintenance agreements
- Ability to manage the deployment and collection of devices and other hardware assets to the associates or offices in a large scale. Demonstrable experience of managing assets in hundreds of thousands will be required.
The Hardware asset manager must have in-depth knowledge and experience of the following:
- Technology asset life cycle (whether digital technology, information, financial or physical assets), best practice governance, tools and services
- Policy, process, and procedure development and implementation
- Technology contracts and their likely cost implications through the technology asset life cycle
- Engagement with sourcing, procurement and vendor management legal, tax and accountancy advisors for additional information
- Data processing, analysis and quality management tools
- Ability to manage effective deployment, collection and disposition of assets and managing the devices through its lifecycle
Additionally, an understanding of IT business continuity, cybersecurity, and integrated risk management and optimization methodologies is highly desirable, along with knowledge of IT management.
The following experience is considered essential:
- A management role, including engagement with sourcing, procurement and vendor management, business stakeholders and frontline operational IT staff
- Prior Hardware Asset Management roles (hardware or software) or demonstrable experience of working with IT asset management professionals to deliver Hardware Asset Management capability and improved business outcomes
- Atleast 16 years of experience with IT, enterprise asset management or information management
- Demonstrated leadership experience building cross-organizational consensus with exposure to technology providers and/or business clients
- Demonstrated experience in liaising with middle and senior management
- Experience with building business requirements for digital technology tools and services, developing business cases, and selecting solutions using RFP documentation and processes
- Experience leading system or tool implementations, with responsibility for verifying capabilities, outputs, dependencies and implementation scope
- Experience managing deployment, collection and disposition of assets in a large scale with globally distributed workforce.
The following experience is considered desirable, but not mandatory:
- 13+ years of experience working in the working in a large IT/ITeS organization
- 13+ years of experience in managing external IT service providers handling digital technology assets
- Experience with ServiceNow platform
This is a sensitive role dealing with commercial risks and significant costs. The organization must have a high level of confidence in the integrity and track record of the individual appointed to this role.
Certifications
The following certifications are considered desirable, but not mandatory:
- ITIL v3 Foundation certification
- Certified IT Asset Manager
- COBIT 5 Foundation
Education and Training
- An undergraduate or postgraduate degree in finance, IT, engineering, business management or a related field
- Tertiary qualifications in financial accounting, project management or data analysis, or a legal or paralegal qualification that includes contract, copyright or intellectual property law are preferred
Job Description:
An Assistant Engineer works as part of an engineering team to support various aspects of production, design, quality control, and maintenance. They are involved in project planning, process improvement, equipment maintenance, and troubleshooting. The role also includes monitoring safety regulations, analyzing test results, and ensuring compliance with industry standards.
Key Responsibilities:
- Technical Support & Project Assistance
- Assist senior engineers in project design, execution, and troubleshooting.
- Conduct research, collect data, and prepare technical reports.
- Participate in product development and testing processes.
- Production & Process Optimization
- Monitor production activities and ensure smooth operation.
- Identify inefficiencies and propose process improvements.
- Coordinate with production teams to maintain workflow efficiency.
- Quality Control & Assurance
- Conduct quality checks and inspections of materials, components, and finished products.
- Ensure products meet industry and regulatory standards.
- Assist in root cause analysis and corrective action implementation.
- Equipment Maintenance & Safety Compliance
- Ensure machines and tools are properly maintained.
- Follow safety protocols and assist in risk assessment.
- Train workers on the safe operation of equipment.
- Documentation & Reporting
- Maintain records of production, quality tests, and project progress.
- Prepare reports for senior engineers and management.
- Assist in preparing SOPs (Standard Operating Procedures).
Supervise customer service team, including coaching, training and daily assignments of
customer service agents; responsible for providing feedback of the agents to the agency
and for course corrections
Liaise with customer service agency project manager, ensuring efficient communication
channels are put in place between Mobile App team and customer agency.
Develop policies and procedures. Prepare a playbook that customer service agents follow to
resolve customer inquiries, including workflows and SOPs. Policies and procedures must
balance satisfying customer needs and supporting the goals of the organization. This would
include constant review and revision as customer needs change and the business evolves.
Ensure that customer service agents are equipped with an up-to-date repository of responses.
Respond to customer queries, as needed.
Responsible for configuring and managing HubSpot dashboard, with the goal to fulfill data &
analytics requirements from different stakeholders.
Lead/Facilitate meetings with different stakeholders at regular intervals to ensure relevant data
is being effectively communicated to the various departments as needed.
Serve as point person for product team to provide future enhancements, bug fixes etc /
back communication from stakeholders. Be the voice of the customer advocating for
changes most requested/needed.
Experience of Linux
Experience using Python or Shell scripting (for Automation)
Hands-on experience with Implementation of CI/CD Processes
Experience working with one cloud platforms (AWS or Azure or Google)
Experience working with configuration management tools such as Ansible & Chef
Experience working with Containerization tool Docker.
Experience working with Container Orchestration tool Kubernetes.
Experience in source Control Management including SVN and/or Bitbucket
& GitHub
Experience with setup & management of monitoring tools like Nagios, Sensu & Prometheus or any other popular tools
Hands-on experience in Linux, Scripting Language & AWS is mandatory
Troubleshoot and Triage development, Production issues
We are looking for a Full Stack Developer to build scalable software solutions and to be part of our cross-functional team that’s responsible for the full software development life cycle, from conception to deployment.
You should have
- Degree in Computer Science or relevant field
- Strong knowledge and understanding of JavaScript and TypeScript
- Knowledge of multiple frontend languages and frameworks including HTML, CSS, SCSS, JavaScript and Vue.
- Knowledge of backend languages and frameworks primarily Node.js
- Familiarity with SQL databases (e.g. MongoDB)
Perks
- Flexible working hours - employee tracking is never a thing
- A fun and close-knit agile team.
- Open office layout that promotes collaboration, conversation, and camaraderie – the result-oriented team.
- Recreational activities to keep you away from the screen and live, laugh, enjoy with the team.









