Collibra - Bengaluru Location - (F2F Interview 0n 7th-Aug-2026)

IMMEDIATE JOINERS ONLY

Key Responsibilities:

1                   Data Flow Mapping & Lineage

2                   Design and produce end-to-end data flow maps for critical data items – from market source/core systems, through Data Lake staging, zone 2 and consumption zone 3, to downstream reporting, dashboards and analytics use cases.

3                   Document data lineage and transformations for critical data elements identified in the Master Data Requirements exercise, ensuring traceability that is audit-defensible and supports regulatory, governance and AI-readiness needs.

4                   Map data availability by country/Data Lake against the Product Hierarchy (to profit-centre level), building the consolidated Commercial Data Landscape view of what exists, where it sits (source only, staging, or consumption) and what is not captured at all.

5                   Document cross-system dependencies and cross-border data flows, flagging data residency, privacy and regulatory considerations to the appropriate governance forums.

6                   Data Inventory, Data Model Alignment & Gap Analysis

•                    Own and mature the data asset inventory, catalogue and data dictionary aligned to the consolidated Master Data Requirements Template – resolving “Not Specified”, “TBD” and “validation required” availability statuses through structured validation with Market IT and Data Engineering.

•                    Align business data requirements to the conceptual and logical data model of the target Data Lakehouse, ensuring the dependency between data model design and lineage tracking of critical data items is explicitly documented.

•                    Perform gap and hydration analysis by market and product, quantifying the scale of data that must be brought into the Data Lakes and providing sizing input to the Data Foundation Lead, DAE architects and Data Engineering delivery plans.

•                    Contribute to the Data Consolidation Plan towards a single source of truth, including recommended sequencing based on business priority (Top 200 fields, Horizon 1–3 priorities).

•                    Stakeholder Engagement & Workshops

•                    Work hand-in-hand with the Data Foundation Lead, Data & Analytics Programme Manager, Data Engineers, DAE/infrastructure architects, Local Market IT and Business stakeholders (Regional and Market) as the connective tissue between business data needs and technical implementation.

•                    Plan and facilitate structured interviews and cross-functional validation workshops with business, technology and data stakeholders to corroborate actual operating data flows against documented processes – establishing a fact-based, evidence-backed view rather than relying on documentation alone.

•                    Translate ambiguous business asks into clear data definitions, mapping rules and acceptance criteria that Data Engineering can build against, and play back findings to senior stakeholders in clear, visual, decision-oriented formats.

•                    Manage competing priorities across multiple markets with tact, keeping momentum in an environment of constrained capacity and differing local system landscapes.

•                    Governance, Ownership & Documentation

•                    Produce and maintain the Data Ownership & Accountability Matrix – mapping data owners, stewards and custodians per data asset and highlighting ownership gaps for remediation.

•                    Classify data assets by sensitivity and business function, aligning to Data Protection Policy, PDPA and applicable regulatory expectations (e.g. MAS guidelines) in partnership with Risk & Compliance.

•                    Ensure all data flow maps, lineage records and inventory artefacts are version-controlled, audit-ready and handed over cleanly into BAU data governance processes.

•                    Produce source to Datalake mapping document and data catalogue for missing data elements to complete Datalake hydration

Key Skills

•                    Data flow mapping and lineage documentation – proven ability to draw clear, accurate source-to-consumption data flow maps across complex multi-system, multi-market landscapes.

•                    Data architecture and modelling – strong grasp of conceptual/logical data models, data lake/lakehouse zone architectures (staging, conformed, consumption), data marts and MDM concepts.

•                    Data inventory, catalogue, dictionary and metadata management; familiarity with lineage/catalogue tooling (e.g. Purview, Collibra) and diagramming tools (e.g. Visio, ErWin, Lucidchart).

•                    Hands-on data profiling and validation skills (SQL essential) to verify what actually exists in source systems and lake zones.

•                    Strong stakeholder management – this is critical: able to build trust and drive outcomes across regional leadership, market business teams, Market IT and engineering, and to facilitate effective cross-functional workshops.

•                    Insurance domain knowledge – ideally commercial and specialty lines, with an understanding of policy, claims, reinsurance and underwriting data.

•                    Data quality, governance and privacy concepts, including cross-border data considerations.

•                    Clear written and visual communication – able to make complex data landscapes understandable to executives.

Experience & Qualification Requirements:

•                    8+ years’ experience in data architecture, data business analysis or information architecture roles, with at least:

•                    3+ years documenting data flows, lineage or data landscapes across multi-system / multi-market environments.

•                    Experience in Insurance or Financial Services, ideally within data foundation, data lake or data governance programmes.

•                    Demonstrated delivery of data inventory / data mapping / current-state data landscape assessments involving both business and IT stakeholders.

•                    Relevant certification (e.g. DAMA CDMP, TOGAF, cloud data platform certifications) preferred.

Mandatory Skills (Must Have)

• SQL and Database Design (5+ years in enterprise data platforms)

• Python development for automation and platform services (5+ years)

• Linux administration and Shell Scripting in production environments

• Metadata-driven systems and data management platform experience

• Technical leadership of offshore or distributed development teams Wissen Technology

• System analysis, troubleshooting, and production support expertise

• Metadata onboarding, integration, and governance processes

• Stakeholder management and delivery ownership across multiple teams

Key Responsibilities

• Lead developers supporting the Pantheon Metadata platform and related services

• Own end-to-end delivery of metadata onboarding and enhancement initiatives

• Design scalable metadata management solutions and automation capabilities

• Collaborate with architecture, infrastructure, and application teams for enterprise integration

• Participate in solution design reviews and establish development best practices

• Prioritize enhancements with business stakeholders and manage delivery commitments

• Provide production support, troubleshooting, and platform stability improvements Wissen Technology

• Drive modernization initiatives related to metadata lifecycle management

SHARE YOUR RESUME : { pavithra.e @vysystems.com },

{ 8 9 2 5 8 0 2 4 5 7 }

Job Description – Data / Metadata Engineer

Position

Data / Metadata Engineer

Location

Pune

Experience

• 6–9 Years: Up to 28 LPA
• 10–17 Years: Up to 32 LPA

Notice Period

Immediate to 15 Days Joiners Preferred

Role Overview

We are looking for an experienced Data / Metadata Engineer with strong expertise in Data Engineering, Metadata Management, Data Modeling, Cloud technologies, Python/PySpark, and ETL/Data Pipelines.

The ideal candidate should have hands-on experience designing and developing data solutions, working with metadata and data models, and building scalable data pipelines on cloud platforms.

Mandatory Skills

• Strong experience in Data Engineering
• Hands-on experience in Metadata Management / Metadata Engineering
• Strong understanding of Data Modeling
• Experience with DDL and ER/Data Modeling
• Experience with Cloud technologies – Any Cloud Platform
• Strong programming experience in Python or PySpark
• Experience with ETL processes and Data Pipelines
• Good understanding of data architecture and data integration concepts

Key Responsibilities

• Design, develop, and maintain scalable data engineering solutions.
• Develop and manage data pipelines and ETL workflows.
• Work with metadata and data cataloging concepts.
• Create and maintain data models, DDLs, ER diagrams, and database structures.
• Develop data processing solutions using Python or PySpark.
• Work with cloud-based data platforms and services.
• Collaborate with Data Architects, Data Engineers, Developers, and Business teams.
• Ensure data quality, consistency, availability, and reliability across data pipelines.
• Troubleshoot and optimize existing data pipelines and data processes.
• Follow best practices for data engineering, modeling, and cloud-based implementations.

Candidate Profile

• Strong experience in Data Engineering + Metadata + Data Modeling.
• Good hands-on experience with Python/PySpark and ETL/Data Pipelines.
• Experience working with any major cloud platform.
• Strong understanding of databases, data structures, and data modeling concepts.
• Candidates who can join immediately or within 15 days are preferred.

Job Details

Role: Data / Metadata Engineer

Location: Pune

Experience: 6–17 Years

Budget: 28 LPA (6–9 Yrs) / 32 LPA (10–17 Yrs)

Notice Period: Immediate to 15 Days

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills

About Us:

The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.

Job Summary:

We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.

As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.

Key Responsibilities:

• Design, develop, test, and maintain optimal data pipeline and ETL architectures.
• Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
• Prepare and optimize data for predictive and prescriptive modeling.
• Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
• Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
• Utilize big data tools and frameworks to optimize data acquisition and preparation.
• Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
• Develop and curate data models for analytics, dashboards, and reports.
• Conduct code reviews, maintain production-level code, and implement testing approaches.
• Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
• Drive innovation and implement efficient new approaches to data engineering tasks.

Must-Have Skills:

• Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
• 5+ years of experience working with enterprise data platforms, including building and managing data lakes.
• 3–5 years of experience designing and implementing data warehouse solutions.
• Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
• Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
• Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
• Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
• Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
• Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
• Strong problem-solving, communication, and collaboration skills.

Good-to-Have Skills:

• Experience in integrating ERP data into data lakes.
• Experience with traditional ETL tools (e.g., Talend, Pentaho).

Competencies:

• Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
• Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
• Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
• Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
• Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.

Why Join Us?

• Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
• Work on impactful projects that make a difference across industries.
• Opportunities for professional growth and continuous learning.
• Competitive salary and benefits package.

Application Details

Ready to make an impact? Apply today and become part of the QX Impact team!

We are looking for a Lead Data Architect to design, build, and scale our data pipelines and entity resolution systems. This role combines deep technical expertise in data engineering with hands-on experience in AI-assisted tooling, entity matching, and data integration from diverse sources. You will lead architectural decisions for our data platform, mentor engineers, and ensure our pipelines are reliable, scalable, and production-grade.

Key Responsibilities

     Architect, build, and maintain robust, scalable data pipelines that ingest, transform, and serve data from multiple internal and external sources.

     Own the end-to-end orchestration of data workflows using tools like Dagster, ensuring observability, reliability, and maintainability of pipelines.

     Design and implement entity resolution workflows — including matching, merging, and survivorship logic — using tools such as Splink, to produce clean, deduplicated, golden records.

     Build and maintain web scrapers to source data from external providers, ensuring resilience to source changes, rate limits, and data quality issues.

     Integrate and reconcile data coming from multiple, often inconsistent, sources into unified, trustworthy datasets.

     Design and maintain data models and schemas across transactional and analytical systems, ensuring consistency, scalability, and performance.

     Leverage AI/LLM-based tools and techniques to enhance data pipeline capabilities — e.g., intelligent data extraction, automated data quality checks, or AI-assisted entity matching.

     Define and enforce best practices around pipeline design, testing, monitoring, and documentation.

     Collaborate closely with data engineers, product managers, and other stakeholders to translate business requirements into scalable data architecture.

     Provide technical leadership and mentorship to the data engineering team.

Required Skills & Experience

     Strong hands-on experience building and maintaining production-grade data pipelines at scale.

     Practical experience with Dagster (or similar orchestration tools like Airflow/Prefect) for pipeline orchestration.

     Experience with Splink or similar probabilistic/deterministic record linkage tools for entity matching, merging, and survivorship.

     Strong proficiency in Python, including experience writing and maintaining web scrapers.

     Proven experience integrating and maintaining data pipelines that pull from multiple, heterogeneous data sources.

     Experience applying AI/ML tools within data engineering workflows (e.g., LLM-assisted data cleaning, extraction, or matching).

     Hands-on experience with relational and distributed databases such as PostgreSQL and Google Cloud Spanner.

     Strong understanding of data modeling principles (normalization, dimensional modeling, schema design) across OLTP and OLAP systems.

     Experience with cloud data warehousing platforms such as BigQuery, Redshift, and cloud platforms (GCP/AWS/Azure).

     Strong communication skills and experience working cross-functionally with engineering and product teams.

     Experience with distributed data processing frameworks (e.g., Spark, Dask).

     Familiarity with data governance, lineage, and cataloging tools.

     Prior experience in a lead or architect-level role guiding a data engineering team.

What We're Looking For

A technically strong, hands-on leader who can balance architectural thinking with the practical grit of debugging a flaky scraper or tuning a matching algorithm — someone who's comfortable owning both the big picture and the messy details of real-world data.

Next

Data Engineer Short Hiring Post

🚨 Hiring: Data Engineer

🔹 Experience: 5–9 Years

🔹 Location: Bangalore / Hyderabad

🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling

🔹 Process: L1 Virtual → L2 F2F Karat Test

🔹 F2F: Bangalore / Hyderabad Location

🔹 Positions: Immediate requirement

⚠️ Note: Candidates must be available for F2F Karat immediately after L1.

#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners

Strong Data Engineer (AWS) Profile

2

Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.

3

Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.

4

Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.

5

Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.

6

Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.

7

Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.

8

Mandatory (Education) – B.Tech / B.E./ M.Tech

Job Description:

We are looking for an experienced Ab Initio Developer to join our team. The candidate will be responsible for developing, enhancing, and maintaining data integration and ETL solutions using Ab Initio.

Roles & Responsibilities:

• Develop and maintain ETL workflows using Ab Initio.
• Design and develop data integration solutions based on business requirements.
• Work with Ab Initio GDE, Co>Operating System, and related components.
• Develop and optimize graphs, plans, and batch processes.
• Write and optimize SQL queries for data processing.
• Perform data validation, debugging, and issue resolution.
• Analyze and improve the performance of existing ETL processes.
• Work with business and technical teams to understand requirements.
• Support production deployments and troubleshoot data-related issues.
• Follow coding, testing, and documentation standards.

Required Skills:

• 5–8 years of experience in Ab Initio Development.
• Strong experience in Ab Initio ETL/Data Integration.
• Hands-on experience with Ab Initio GDE and Co>Operating System.
• Good knowledge of SQL and database concepts.
• Experience in ETL development, debugging, and performance tuning.
• Good communication and problem-solving skills.

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

Bang/hyderabad

immediate to 15days.