Key Responsibilities
ā Design & Development
āĀ Architect and implement dataĀ ingestion pipelines using Microsoft Fabric DataĀ Factory (Dataflows) and OneLake sources
ā Build and optimize Lakehouse and Warehouse solutionsĀ leveraging Delta Lake, Spark Notebooks, and SQL Endpoints
ā Define and enforce Medallion (BronzeāSilverāGold) architecture patterns for raw, enriched, and curated datasets
ā DataĀ Modeling & Transformation
āĀ Develop scalable transformation logic in Spark (PySpark/Scala) and Fabric SQL to support reporting and analytics
ā Implement slowly changing dimensions (SCD Type 2), change-data-capture (CDC) feeds, and time-windowed aggregations
ā Performance Tuning & Optimization
ā Monitor and optimize dataĀ pipelines for throughput, cost efficiency, and reliability
ā Apply partitioning, indexing, caching, and parallelism best practices in Fabric Lakehouses and Warehouse compute
ā DataĀ Quality & Governance
ā Integrate Microsoft Purview for metadata cataloging, lineage tracking, and dataĀ discovery
ā Develop automated quality checks, anomaly detection rules, and alerts for dataĀ reliability
ā CI/CD & Automation
ā Implement infrastructure-as-code (ARM templates or Terraform) for Fabric workspaces, pipelines, and artifacts
ā Set up Git-based version control, CI/CD pipelines (e.g.Ā AzureĀ DevOps) for seamless deployment across environments
ā Collaboration & Support
ā Partner with dataĀ scientists, BI developers, and business analysts to understand requirements and deliver dataĀ solutions
ā Provide production support, troubleshoot pipeline failures, and drive root-cause analysis
Required Qualifications
ā 5+ years of professional experience in dataĀ engineering roles, with at least 1 year working hands-on in Microsoft Fabric
ā Strong proficiency in:
ā Languages: SQL (T-SQL), Python, and/or Scala
ā Fabric Components: DataĀ Factory Dataflows, OneLake, Spark Notebooks, Lakehouse, Warehouse
ā DataĀ Storage: Delta Lake, Parquet, CSV, JSON formats
ā Deep understanding of dataĀ modeling principles (star schemas, snowflake schemas, normalized vs. denormalized)
ā Experience with CI/CD and infrastructure-as-code for dataĀ platforms (ARM templates, Terraform, Git)
ā Familiarity with dataĀ governance tools, especially Microsoft Purview
ā Excellent problem-solving skills and ability to communicate complex technical concepts clearly
NOTE: Candidate should be willing to take one technical round F2F from any of the branch location. (Pune/ Mumbai/ Bangalore)