Senior Data engineer

At Mitratech, we are a team of technocrats focused on building world-class products that simplify operations in the Legal, Risk, Compliance, and HR functions. We are a close-knit, globally dispersed team that thrives in an ecosystem that supports individual excellence and takes pride in its diverse and inclusive work culture centered around great people practices, learning opportunities, and having fun! Our culture is the ideal blend of entrepreneurial spirit and enterprise investment, enabling the chance to move at a rapid pace with some of the most complex, leading-edge technologies available.

For over 35 years, the experts at Mitratech have been focused on solving the complex needs. Today, we serve 20,000 client companies of all sizes globally, representing 30% of the Fortune 500 and over 500,000 users in over 160 countries.

As we continue to grow, we’re always looking for resourceful, enthusiastic, and fresh perspectives. Join our global team and see what makes Mitratech a truly exceptional place to work!

Job Overview 

Principal Data Engineer

About Engineering at Mitratech Legal Solutions 

Mitratech's engineering organization is a collaborative and dynamic environment where engineers are empowered to drive technical direction and innovation. Our engineers are passionate about delivering high-quality products and solutions that meet the evolving needs of our customers, and we're committed to fostering a culture of continuous learning and growth. 

About the Role 

Mitratech is a fast-paced and dynamic environment, and this role requires someone who is adaptable, resilient, and able to thrive in a rapidly changing landscape. If you’re a seasoned engineer with a passion for technical leadership, innovation, and collaboration — including building the data foundations that power trusted reporting and agentic AI-driven products — we’d love to hear from you. 

What You Will Do

•  Drive technical direction for a significant product domain or platform capability, ensuring alignment with business objectives and customer needs 

•  Design and maintain data pipelines and reporting models that power trusted business metrics and increasingly feed agentic AI systems (e.g., RAG ingestion, embeddings, vector stores, AI agent workflows) 

•  Use AI-assisted and agentic engineering tools (e.g., Claude Code, Copilot, Cursor, AI agents) as part of your own workflow, and help other engineers adopt agentic development practices effectively 

•  Reduce systemic complexity by identifying and leading architectural debt remediation, and developing strategies for ongoing technical debt management 

•  Partner with Product and Engineering leadership to inform multi-quarter roadmap feasibility, and provide technical guidance and oversight to ensure successful implementation 

•  Elevate engineering craft across multiple teams through RFCs, mentorship, and knowledge sharing, and develop training programs to improve engineering skills and knowledge 

•  Represent Mitratech’s technical capabilities externally, including speaking at conferences, contributing to open-source projects, and engaging with industry peers and thought leaders 

What We Are Looking For 

To be successful in this role, you will need: 

•  10+ years of experience in software engineering, with a focus on technical leadership and architecture 

•  Deep understanding of data engineering principles, including data modeling, data warehousing, reporting, and data governance 

•  Strong technical expertise in SQL, PostgreSQL, ETL/ELT pipelines, BI tools, and analytics platforms 

•  Practical experience with AI/LLM-adjacent and agentic AI data work — e.g., RAG ingestion pipelines, embedding generation, vector store management, or building/operating AI agent workflows over data — using AI coding assistants (Claude Code, Copilot, Cursor, or similar) as a regular part of the engineering workflow 

•  Working knowledge of modern cloud platforms such as AWS 

•  Experience with BI, reporting, dashboards, and customer-facing analytics 

•  Experience leading cross-functional initiatives with product, engineering, analytics, and business teams 

Nice to Have 

•  Working knowledge of Ruby on Rails and React 

•  Experience with a semantic or metrics layer (e.g., dbt Semantic Layer, headless BI) 

•  Understanding of CI/CD, Git-based workflows, and infrastructure-as-code 

The Stack Context 

•  Modern data stack: Fivetran, Airbyte, dbt, Snowflake, GitHub, Terraform, or similar tools 

•  Application context (nice to have): Ruby on Rails, React, or similar backend/frontend frameworks 

•  Data modeling: SQL, analytics models, documentation, testing, naming standards, and version control 

•  Infrastructure: cloud-based data infrastructure, infrastructure-as-code, CI/CD, monitoring, and cloud storage 

•  Data workflows: ingestion, transformation, orchestration, reporting, deployment, and change management 

•  Reporting focus: trusted metrics, scalable reporting models, dashboards, exports, and data quality 

•  AI surface: data pipelines and quality practices supporting AI/LLM and agentic AI use cases (RAG, embeddings, vector stores, AI agents) alongside traditional BI 

Why This Role 

This role offers a unique opportunity to drive technical direction and innovation at a rapidly growing company, while also mentoring and coaching engineers to improve their craft. As a Principal Data Engineer at Mitratech, you will have the chance to work on complex and challenging problems spanning trusted reporting and agentic AI systems, collaborate with cross-functional teams, and represent the company's technical capabilities externally. If you're looking for a role that offers a mix of technical leadership, data and reporting depth, agentic AI innovation, and collaboration, this could be the perfect fit for you. 

We are an equal-opportunity employer that values diversity at all levels. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, national origin, age, sexual orientation, gender identity, disability, or veteran status.

Job Summary

Role Overview

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.

Experience with Google Cloud Platform (GCP) will be an added advantage.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
• Develop complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain reliable data integration workflows across multiple data sources.
• Perform data cleansing, validation, transformation, and quality checks.
• Analyze data and provide insights to support business and technical requirements.
• Implement and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
• Troubleshoot data pipeline failures, performance issues, and production incidents.
• Optimize data processing workflows for performance, scalability, and reliability.
• Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
• Follow best practices for version control, testing, documentation, and deployment.
• Contribute to cloud-based data engineering initiatives, preferably on GCP.

Required Skills

• 5–7 years of hands-on experience in Data Engineering.
• Strong programming skills in Python.
• Strong expertise in Advanced SQL and database concepts.
• Hands-on experience with ETL/ELT processes and data pipelines.
• Good understanding of Data Warehousing and Data Modeling concepts.
• Experience with CI/CD practices and tools.
• Strong understanding of DevOps principles, automation, and deployment processes.
• Strong data analytics and problem-solving skills.
• Experience working with large datasets and performance optimization.
• Good understanding of Git/version control and software development best practices.

Good to Have

• Hands-on experience with Google Cloud Platform (GCP).
• Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
• Experience with containerization/orchestration technologies such as Docker/Kubernetes.
• Experience with workflow orchestration tools such as Airflow.
• Knowledge of cloud-based data architecture and distributed data processing.

Preferred Candidate Profile

• Strong analytical and problem-solving abilities.
• Good communication and stakeholder management skills.
• Ability to work independently as well as in a collaborative team environment.
• Strong ownership of data pipelines and production systems.
• Candidates who can join at short notice are preferred.

Mandatory Skills

 Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops

Job Summary

The Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python. The role involves designing, developing, and implementing data solutions while ensuring they meet the business requirements and align with best practices. (1.) Key Responsibilities

1. Lead and manage end-to-end data engineering projects using azure data factory, azure databricks, sql, oracle pl/sql, and python.

2. Collaborate with stakeholders to gather and understand requirements for data pipelines and analytics solutions.

3. Design and develop etl processes, data models, and data integration solutions.

4. Provide technical guidance and mentorship to the team members.

5. Ensure data quality, data governance, and data security standards are maintained throughout the project lifecycle.

6. Troubleshoot and optimize data pipelines and processes for performance and efficiency.

7. Stay updated on the latest trends and technologies in data engineering and contribute to continuous improvement efforts.

Skill Requirements

1. Proficiency in azure data factory (adf) and azure databricks for building and managing data pipelines.

2. Strong experience with sql and oracle pl/sql for data querying and manipulation.

3. Advanced programming skills in python for scripting and data processing tasks.

4. Knowledge of data modeling, data warehousing concepts, and database design principles.

5. Ability to work in a collaborative team environment and communicate effectively with stakeholders.

6. Strong analytical and problem-solving skills with attention to detail.

7. Experience in data visualization tools and techniques is a plus.

Certifications: Relevant certifications in Azure Data Factory, Azure Databricks, SQL, Oracle PL/SQL, or Python are advantageous.

Skill (Primary)

Data Fabric-Azure-Azure Data Factory (ADF)

Key Responsibilities

• Lead end-to-end data migration initiatives, including assessment, planning, mapping, transformation, validation, and reconciliation.
• Define and implement data governance frameworks, standards, policies, and processes.
• Design and manage data solutions using Microsoft Azure Data Services.
• Lead development of BI dashboards, reports, KPIs, and analytics solutions.
• Work with business and technical stakeholders to understand reporting and data requirements.
• Develop and maintain data models, data pipelines, ETL/ELT processes, and reporting architecture.
• Ensure data quality, consistency, integrity, security, and compliance throughout migration and reporting processes.
• Identify data risks, dependencies, gaps, and migration challenges and drive their resolution.
• Establish data validation and reconciliation mechanisms to ensure migration accuracy.
• Provide technical leadership and guidance to data engineers, BI developers, and other project team members.
• Collaborate with application, cloud, infrastructure, and business teams during project implementation.
• Monitor data migration and BI deliverables against project timelines, quality standards, and business objectives.
• Prepare technical documentation, data dictionaries, mapping documents, governance guidelines, and project reports.

Required Skills & Experience

• 7+ years of experience in data, migration, BI, or related technology roles.
• Strong experience working on software/IT projects and managing data-related workstreams.
• Hands-on experience with Microsoft Azure Data Services.
• Strong understanding of data migration methodologies, ETL/ELT, data transformation, and reconciliation.
• Experience in Data Governance, Data Quality, Master Data, Metadata Management, and Data Security.
• Strong experience with BI reporting and dashboard development.
• Good understanding of SQL and relational databases.
• Experience with Power BI and data visualization is highly desirable.
• Knowledge of Azure services such as Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, or equivalent.
• Strong understanding of data architecture and data lifecycle management.
• Excellent stakeholder management, communication, analytical, and problem-solving skills.

Data Engineer – Microsoft Fabric

Location: Pune, India

Work Mode: Hybrid

Experience: 6+ Years

Employment Type: Full-time contactor

Compensation: As per market standards, commensurate with experience and expertise

Shift Timings: 2:00 PM – 11:00 PM IST

Notice Period: 0 – 15 days

About the Role

Jade Business Services (JBS) is seeking a Data Engineer – Microsoft Fabric to join our Pune team and work on enterprise-scale data transformation and analytics initiatives.

We are looking for a hands-on Data Engineer with strong experience in Microsoft Fabric, SQL, Python/PySpark and modern data engineering practices. The candidate will be responsible for building scalable data pipelines, implementing Lakehouse and Warehouse solutions, developing data models and supporting governed, reliable and AI-ready data platforms.

The ideal candidate should be comfortable working with architects, engineering teams and client stakeholders to translate business requirements into scalable and production-ready data solutions.

Roles and Responsibilities

• Design and develop data solutions using Microsoft Fabric, including OneLake, Lakehouse, Warehouse and Data Factory pipelines.
• Build and maintain scalable ETL/ELT pipelines for batch and incremental data processing.
• Develop data ingestion and transformation pipelines using Fabric Data Factory, SQL, Python and/or PySpark.
• Implement Medallion Architecture using Bronze, Silver and Gold layers.
• Work with Lakehouse and Fabric Warehouse for enterprise data processing and analytics.
• Develop and maintain data models, tables, views and optimized SQL queries.
• Build and support semantic models for Power BI and analytical workloads.
• Implement data quality, validation, monitoring and error-handling mechanisms.
• Work with metadata, lineage and governance requirements using Microsoft Purview.
• Implement data security, access controls and role-based permissions across data platforms.
• Support Data Product and domain-oriented data architecture principles.
• Follow DataOps practices including CI/CD, deployment, monitoring and production support.
• Troubleshoot pipeline failures, performance issues and data quality problems.
• Optimize data pipelines, queries and storage for performance and cost efficiency.
• Work closely with Data Architects and business stakeholders to understand requirements and implement technical solutions.
• Participate in technical design discussions, code reviews and architecture reviews.
• Maintain technical documentation, data flow diagrams and pipeline documentation.
• Support production deployments, incident resolution and SLA-driven data platform operations.
• Identify opportunities for automation and AI-assisted improvements across data engineering processes.

Qualifications and Skills

• 6+ years of experience in Data Engineering, Data Integration or Data Platform development.
• Strong hands-on experience with Microsoft Fabric.
• Experience with:
• Microsoft Fabric Lakehouse
• Fabric Warehouse
• OneLake
• Fabric Data Factory / Pipelines
• Semantic Models
• Strong understanding of Lakehouse and Medallion Architecture.
• Strong SQL development and query optimization skills.
• Hands-on experience with Python and/or PySpark.
• Experience developing enterprise ETL/ELT and data integration pipelines.
• Experience with batch and incremental data processing.
• Understanding of data modelling concepts including dimensional modelling.
• Knowledge of data quality, metadata, lineage and data governance.
• Working knowledge of Microsoft Purview.
• Understanding of Data Mesh and Data Product concepts.
• Experience with CI/CD, version control, monitoring and DataOps practices.
• Understanding of cloud security, access controls and data privacy.
• Good troubleshooting and problem-solving skills.
• Strong communication skills and ability to work with distributed and client-facing teams.

Preferred Skills

• Microsoft Fabric or Azure Data certifications.
• Experience migrating workloads from Azure Synapse, SQL Server, Databricks or other data platforms to Microsoft Fabric.
• Experience implementing Medallion Architecture on Microsoft Fabric.
• Experience with Power BI and semantic modelling.
• Exposure to AI/ML, Generative AI or Agentic AI use cases on enterprise data platforms.
• Experience working with Data Products or domain-oriented data solutions.
• Experience in Energy & Utilities, Healthcare, Financial Services or Insurance.
• Experience working with US or international enterprise clients.

What We Expect

The ideal candidate should be hands-on first and capable of independently building, troubleshooting and optimizing Fabric data solutions. You should be able to explain the technical decisions behind your implementation and work effectively with architects and engineering teams to deliver production-ready solutions.

Job Description

We are looking for an experienced Data Engineer with strong expertise in Python, ETL, SQL, CI/CD, and DevOps to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Develop data processing solutions using Python.
• Write complex and optimized SQL queries, stored procedures, and data transformations.
• Build and maintain data ingestion and integration workflows.
• Implement data quality, validation, monitoring, and error-handling processes.
• Develop and maintain CI/CD pipelines for data engineering applications.
• Work with DevOps tools and practices for automated build, deployment, and infrastructure management.
• Collaborate with data analysts, data scientists, software engineers, and business teams.
• Optimize data pipelines for performance, reliability, and scalability.
• Troubleshoot production data issues and ensure timely resolution.
• Follow best practices for version control, code quality, testing, and deployment.

Mandatory Skills

• Python
• ETL
• SQL
• CI/CD
• DevOps
• Git / Version Control
• Strong problem-solving and debugging skills

About Us:

The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.

Job Summary:

We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.

As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.

Key Responsibilities:

• Design, develop, test, and maintain optimal data pipeline and ETL architectures.
• Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
• Prepare and optimize data for predictive and prescriptive modeling.
• Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
• Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
• Utilize big data tools and frameworks to optimize data acquisition and preparation.
• Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
• Develop and curate data models for analytics, dashboards, and reports.
• Conduct code reviews, maintain production-level code, and implement testing approaches.
• Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
• Drive innovation and implement efficient new approaches to data engineering tasks.

Must-Have Skills:

• Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
• 5+ years of experience working with enterprise data platforms, including building and managing data lakes.
• 3–5 years of experience designing and implementing data warehouse solutions.
• Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
• Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
• Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
• Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
• Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
• Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
• Strong problem-solving, communication, and collaboration skills.

Good-to-Have Skills:

• Experience in integrating ERP data into data lakes.
• Experience with traditional ETL tools (e.g., Talend, Pentaho).

Competencies:

• Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
• Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
• Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
• Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
• Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.

Why Join Us?

• Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
• Work on impactful projects that make a difference across industries.
• Opportunities for professional growth and continuous learning.
• Competitive salary and benefits package.

Application Details

Ready to make an impact? Apply today and become part of the QX Impact team!

Role Overview

We are looking for a GCP Data Engineer with 10+ years of experience to design, develop, and optimize scalable cloud-based data solutions. The ideal candidate will have strong hands-on expertise in GCP, BigQuery, and advanced SQL, with experience building data pipelines and working with large-scale datasets.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT processes on GCP.
• Build, optimize, and maintain data solutions using Google BigQuery.
• Develop complex SQL queries for data transformation, aggregation, and analysis.
• Design efficient data models and optimize pipelines for performance, scalability, and cost.
• Integrate data from multiple sources and ensure data quality, reliability, and availability.
• Troubleshoot pipeline and data issues and drive continuous improvement.
• Collaborate with data architects, analysts, application teams, and business stakeholders.
• Follow best practices for cloud security, data governance, testing, and documentation.

Required Skills

• 8+ years of Data Engineering experience
• Strong hands-on experience with GCP, Django, and MongoDB
• Extensive experience with BigQuery
• Advanced SQL skills
• Strong understanding of ETL/ELT and data pipeline development
• Data modeling and data warehousing experience
• Experience handling large-scale datasets and performance optimization
• Strong problem-solving and communication skills

Good to Have

• GCP services such as Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, or Cloud Functions
• Python or other data engineering languages
• Experience with data governance and security
• Agile development experience

Job Title : Senior Data Engineer – Databricks

Experience : 14 to 20 Years

Location : HSR Layout, Bangalore

Work Mode : Hybrid – 3 Days WFO

Shift : 11:30 AM – 07:30 PM IST

Positions : 2

Notice Period : Immediate Joiners Only

Interview : 1 Technical Round + 2 Client Rounds

Role Overview :

We are looking for a Senior Data Engineer to build and lead enterprise-scale data platforms for a Switzerland-based commodity client.

The role requires a strong hands-on Data Engineering professional with expertise in Databricks, PySpark, Python, SQL, and AWS, along with technical leadership and stakeholder management experience.

Must-Have Skills :

• 14 to 20 years of Data Engineering experience
• Databricks & Apache Spark / PySpark
• Python & SQL
• AWS Cloud
• Lakehouse Architecture
• ETL / ELT & Distributed Data Processing
• Batch & Streaming Pipelines
• Data Pipeline Optimization & Data Modeling
• CDC & Incremental Processing
• Git, CI/CD & Testing
• Data Quality, Monitoring & Observability
• Technical Leadership & Stakeholder Management

Key Responsibilities :

• Design and build scalable data pipelines using Databricks, PySpark, Python, SQL, and AWS.
• Own data products from design through production.
• Develop batch / streaming pipelines and reusable ETL / ELT frameworks.
• Optimize pipelines for performance, scalability, reliability, and cost.
• Design scalable data architectures and data models.
• Implement data quality, monitoring, lineage, and CI/CD practices.
• Lead technical discussions and mentor engineering teams.
• Collaborate with business stakeholders, architects, product owners, and engineering teams.
• Remain hands-on while providing technical leadership.

Ideal Candidate :

A 14 to 20 years experienced, hands-on Data Engineering leader with strong Databricks + PySpark + AWS expertise, excellent communication, stakeholder management, and experience delivering enterprise-scale data platforms.

🔴 Super Urgent : Only Bangalore-based immediate joiners.

Job Description:

Position: Lead / Senior Data Engineer

Location: Chennai

Shift: US Eastern Time ( 5:00 PM – 2:00 AM )

Experience : 8+ years

Notice Period: Immediate Joiner only

Roles and Responsibilities:

Role Overview

The Lead Data Engineer will be responsible for designing, developing, and delivering high-quality software and data solutions while leading a team of engineers. The role involves hands-on technical work, architectural decision-making, mentoring junior developers, and collaborating with cross-functional teams to ensure successful delivery of scalable data platforms and analytical solutions.

Key Responsibilities:

Lead the end-to-end design, development, and delivery of software systems, data pipelines, and components.

Define technical strategy, architecture, and best practices for development and data engineering.

Design and build optimized data pipelines using cutting-edge technologies in a cloud environment.

Construct infrastructure for efficient ETL processes from various sources and storage systems.

Architect, design, and maintain database pipeline architectures, ensuring readiness for AI/ML transformations.

Lead the implementation of algorithms and prototypes to transform raw data into useful information.

Review code for quality, scalability, and performance.

Collaborate with Product Managers, Business Managers, Designers, and QA teams to translate business requirements into technical solutions.

Develop analytical tools, programs, and reporting mechanisms.

Create data validation methods and data analysis tools.

Interpret data trends and patterns to establish operational alerts.

Conduct complex data analysis and present results effectively.

Prepare data for prescriptive and predictive modeling.

Ensure compliance with data governance and security policies.

Troubleshoot, debug, and resolve complex technical issues.

Drive continuous improvement in software and data development processes, tools, and methodologies.

Mentor and guide engineers through code reviews, technical discussions, and training.

Ensure timely delivery of projects while maintaining high engineering standards.

Continuously explore opportunities to enhance data quality and reliability.

Apply strong programming and problem-solving skills to develop scalable solutions.

Demonstrate passion for testing strategy, problem-solving, and continuous learning.

Willingness to acquire new skills and knowledge.

Possess a product/engineering mindset to drive impactful data solutions.

Experience working in distributed environments with global teams.

Stay current with emerging technologies and industry trends to propose innovative solutions.

Technical Skills and Experience Requirements

Minimum 8+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.

High proficiency in Python, Scala, and Spark for applied large-scale data processing.

Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive.

Solid understanding of batch and streaming data processing techniques.

Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.

Expert-level ability to write complex, optimized SQL queries across extensive data volumes.

Experience with RDBMS and OLAP databases such as MySQL and Snowflake.

Familiarity with Agile methodologies.

Obsession for service observability, instrumentation, monitoring, and alerting.

Knowledge or experience in architectural best practices for building data lakes.

Qualifications - Bachelor’s degree in computer science, Engineering, Information Systems, or related field.