4+ Data security Jobs in Hyderabad | Data security Job openings in Hyderabad

JOB DETAILS:

* Job Title: Lead II - Software Engineering - AWS, Apache Spark (PySpark/Scala), Apache Kafka

* Industry: Global digital transformation solutions provider

* Salary: Best in Industry

* Experience: 5-8 years

* Location: Hyderabad

Job Summary

We are seeking a skilled Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms. The role involves working with large-scale batch and real-time data processing systems, collaborating with cross-functional teams, and ensuring data reliability, security, and performance across the data lifecycle.

Key Responsibilities

ETL Pipeline Development & Optimization

• Design, develop, and maintain complex end-to-end ETL pipelines for large-scale data ingestion and processing.
• Optimize data pipelines for performance, scalability, fault tolerance, and reliability.

Big Data Processing

• Develop and optimize batch and real-time data processing solutions using Apache Spark (PySpark/Scala) and Apache Kafka.
• Ensure fault-tolerant, scalable, and high-performance data processing systems.

Cloud Infrastructure Development

• Build and manage scalable, cloud-native data infrastructure on AWS.
• Design resilient and cost-efficient data pipelines adaptable to varying data volume and formats.

Real-Time & Batch Data Integration

• Enable seamless ingestion and processing of real-time streaming and batch data sources (e.g., AWS MSK).
• Ensure consistency, data quality, and a unified view across multiple data sources and formats.

Data Analysis & Insights

• Partner with business teams and data scientists to understand data requirements.
• Perform in-depth data analysis to identify trends, patterns, and anomalies.
• Deliver high-quality datasets and present actionable insights to stakeholders.

CI/CD & Automation

• Implement and maintain CI/CD pipelines using Jenkins or similar tools.
• Automate testing, deployment, and monitoring to ensure smooth production releases.

Data Security & Compliance

• Collaborate with security teams to ensure compliance with organizational and regulatory standards (e.g., GDPR, HIPAA).
• Implement data governance practices ensuring data integrity, security, and traceability.

Troubleshooting & Performance Tuning

• Identify and resolve performance bottlenecks in data pipelines.
• Apply best practices for monitoring, tuning, and optimizing data ingestion and storage.

Collaboration & Cross-Functional Work

• Work closely with engineers, data scientists, product managers, and business stakeholders.
• Participate in agile ceremonies, sprint planning, and architectural discussions.

Skills & Qualifications

Mandatory (Must-Have) Skills

1. AWS Expertise

• Hands-on experience with AWS Big Data services such as EMR, Managed Apache Airflow, Glue, S3, DMS, MSK, and EC2.
• Strong understanding of cloud-native data architectures.

1. Big Data Technologies

• Proficiency in PySpark or Scala Spark and SQL for large-scale data transformation and analysis.
• Experience with Apache Spark and Apache Kafka in production environments.

1. Data Frameworks

• Strong knowledge of Spark DataFrames and Datasets.

1. ETL Pipeline Development

• Proven experience in building scalable and reliable ETL pipelines for both batch and real-time data processing.

1. Database Modeling & Data Warehousing

• Expertise in designing scalable data models for OLAP and OLTP systems.

1. Data Analysis & Insights

• Ability to perform complex data analysis and extract actionable business insights.
• Strong analytical and problem-solving skills with a data-driven mindset.

1. CI/CD & Automation

• Basic to intermediate experience with CI/CD pipelines using Jenkins or similar tools.
• Familiarity with automated testing and deployment workflows.

Good-to-Have (Preferred) Skills

• Knowledge of Java for data processing applications.
• Experience with NoSQL databases (e.g., DynamoDB, Cassandra, MongoDB).
• Familiarity with data governance frameworks and compliance tooling.
• Experience with monitoring and observability tools such as AWS CloudWatch, Splunk, or Dynatrace.
• Exposure to cost optimization strategies for large-scale cloud data platforms.

Skills: big data, scala spark, apache spark, ETL pipeline development

******

Notice period - 0 to 15 days only

Job stability is mandatory

Location: Hyderabad

Note: If a candidate is a short joiner, based in Hyderabad, and fits within the approved budget, we will proceed with an offer

F2F Interview: 14th Feb 2026

3 days in office, Hybrid model.

ROLES AND RESPONSIBILITIES:

You will be responsible for architecting, implementing, and optimizing Dremio-based data Lakehouse environments integrated with cloud storage, BI, and data engineering ecosystems. The role requires a strong balance of architecture design, data modeling, query optimization, and governance enablement in large-scale analytical environments.

• Design and implement Dremio lakehouse architecture on cloud (AWS/Azure/Snowflake/Databricks ecosystem).
• Define data ingestion, curation, and semantic modeling strategies to support analytics and AI workloads.
• Optimize Dremio reflections, caching, and query performance for diverse data consumption patterns.
• Collaborate with data engineering teams to integrate data sources via APIs, JDBC, Delta/Parquet, and object storage layers (S3/ADLS).
• Establish best practices for data security, lineage, and access control aligned with enterprise governance policies.
• Support self-service analytics by enabling governed data products and semantic layers.
• Develop reusable design patterns, documentation, and standards for Dremio deployment, monitoring, and scaling.
• Work closely with BI and data science teams to ensure fast, reliable, and well-modeled access to enterprise data.

IDEAL CANDIDATE:

• Bachelor’s or Master’s in Computer Science, Information Systems, or related field.
• 5+ years in data architecture and engineering, with 3+ years in Dremio or modern lakehouse platforms.
• Strong expertise in SQL optimization, data modeling, and performance tuning within Dremio or similar query engines (Presto, Trino, Athena).
• Hands-on experience with cloud storage (S3, ADLS, GCS), Parquet/Delta/Iceberg formats, and distributed query planning.
• Knowledge of data integration tools and pipelines (Airflow, DBT, Kafka, Spark, etc.).
• Familiarity with enterprise data governance, metadata management, and role-based access control (RBAC).
• Excellent problem-solving, documentation, and stakeholder communication skills.

PREFERRED:

• Experience integrating Dremio with BI tools (Tableau, Power BI, Looker) and data catalogs (Collibra, Alation, Purview).
• Exposure to Snowflake, Databricks, or BigQuery environments.
• Experience in high-tech, manufacturing, or enterprise data modernization programs.

Review Criteria

• Strong Dremio / Lakehouse Data Architect profile
• 5+ years of experience in Data Architecture / Data Engineering, with minimum 3+ years hands-on in Dremio
• Strong expertise in SQL optimization, data modeling, query performance tuning, and designing analytical schemas for large-scale systems
• Deep experience with cloud object storage (S3 / ADLS / GCS) and file formats such as Parquet, Delta, Iceberg along with distributed query planning concepts
• Hands-on experience integrating data via APIs, JDBC, Delta/Parquet, object storage, and coordinating with data engineering pipelines (Airflow, DBT, Kafka, Spark, etc.)
• Proven experience designing and implementing lakehouse architecture including ingestion, curation, semantic modeling, reflections/caching optimization, and enabling governed analytics
• Strong understanding of data governance, lineage, RBAC-based access control, and enterprise security best practices
• Excellent communication skills with ability to work closely with BI, data science, and engineering teams; strong documentation discipline
• Candidates must come from enterprise data modernization, cloud-native, or analytics-driven companies

Preferred

• Preferred (Nice-to-have) – Experience integrating Dremio with BI tools (Tableau, Power BI, Looker) or data catalogs (Collibra, Alation, Purview); familiarity with Snowflake, Databricks, or BigQuery environments

Job Specific Criteria

• CV Attachment is mandatory
• How many years of experience you have with Dremio?
• Which is your preferred job location (Mumbai / Bengaluru / Hyderabad / Gurgaon)?
• Are you okay with 3 Days WFO?
• Virtual Interview requires video to be on, are you okay with it?

Role & Responsibilities

You will be responsible for architecting, implementing, and optimizing Dremio-based data lakehouse environments integrated with cloud storage, BI, and data engineering ecosystems. The role requires a strong balance of architecture design, data modeling, query optimization, and governance enablement in large-scale analytical environments.

• Design and implement Dremio lakehouse architecture on cloud (AWS/Azure/Snowflake/Databricks ecosystem).
• Define data ingestion, curation, and semantic modeling strategies to support analytics and AI workloads.
• Optimize Dremio reflections, caching, and query performance for diverse data consumption patterns.
• Collaborate with data engineering teams to integrate data sources via APIs, JDBC, Delta/Parquet, and object storage layers (S3/ADLS).
• Establish best practices for data security, lineage, and access control aligned with enterprise governance policies.
• Support self-service analytics by enabling governed data products and semantic layers.
• Develop reusable design patterns, documentation, and standards for Dremio deployment, monitoring, and scaling.
• Work closely with BI and data science teams to ensure fast, reliable, and well-modeled access to enterprise data.

Ideal Candidate

• Bachelor’s or master’s in computer science, Information Systems, or related field.
• 5+ years in data architecture and engineering, with 3+ years in Dremio or modern lakehouse platforms.
• Strong expertise in SQL optimization, data modeling, and performance tuning within Dremio or similar query engines (Presto, Trino, Athena).
• Hands-on experience with cloud storage (S3, ADLS, GCS), Parquet/Delta/Iceberg formats, and distributed query planning.
• Knowledge of data integration tools and pipelines (Airflow, DBT, Kafka, Spark, etc.).
• Familiarity with enterprise data governance, metadata management, and role-based access control (RBAC).
• Excellent problem-solving, documentation, and stakeholder communication skills.

What You Will Do :

As a Data Governance Lead at Kanerika, you will be responsible for defining, leading, and operationalizing the data governance framework, ensuring enterprise-wide alignment and regulatory compliance.

Required Qualifications :

- 7+ years of experience in data governance and data management.

- Proficient in Microsoft Purview and Informatica data governance tools.

- Strong in metadata management, lineage mapping, classification, and security.

- Experience with ADF, REST APIs, Talend, dbt, and automation via Azure tools.

- Knowledge of GDPR, CCPA, HIPAA, SOX and related compliance needs.

- Skilled in bridging technical governance with business and compliance goals.

Tools & Technologies :

- Microsoft Purview, Collibra, Atlan, Informatica Axon, IBM IG Catalog

- Microsoft Purview capabilities :

1. Label creation & policy setup

2. Auto-labeling & DLP

3. Compliance Manager, Insider Risk, Records & Lifecycle Management

4. Unified Catalog, eDiscovery, Data Map, Audit, Compliance alerts, DSPM.

Key Responsibilities :

1. Governance Strategy & Stakeholder Alignment :

- Develop and maintain enterprise data governance strategies, policies, and standards.

- Align governance with business goals : compliance, analytics, and decision-making.

- Collaborate across business, IT, legal, and compliance teams for role alignment.

- Drive governance training, awareness, and change management programs.

2. Microsoft Purview Administration & Implementation :

- Manage Microsoft Purview accounts, collections, and RBAC aligned to org structure.

- Optimize Purview setup for large-scale environments (50TB+).

- Integrate with Azure Data Lake, Synapse, SQL DB, Power BI, Snowflake.

- Schedule scans, set classification jobs, and maintain collection hierarchies.

3. Metadata & Lineage Management :

- Design metadata repositories and maintain business glossaries and data dictionaries.

- Implement ingestion workflows via ADF, REST APIs, PowerShell, Azure Functions.

- Ensure lineage mapping (ADF ? Synapse ? Power BI) and impact analysis.

4. Data Classification & Security Governance :

- Define classification rules and sensitivity labels (PII, PCI, PHI).

- Integrate with MIP, DLP, Insider Risk Management, and Compliance Manager.

- Enforce records management, lifecycle policies, and information barriers.

5. Data Quality & Policy Management :

- Define KPIs and dashboards to monitor data quality across domains.

- Collaborate on rule design, remediation workflows, and exception handling.

- Ensure policy compliance (GDPR, HIPAA, CCPA, etc.) and risk management.

6. Business Glossary & Stewardship :

- Maintain business glossary with domain owners and stewards in Purview.

- Enforce approval workflows, standard naming, and steward responsibilities.

- Conduct metadata audits for glossary and asset documentation quality.

7. Automation & Integration :

- Automate governance processes using PowerShell, Azure Functions, Logic Apps.

- Create pipelines for ingestion, lineage, glossary updates, tagging.

- Integrate with Power BI, Azure Monitor, Synapse Link, Collibra, BigID, etc.

8. Monitoring, Auditing & Compliance :

- Set up dashboards for audit logs, compliance reporting, metadata coverage.

- Oversee data lifecycle management across its phases.

- Support internal and external audit readiness with proper documentation.