Hadoop Admin at MNC · Bengaluru (Bangalore), Hyderabad · 7 - 15 years · ₹1L - ₹20L / yr · Posted 17 Sep 2026

Job Summary:
- We are seeking an experienced Hadoop Engineer with strong hands-on expertise in MAPR and Hortonworks platform engineering and administration. The successful candidate will join our Hadoop Platform Engineering & Automation team to support our critical enterprise data lake platform. This role emphasizes platform support, patching, vulnerability remediation, L3 issue resolution, and automation initiatives utilizing Linux, Shell Scripting, and Ansible.
Responsibilities:
- Platform Engineering & Administration: Administer, maintain, and optimize MAPR and Hortonworks Hadoop clusters.
- Provide L3 support for complex technical issues related to cluster functioning, performance, and reliability.
- Monitor cluster health, capacity, performance, and security configurations.
- Patching & Vulnerability Remediation: Execute OS-level and platform-level patching across large-scale Hadoop clusters.
- Implement remediation for platform vulnerabilities in accordance with organizational InfoSec policies.
- Collaborate with other support teams to ensure compliance with standards and mitigation timelines.
- Automation Development: Develop, enhance, and maintain automation workflows using Linux, Shell Scripting, and Ansible.
- Automate recurring operational tasks including cluster patch deployment, configuration management, monitoring and ing integrations, and system health checks.
- Operational Support: Troubleshoot node failures, service crashes, cluster imbalance, and distributed computing issues.
- Perform root cause analysis for high-severity incidents.
- Ensure high availability and optimal performance of Hadoop platform services.
- Work closely with engineering teams to support consistency in deployment and configuration processes.
Mandatory Skills:
- Hands-on experience in Hadoop engineering and administration.
- Strong proficiency in MAPR and Hortonworks Administration.
- Deep understanding of Hadoop ecosystem components including HDFS, YARN, MapReduce, Hive, HBase, Spark, and Zookeeper.
- Experience with Linux system administration.
- Strong expertise in Shell Scripting and Ansible Automation.
- Experience with patching and security remediation for large-scale distributed systems.
- Understanding of configuration management, service orchestration, and cluster operations.
Preferred Skills:
- Exposure to DevOps tools such as Git.
- Experience with monitoring tools like Grafana, Prometheus, and Ambari.
- Understanding of ITIL processes including Incident, Change, and Problem management.
Qualifications:
- Strong analytical and troubleshooting skills for L3 support.
- Ability to work independently and in cross-functional teams.
- Excellent communication and documentation skills.
- Strong ownership mindset toward reliability and stability of platforms.

Similar jobs (9)

Shift Timing: 3:00 PM to 12:00 AM
Summary of the Role
We are looking for a “Senior Data Platform Engineer” with strong hands-on experience in HBase, Amazon EMR, Hadoop, and Spark to support a production data platform modernisation and migration programme.
The role will focus on migrating and modernising distributed data platforms while maintaining application compatibility, data integrity, performance, and operational stability. The ideal candidate must have real production experience operating and migrating HBase/EMR environments, not only experience with Spark, Databricks, or ETL development.
The engineer will participate in migration planning, cluster-level troubleshooting, data validation, snapshot/restore activities, cutover, rollback, and post-migration verification.
Responsibilities
Design and execute migration and modernization activities for HBase, EMR, Hadoop, and Spark environments.
Support production migration of HBase/EMR workloads across environments.
Perform cluster-level troubleshooting and resolve performance, availability, and compatibility issues.
Plan and execute HBase snapshots, restore, and migration activities.
Validate migrated data using row/region counts, checksums, reconciliation, application behavior, and other appropriate validation methods.
Support HBase data movement using relevant migration mechanisms such as Snapshot/ExportSnapshot/HFile.
Analyze dependencies between HBase, Spark, Hadoop, applications, and supporting platform components.
Support EMR version upgrades and platform modernization.
Participate in production cutovers, maintenance windows, rollback planning, and post-cutover validation.
Troubleshoot migration issues involving data integrity, cluster configuration, application connectivity, performance, and compatibility.
Work with Backend, Cloud, Architect, DevSecOps, Security, and QA teams to resolve cross-platform dependencies.
Support data validation and reconciliation between legacy and target environments.
Document migration procedures, validation results, issues, resolutions, and operational runbooks.
Contribute to automation and repeatable processes for migration and validation activities.
Must-Have Skills
5+ years of experience in Data Engineering / Big Data Engineering.
Strong hands-on production experience with HBase.
Strong experience with Amazon EMR.
Strong experience with Apache Spark.
Strong understanding of Hadoop ecosystem.
Experience with Scala and/or Spark programming.
Experience with S3 and AWS-based data platforms.
Hands-on experience with HBase/EMR migration or modernization.
Experience with production data migration and cutover activities.
Experience validating migrated data through row/region counts, reconciliation, checksums, or equivalent validation techniques.
Experience troubleshooting distributed data platforms at the cluster level.
Strong understanding of dependencies between HBase, Spark, Hadoop, and application workloads.
Experience with production support and migration-related troubleshooting.
Good-to-Have Skills
Experience with HBase 1.x and HBase 2.x.
Experience with EMR 5.x and EMR 7.x.
Experience with HBase Snapshot and ExportSnapshot.
Experience working with HFiles and Coprocessors.
Experience with Apache Phoenix.
Experience with Hive, Presto, and/or Iceberg.
Experience with Kerberos in Hadoop/HBase/EMR environments.
Experience with KMS and encrypted data platforms.
Experience with data integrity validation using checksums.
Experience with rollback and snapshot/restore strategies.
Experience supporting large-scale enterprise data platform modernization.
We are hiring experienced Data Engineers for a permanent opportunity in Bangalore.
Job Description
Role: Data Engineer
Location: Bangalore
Experience: 6+ Years
Key Responsibilities:
- Design, develop, and maintain scalable data processing pipelines using Amazon EMR.
- Develop data engineering solutions using Python and Scala.
- Process and transform large volumes of structured and unstructured data.
- Build and optimize distributed data processing workflows for performance and scalability.
- Work with big data technologies and cloud-based data processing environments.
- Perform data integration, validation, transformation, and quality checks.
- Troubleshoot data pipeline issues and optimize processing performance.
- Collaborate with data architects, analysts, and development teams to deliver reliable data solutions.
Required Skills:
- Strong hands-on experience with Amazon EMR.
- Proficiency in Python and Scala.
- Experience in big data processing and data pipeline development.
- Good understanding of distributed computing and data transformation techniques.
- Experience in performance tuning and troubleshooting data processing jobs.
- Strong analytical, problem-solving, and communication skills.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Join our product development team at Planview as a Senior Software Engineer I and become a pivotal force on the Viz Core Team. This role offers the unique opportunity to shape and lead the development of data-processing pipelines and APIs that are at the heart of our software solutions. These solutions are designed to streamline and enhance the efficiency of software delivery, resonating deeply with software engineers who strive to build better and faster.
At Planview Viz, which is powered by the innovative “Flow Framework,” you will tackle complex data challenges and develop scalable solutions within an AWS cloud environment. Your efforts will be crucial in revolutionizing how businesses harness and interpret vast amounts of workflow data, transforming it into actionable insights that propel organizational efficiency and effectiveness.
Responsibilities (What you'll do)
- Create and refine powerful data-processing architectures that integrate seamlessly with a diverse array of external tools, enhancing the way software is delivered across industries.
- Drive operational excellence by critically analyzing problems, defining requirements, and devising robust solutions that push the boundaries of technology.
- Lead and inspire a team of talented engineers, promoting a culture of ownership, meticulous attention to quality, and proactive problem-solving.
- Stay at the cutting edge of technology by updating and expanding your team’s knowledge of cloud architectures, data processing, and advanced analytics, ensuring that you and your team remain leaders in technological innovation.
- Make a direct impact on the efficiency and effectiveness of software delivery worldwide through innovation and leadership.
Qualifications (What you'll bring)
Who We’re Looking For
The ideal candidate is a seasoned professional in cloud software development with a strong foundation in data processing and scalable software systems. You thrive in collaborative environments and are passionate about advancing cloud technology and data architecture to new heights. You are a leader who enjoys mentoring, guiding, and inspiring others.
Preferred Qualifications
Skills, Knowledge, and Expertise
- A degree in Computer Science, Engineering, or a related field.
- 6+ years of experience with a modern programming language, with a focus on back-end systems and cloud-based technologies.
- Strong capability in architecting and designing robust, scalable software systems.
- Proficiency in AWS or another popular cloud platform.
Additional Qualifications
- Experience with big-data technologies such as Spark and Apache Kafka.
- Experience with Java (version 17), Scala, or other JVM programming languages.
- Experience with tools such as Amazon Redshift and MongoDB.
- Experience with continuous integration and continuous deployment tools such as GitHub, Jenkins, Travis CI, or similar platforms.
- Proficiency in test-driven development.
- Experience with containers and orchestration tools such as Kubernetes.
- Experience working on remote or distributed teams and projects.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Very strong infra engineer to administer Linux based VMs (and few Windows VMs as well).
Must have strong knowledge and hands-on on docker, Kubernetes
Must be good at monitoring (AppDynamics, Prometheus, Grafana) and logging (Splunk)
Develop automation scripts using Shell and Python. Must have good knowledge in Ansible.
Should be good at optimize/finetune server configurations.
Should be good in troubleshoot issues and fix them.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
Primary Skills: Linux Administration, Production Support
Secondary Skills: Oracle SQL, Splunk, Grafana, AppDynamics, Cloud (OCP/AWS/Azure/GCP), Incident Management, AI/Automation.
🚨 HIRING | Data Engineer – Amazon EMR 🔥
We are looking for experienced Data Engineers with strong hands-on expertise in Amazon EMR, Python, and Scala.
📌 JOB DETAILS:
💼 Experience: 6+ Years
💰 CTC: Up to 30 LPA
📍 Location: Bangalore
🔑 MANDATORY SKILLS:
• Amazon EMR
• Python
• Scala
• Data Engineering
📋 KEY REQUIREMENTS:
• 6+ years of relevant experience in Data Engineering.
• Strong hands-on experience with Amazon EMR.
• Strong programming skills in Python and Scala.
• Experience with large-scale data processing and data engineering solutions.
• Strong problem-solving and communication skills.
📩 Interested candidates can share their updated resumes:
📧 pavithra.e @vysystems. com
📞 8 9 2 5 8 0 2 4 5 7
🔄 Know someone who would be a good fit? Tag them or share this post!
#Hiring #DataEngineer #DataEngineering #AmazonEMR #Python #Scala #AWSJobs #BangaloreJobs #ITJobs #TechHiring #HiringNow
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
We’re seeking a highly skilled, execution-focused Senior Backend Engineer with a minimum of 5 years of experience to join our team. This role demands hands-on expertise in building and scaling distributed systems, strong proficiency in Java, and deep knowledge of cloud-native infrastructure. You will be expected to design robust backend services, optimize performance across storage and caching layers, and enable seamless integrations using modern messaging and CI/CD pipelines.
You’ll be working in a high-scale, high-impact environment where reliability, speed, and efficiency are paramount. If you enjoy solving complex engineering challenges and have a passion for distributed systems, this is the right role for you.
Responsibilities—
- Design, develop, and maintain distributed backend systems at scale.
- Write high-performance, production-grade code in Java.
- Architect and optimize storage systems, ensuring efficient query performance and scalable data models.
- Implement caching strategies to reduce latency and improve system throughput.
- Build and manage services leveraging AWS cloud infrastructure.
- Develop resilient messaging pipelines using Kafka (or equivalent) for real-time data processing.
- Define and streamline CI/CD pipelines, ensuring rapid and reliable deployment cycles.
- Collaborate with product managers, frontend engineers, and DevOps to deliver end-to-end solutions.
- Monitor system performance, identify bottlenecks, and apply proactive fixes.
- Drive best practices in software engineering, testing, and code reviews.
Requirements—
- 5+ years of experience in backend engineering, with deep hands-on coding experience.
- Strong proficiency in Java and familiarity with modern frameworks.
- Proven track record in building scalable distributed systems.
- Hands-on expertise with AWS services (e.g., EC2, S3, Lambda, DynamoDB, RDS).
- Solid understanding of messaging systems like Kafka, RabbitMQ, or similar.
- Strong grasp of query performance optimization and storage system design.
- Experience with caching solutions (Redis, Memcached, etc.).
- Familiarity with CI/CD tools (Jenkins, GitHub Actions, GitLab CI, etc.).
- Excellent problem-solving skills and ability to thrive in fast-paced environments.
- Strong communication and collaboration skills, with a proactive mindset.
Benefits—
- Best in class salary: We hire only the best, and we pay accordingly.
- Proximity Talks: Meet other designers, engineers, and product geeks — and learn from experts in the field.
- Keep on learning with a world-class team: Work with the best in the field, challenge yourself constantly, and learn something new every day.
About us—
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.

Job Title : Tech Lead – Data Lake Platform
Number of Positions : 2
Experience : 7+ Years
Role Type : Technical Lead / Data Platform Lead
Domain : Data Engineering / Data Platform / AWS
About the Role :
We are looking for an experienced Tech Lead – Data Lake Platform to lead the design, development, and operations of an enterprise-scale AWS-based Data Lake Platform.
The platform will ingest data from multiple business systems, process it through structured data layers, and serve data for analytics, reporting, APIs, and operational applications.
As a Tech Lead, you will be responsible for setting the technical direction, leading data engineering teams, driving platform reliability and performance, and owning the platform's delivery, governance, and production support end to end.
Core Tech Stack :
AWS | S3 | EMR | Glue | Athena | Redshift | DMS | Lambda | RDS | IAM | Airflow | Spark/PySpark | SQL | Data Modeling | Hasura | GraphQL | DBT | PostgreSQL/Aurora | Kafka
Key Responsibilities :
- Own the overall Data Lake Platform architecture, covering data ingestion, staging, curated layers, consumption, analytics, and API/data serving.
- Lead the design and development of production-grade data pipelines using AWS Glue, EMR/Spark, Airflow, DBT, Athena, and Redshift.
- Design scalable batch and analytics pipelines with a focus on reliability, performance, data quality, and maintainability.
- Own the API/data-serving architecture from consumption data → RDS/PostgreSQL → Hasura GraphQL → Lambda/API Gateway.
- Drive improvements in platform stability, including orchestration failures, cluster sizing, pipeline SLAs, query performance, and production reliability.
- Design and implement appropriate AWS security, access control, IAM, PII handling, and data governance practices.
- Lead technical discussions, architecture decisions, code reviews, and engineering best practices.
- Mentor and guide data engineers while ensuring high-quality and scalable engineering delivery.
- Own production support, incident management, troubleshooting, and root-cause analysis (RCA) for critical data platform issues.
- Develop and maintain runbooks, operational procedures, monitoring, and incident response practices.
- Collaborate with business, product, application, and analytics teams to onboard new datasets and support reporting, API, and data consumption requirements.
- Ensure the platform meets defined availability, performance, security, data quality, and compliance requirements.
Must-Have Skills :
- 7+ years of experience in Data Engineering, Data Platform Engineering, or related areas, including experience in technical leadership or architecture.
- Strong hands-on experience with AWS Data Services, including:
- Amazon S3
- AWS EMR
- AWS Glue
- Amazon Athena
- Amazon Redshift
- AWS DMS
- AWS Lambda
- Amazon RDS
- AWS IAM
- Strong production experience with Apache Airflow.
- Strong hands-on experience with Apache Spark / PySpark.
- Strong SQL skills and experience with data modeling, including layered data architecture, data marts, and enterprise data models.
- Experience with Hasura or a similar GraphQL layer for PostgreSQL-based data/API serving.
- Strong understanding of data lake architecture and enterprise data platforms.
- Proven experience leading engineers and driving technical decisions.
- Hands-on experience with production support, troubleshooting, incident management, and RCA.
- Strong understanding of data platform performance, scalability, reliability, and SLA management.
Good-to-Have Skills :
- Experience with DBT and modern data transformation practices.
- Experience with lakehouse table formats on Amazon S3, such as Apache Iceberg or similar technologies.
- Strong knowledge of Amazon Redshift workload optimization, including :
- Distribution keys
- Sort keys
- Spectrum
- External tables
- Experience with Kafka or other streaming/data ingestion technologies.
- Experience with PostgreSQL / Amazon Aurora as a data-serving layer.
- Experience with Lambda and API Gateway for API-based data serving.
- Experience with enterprise data governance, data quality, security, and compliance.
- Experience in BFSI / Banking / Financial Services / Insurance domain.
Ideal Candidate :
The ideal candidate is a hands-on Data Platform / Data Engineering Lead who can operate at both the architecture and implementation level. You should be comfortable designing an AWS Data Lake from end to end, leading engineers, troubleshooting production issues, and working closely with business and application teams to deliver reliable data products.
Job Description:
Mandatory Skills: Linux Administration, Production Support, Database Support, and Monitoring & Observability, with exposure to Cloud and Automation.
Primary Skills: Linux Administration, Production Support
Secondary Skills: Oracle SQL, Splunk, Grafana, AppDynamics, Cloud (OCP/AWS/Azure/GCP), Incident Management, AI/Automation (Good to have)








