Cutshort logo
For Employers
Smartavya logo
Hadoop Admin Lead
Smartavya
Hadoop Admin Lead

Hadoop Admin Lead at Smartavya · Mumbai · 10 - 12 years · ₹20L - ₹30L / yr · Posted 2 Jan 2025

Pluginlive's logo

Hadoop Admin Lead

at Smartavya

Agency job
10 - 12 yrs
₹20L - ₹30L / yr
Mumbai
Skills
Hadoop
Apache Hive
HDFS
Spark
Data cluster
Cloudera
Shell Scripting
Big Data
Nagios
Linux administration

Key Responsibilities:


• Install, configure, and maintain Hadoop clusters.

• Monitor cluster performance and ensure high availability.

• Manage Hadoop ecosystem components (HDFS, YARN, Ozone, Spark, Kudu, Hive).

• Perform routine cluster maintenance and troubleshooting.

• Implement and manage security and data governance.

• Monitor systems health and optimize performance.

• Collaborate with cross-functional teams to support big data applications.

• Perform Linux administration tasks and manage system configurations.

• Ensure data integrity and backup procedures.

Read more
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos

Similar jobs (10)

VY SYSTEMS PRIVATE LIMITED
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 10 yrs
₹5L - ₹20L / yr
Hadoop
PySpark

Dear Candidate,

Thanks for showing interest in the opportunity!!!

As discussed, we have an opening for ETL Developer – Hadoop & PySpark with Mphasis for the Bangalore location – Permanent.


Job Description:

We are looking for an experienced ETL Developer with strong expertise in ETL development, Hadoop, and PySpark. The candidate should have hands-on experience in data processing, building and maintaining ETL pipelines, data transformation, and handling large datasets using Big Data technologies.

Primary & Mandatory Skills:

  • ETL Development
  • Hadoop / HDFS
  • PySpark / Apache Spark
  • Data Extraction, Transformation & Loading
  • SQL and Data Processing

Good to Have:

  • Hive / Spark SQL
  • Data Pipeline Development
  • Big Data Processing
  • Python Programming

Company: Mphasis

Role: ETL Developer – Hadoop & PySpark

Experience: As per requirement

Location: Bangalore

Employment Type: Permanent

Read more
VY SYSTEMS PRIVATE LIMITED
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 12 yrs
₹5L - ₹20L / yr
Amazon EMR
skill iconPython
skill iconScala

We are hiring experienced Data Engineers for a permanent opportunity in Bangalore.

Job Description

Role: Data Engineer

Location: Bangalore

Experience: 6+ Years

Key Responsibilities:

  • Design, develop, and maintain scalable data processing pipelines using Amazon EMR.
  • Develop data engineering solutions using Python and Scala.
  • Process and transform large volumes of structured and unstructured data.
  • Build and optimize distributed data processing workflows for performance and scalability.
  • Work with big data technologies and cloud-based data processing environments.
  • Perform data integration, validation, transformation, and quality checks.
  • Troubleshoot data pipeline issues and optimize processing performance.
  • Collaborate with data architects, analysts, and development teams to deliver reliable data solutions.

Required Skills:

  • Strong hands-on experience with Amazon EMR.
  • Proficiency in Python and Scala.
  • Experience in big data processing and data pipeline development.
  • Good understanding of distributed computing and data transformation techniques.
  • Experience in performance tuning and troubleshooting data processing jobs.
  • Strong analytical, problem-solving, and communication skills.


Read more
MNC
MNC
Agency job
via VY SYSTEMS PRIVATE LIMITED by Akilandeswari Panneerselvam
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 9 yrs
₹8L - ₹15L / yr
ETL
Informatica
Hadoop
Apache Spark
Snow flake schema

Position: Backup Data ETL Developer

Experience: 6–9 Years

Location:Mumbai

Employment Type: Full-time

Job Description:

We are looking for a Backup Data ETL Developer with 6–9 years of experience in designing, developing, and supporting data extraction, transformation, and loading (ETL) processes.

The candidate should have strong hands-on experience with Hadoop, Spark, Informatica, and Snowflake, along with expertise in large-scale data processing and data integration technologies.

Key Skills:

Hadoop

Spark

Informatica

Snowflake

ETL Development

Data Integration

Data Transformation

Large-Scale Data Processing

ETL Workflows and Data Pipelines

Troubleshooting and Problem-Solving

Strong analytical and collaboration skills

Read more
VY SYSTEMS PRIVATE LIMITED
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 13 yrs
₹5L - ₹30L / yr
Amazon EMR
skill iconPython
skill iconScala

🚨 HIRING | Data Engineer – Amazon EMR 🔥


We are looking for experienced Data Engineers with strong hands-on expertise in Amazon EMR, Python, and Scala.


📌 JOB DETAILS:


💼 Experience: 6+ Years

💰 CTC: Up to 30 LPA

📍 Location: Bangalore


🔑 MANDATORY SKILLS:


• Amazon EMR

• Python

• Scala

• Data Engineering


📋 KEY REQUIREMENTS:


• 6+ years of relevant experience in Data Engineering.

• Strong hands-on experience with Amazon EMR.

• Strong programming skills in Python and Scala.

• Experience with large-scale data processing and data engineering solutions.

• Strong problem-solving and communication skills.


📩 Interested candidates can share their updated resumes:

📧 pavithra.e @vysystems. com

📞 8 9 2 5 8 0 2 4 5 7


🔄 Know someone who would be a good fit? Tag them or share this post!


#Hiring #DataEngineer #DataEngineering #AmazonEMR #Python #Scala #AWSJobs #BangaloreJobs #ITJobs #TechHiring #HiringNow

Read more
Building enterprise data, cloud, and AI solutions.
"Building enterprise data, cloud, and AI solutions."
Exclusive
job in Cutshort Lightning by Nikita Sinha
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
5 - 9 yrs
Upto ₹30L / yr (Varies
)
skill iconPython
SQL server
Snow flake schema
CI/CD
Data Warehouse (DWH)

Snowflake Administrator


What You’ll Do

  • Observability: Monitor Snowflake queries, virtual warehouses, tasks, data loads, and overall account health. Identify and resolve operational issues.
  • FinOps & Cost Optimization: Monitor credit consumption, storage, and warehouse costs. Optimize warehouse sizing, auto-suspend settings, and resource monitors.
  • User Management: Manage Snowflake users, roles, onboarding, offboarding, and Role-Based Access Control (RBAC).
  • Privilege Access Management: Review grants, privileged roles, access policies, and audit logs to enforce the principle of least privilege.
  • Platform Maintenance: Maintain Snowflake account settings, integrations, and routine operational health checks.
  • Service Desk Management: Triage incidents and service requests, investigating query failures, access issues, and platform-related problems.
  • Enhancements & Automation: Improve Snowflake configurations, automate operational tasks, and streamline support processes.
  • Business Continuity & Recovery: Support backup and retention policies, Time Travel, Fail-safe, and data recovery procedures.
  • Deployment & CI/CD: Support controlled releases of Snowflake objects, roles, scripts, and configurations through CI/CD pipelines.
  • Data Load Troubleshooting: Resolve issues related to Snowpipe, COPY commands, stages, file formats, connectivity, and scheduled data loads.
  • Transformation & Data Model Support: Perform first-line troubleshooting of failed tasks, SQL errors, and schema changes.
  • Functional Data Support: Investigate initial data availability and quality issues, coordinating with data owners for resolution.
  • Project Support: Assist engineering teams with Snowflake environments, integrations, and migrations.
  • Documentation: Maintain runbooks, access management procedures, operational guides, and knowledge base articles.

What We’re Looking For

  • 5+ years of experience in data platform administration, operations, or production support, including hands-on Snowflake experience.
  • Strong knowledge of Snowflake administration, including RBAC, virtual warehouses, query history, performance tuning, and cost optimization.
  • Hands-on experience with Snowflake data loading, stages, tasks, Snowpipe, and cloud platforms such as AWS or Azure.
  • Proficiency in SQL and familiarity with Python for scripting and automation.
  • Experience with CI/CD pipelines, Jira, ITSM processes, and monitoring or automation tools.
  • Strong troubleshooting, documentation, communication, and stakeholder management skills.


Read more
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 10 yrs
₹2L - ₹22L / yr
ETL Development
Hadoop
PySpark
skill iconPython
SQL
+3 more

Job Description

We are looking for an experienced ETL Developer with strong expertise in ETL development, Hadoop, PySpark, Python, and SQL. The candidate should have hands-on experience in developing data pipelines, processing and transforming large datasets, and working with data warehousing and big data technologies.

Key Responsibilities

  • Design, develop, and maintain ETL processes and data pipelines.
  • Work with Hadoop and PySpark to process and transform large volumes of data.
  • Develop and optimize Python scripts for data processing and automation.
  • Write complex SQL queries for data extraction, transformation, and validation.
  • Perform data cleansing, integration, transformation, and validation.
  • Work with data warehousing concepts and large-scale data processing systems.
  • Optimize ETL workflows for performance, scalability, and reliability.
  • Troubleshoot data pipeline failures and resolve data quality issues.
  • Collaborate with data engineers, analysts, and other technical teams.
  • Follow data governance, quality, and development best practices.

Required Skills

  • Strong experience in ETL Development – Mandatory
  • Hands-on experience with Hadoop – Mandatory
  • Strong experience in PySpark – Mandatory
  • Good programming skills in Python – Mandatory
  • Strong knowledge of SQL – Mandatory
  • Experience in Data Processing and Transformation – Mandatory
  • Knowledge of Data Warehousing – Mandatory
  • Experience with Big Data Technologies – Mandatory

Opportunity Details

Role: ETL Developer – Hadoop & PySpark

Experience: 6–10 Years

Location: Bangalore

Mandatory Skills

ETL Development | Hadoop | PySpark | Python | SQL | Data Processing & Transformation | Data Warehousing | Big Data Technologies

Read more
NeoGenCode Technologies Pvt Ltd
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
5 - 8 yrs
₹20L - ₹35L / yr
Data engineering
Google Cloud Platform (GCP)
Apache Spark
Google BigQuery
SQL
+8 more

Job Title : Senior Data Engineer – GCP & Apache Spark

Job Details :

  • Number of Positions : 3
  • Employment Type : Contract-to-Hire (C2H)
  • Experience Required : 5 to 8 Years
  • Location : Manyata Tech Park, Bengaluru
  • Work Mode : Hybrid – 3 Days in Office per Week
  • Preferred Notice Period : Immediate to 15–20 Days
  • Project Duration : Initially 6–8 months, with a strong possibility of extension for long-term projects
  • Relocation : Candidates from outside Bengaluru can apply if they are willing to relocate.


Role Overview :

We are looking for experienced Senior Data Engineers with strong expertise in Google Cloud Platform (GCP), Apache Spark, BigQuery, and SQL to design, develop, and maintain scalable data pipelines and distributed data processing solutions.

The ideal candidate should have hands-on experience with cloud-based data engineering, end-to-end pipeline architecture, system design, performance optimization, debugging, and CI/CD practices.


Mandatory skills :

Apache Spark (PySpark / Scala), SQL, GCP, BigQuery, GCP Dataproc, Data Pipelines, System Design, CI/CD, Debugging & Problem-Solving.


Key Responsibilities :

  • Design, develop, and maintain scalable data pipelines using Apache Spark, PySpark, and/or Scala.
  • Build and optimize batch and distributed data processing workflows on GCP.
  • Work extensively with BigQuery for data transformation, validation, analytics, and query optimization.
  • Develop and maintain data workflows using GCP services, including Dataproc, BigQuery, and Cloud Storage.
  • Design end-to-end data pipeline architectures with a focus on scalability, reliability, and performance.
  • Write and optimize complex SQL queries for large-scale datasets.
  • Troubleshoot and resolve data processing issues, pipeline failures, and performance bottlenecks.
  • Implement CI/CD practices for data engineering workflows, testing, and deployments.
  • Monitor pipeline performance and identify opportunities for continuous improvement.
  • Collaborate with cross-functional teams to understand requirements and deliver reliable data solutions.
  • Participate in code reviews, testing, production deployments, and production support.


Mandatory Skills :

  • Strong hands-on experience with Apache Spark, including PySpark and/or Scala.
  • Strong SQL skills and experience working with large-scale datasets.
  • Hands-on experience with Google Cloud Platform (GCP).
  • Practical experience with BigQuery and data warehousing concepts.
  • Working knowledge of GCP Dataproc.
  • Strong understanding of data pipelines, end-to-end workflows, and system design.
  • Practical experience with CI/CD and deployment processes.
  • Excellent debugging, troubleshooting, and problem-solving skills.


Preferred Skills :

  • Experience with Apache Airflow, Google Cloud Composer, or similar orchestration tools.
  • Knowledge of Kafka or other streaming and messaging technologies.
  • Proficiency in Python for data engineering and automation.
  • Experience with Git-based development workflows.
  • Knowledge of Linux/Unix environments and shell scripting.
  • Familiarity with monitoring, logging, data quality, and governance practices.


Required Qualifications :

  • 5 to 8 years of experience in Data Engineering, Big Data Engineering, or a related field.
  • Proven experience developing and maintaining scalable data pipelines.
  • Strong understanding of distributed data processing and cloud-based data architecture.
  • Ability to troubleshoot complex technical issues and optimize pipeline performance.
  • Strong communication, collaboration, and ownership skills.


Interview Process :

Round 1 : AI-Based Interview

  • Candidates must complete the AI interview as soon as possible after receiving the invitation.

Round 2 : Virtual Interview Drive

  • Interviews are planned for the upcoming drive; exact dates will be communicated separately.

Round 3 : Online Client Interview

  • Shortlisted candidates will attend the online interview.
  • Candidates must be available throughout the day on the scheduled interview date.
Read more
Wissen Technology
at Wissen Technology
4 recruiters
Umamaheshwaran V
Posted by Umamaheshwaran V
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Gurugram
4 - 10 yrs
Best in industry
Data engineering
skill iconPython
Spark
skill iconAmazon Web Services (AWS)
PySpark

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Python, Hadoop, and Apache Spark.
  • Develop efficient data processing and transformation workflows for large datasets.
  • Write, optimize, and maintain complex SQL queries and PL/SQL procedures.
  • Perform data extraction, transformation, and loading (ETL) from multiple data sources.
  • Work with structured and unstructured data to support analytics and reporting requirements.
  • Optimize data pipelines and queries for performance, scalability, and reliability.
  • Ensure data quality, consistency, security, and integrity across data platforms.
  • Collaborate with cross-functional teams to understand business requirements and deliver data engineering solutions.
  • Use Git, automation, and CI/CD practices to support development and deployment.

Required Skills

  • 4.5–10 years of experience in software development or data engineering.
  • Strong proficiency in AWS
  • Hands-on experience with Hadoop and Apache Spark.
  • Good knowledge of SQL and PL/SQL, including query optimization and stored procedures.
  • Experience in ETL development, data processing, and data pipeline design.
  • Understanding of data warehousing concepts and distributed computing.
  • Strong problem-solving, debugging, and performance-tuning skills.
Read more
VY SYSTEMS PRIVATE LIMITED
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore), Hyderabad
5 - 10 yrs
₹18L - ₹28L / yr
skill iconPython
PySpark
SQL

1st virtual , 2nd round F2F

Python pyspark, SQL, data engineer

5+yrs

9+yrs

Bangalore/Hyderabad

immediate to 15days.

Read more
VY SYSTEMS PRIVATE LIMITED
icon

Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.

Bengaluru (Bangalore)
6 - 10 yrs
₹5L - ₹30L / yr
ETL
ETL Development
skill iconPython
PySpark

Hiring: ETL Developer – Hadoop & PySpark

Location: Bangalore

Experience: 6–10 Years

Budget: 25–30 LPA

Work Mode: Weekdays

Job Description

We are looking for an experienced ETL Developer with strong expertise in Hadoop, PySpark, Python, and SQL to design, develop, and optimize data processing and transformation solutions. The ideal candidate should have hands-on experience working with big data technologies, building scalable ETL pipelines, and managing data for data warehousing and analytics.

Key Responsibilities

  • Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
  • Develop data transformation workflows using PySpark and Python.
  • Work with Hadoop and other big data technologies to process and manage large datasets.
  • Write complex SQL queries for data extraction, transformation, validation, and analysis.
  • Implement data cleansing, data quality checks, and data validation processes.
  • Develop and optimize data integration workflows for data warehousing solutions.
  • Troubleshoot ETL failures, performance issues, and data discrepancies.
  • Optimize PySpark jobs and SQL queries for performance and scalability.
  • Collaborate with data engineers, architects, analysts, and other stakeholders to deliver reliable data solutions.
  • Ensure data accuracy, consistency, and adherence to established data governance standards.

Mandatory Skills

  • Strong hands-on experience in ETL development.
  • Good experience with Hadoop and big data technologies.
  • Proficiency in PySpark and Python programming.
  • Strong SQL skills, including complex queries and data manipulation.
  • Experience in data processing, data transformation, and data integration.
  • Understanding of data warehousing concepts and architecture.
  • Experience in developing, debugging, and optimizing ETL pipelines.

Required Experience

  • 6–10 years of relevant experience in ETL development and data engineering.
  • Practical experience handling large datasets using Hadoop and PySpark.
  • Strong analytical, problem-solving, and troubleshooting skills.
  • Ability to work independently and collaborate effectively within a team.


Interested candidates can share their updated resumes along with current CTC, expected CTC, notice period, and total relevant experience.

Read more
Why apply to jobs via Cutshort
people_solving_puzzle
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
people_verifying_people
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly.
ai_chip
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
companies logo
companies logo
companies logo
companies logo
companies logo
Get to hear about interesting companies hiring right now
Company logo
Company logo
Company logo
Company logo
Company logo
Linkedin iconFollow Cutshort
Users love Cutshort
Read about what our users have to say about finding their next opportunity on Cutshort.
Shubham Vishwakarma's profile image

Shubham Vishwakarma

Full Stack Developer - Averlon
I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.
Companies hiring on Cutshort
companies logos