Principal Engineer - Hadoop

Job Summary:

• We are seeking an experienced Hadoop Engineer with strong hands-on expertise in MAPR and Hortonworks platform engineering and administration. The successful candidate will join our Hadoop Platform Engineering & Automation team to support our critical enterprise data lake platform. This role emphasizes platform support, patching, vulnerability remediation, L3 issue resolution, and automation initiatives utilizing Linux, Shell Scripting, and Ansible.

Responsibilities:

• Platform Engineering & Administration: Administer, maintain, and optimize MAPR and Hortonworks Hadoop clusters.
• Provide L3 support for complex technical issues related to cluster functioning, performance, and reliability.
• Monitor cluster health, capacity, performance, and security configurations.
• Patching & Vulnerability Remediation: Execute OS-level and platform-level patching across large-scale Hadoop clusters.
• Implement remediation for platform vulnerabilities in accordance with organizational InfoSec policies.
• Collaborate with other support teams to ensure compliance with standards and mitigation timelines.
• Automation Development: Develop, enhance, and maintain automation workflows using Linux, Shell Scripting, and Ansible.
• Automate recurring operational tasks including cluster patch deployment, configuration management, monitoring and ing integrations, and system health checks.
• Operational Support: Troubleshoot node failures, service crashes, cluster imbalance, and distributed computing issues.
• Perform root cause analysis for high-severity incidents.
• Ensure high availability and optimal performance of Hadoop platform services.
• Work closely with engineering teams to support consistency in deployment and configuration processes.

Mandatory Skills:

• Hands-on experience in Hadoop engineering and administration.
• Strong proficiency in MAPR and Hortonworks Administration.
• Deep understanding of Hadoop ecosystem components including HDFS, YARN, MapReduce, Hive, HBase, Spark, and Zookeeper.
• Experience with Linux system administration.
• Strong expertise in Shell Scripting and Ansible Automation.
• Experience with patching and security remediation for large-scale distributed systems.
• Understanding of configuration management, service orchestration, and cluster operations.

Preferred Skills:

• Exposure to DevOps tools such as Git.
• Experience with monitoring tools like Grafana, Prometheus, and Ambari.
• Understanding of ITIL processes including Incident, Change, and Problem management.

Qualifications:

• Strong analytical and troubleshooting skills for L3 support.
• Ability to work independently and in cross-functional teams.
• Excellent communication and documentation skills.
• Strong ownership mindset toward reliability and stability of platforms.

Location : Mumbai

Big Picture (The Opportunity) :

Are you looking for an opportunity to advance your Career? & If you are able to maintain a positive attitude even when everything goes wrong, if you are detail oriented and self motivated with a passion to learn and improve your skills and knowledge, we have a perfect job for you !

What do we want from you ? (Our Expectations) :

• Zero to 2 years experience in Linux Operating System.
• Flexible working hours - able to support occasional nights, weekends, and call-ins, able to quickly adapt to a constantly changing faced paced environment. Open to travel to sites.
• An ideal person who is excited and motivated about running and supporting a production - grade critical infrastructure and looks for opportunities to improve processes with automation.

What are you required to do ? (Your Responsibilities) :

• Proactively maintain and develop all linux infrastructure technology to maintain a 24*7*365 uptime service.
• Engineering of systems administration-related various solutions for our various SAAS products and projects as well as operational needs.
• Proactively monitoring system performance and capacity planning.
• Providing technical support to customers for Applications, Operating systems, and networking.
• Will also be the first point of contact for our clients, where installations are placed on a permanent basis, for basic troubleshooting and problem solving.
• Fault finding, analysis and logging information for reporting of performance exceptions.
• Maintain best practices on managing systems and services across all environments.

Skills & Qualification Required (Add Value) :

• Graduate - Preferred to have Bachelor‘s Degree in Engineering, Computer Science or related field.
• He / She should be familiar with the installation and configuration of Linux operating systems and setup and operation of TCP/IP networking on Linux systems also familiar with Internet concepts including SMTP, IMAP, POP, HTTP, DNS, LDAP and related protocols.
• You should possess excellent communication skills .
• Knowledge of Email concepts, Helpdesk Concept , VoIP Concept , Cloud computing will be an added advantage.