16+ Hadoop Jobs in Chennai | Hadoop Job openings in Chennai

• KSQL
• Data Engineering spectrum (Java/Spark)
• Spark Scala / Kafka Streaming
• Confluent Kafka components
• Basic understanding of Hadoop

Location:  Chennai
Education: BE/BTech
Experience: Minimum 3+ years of experience as a Data Scientist/Data Engineer

Domain knowledge: Data cleaning, modelling, analytics, statistics, machine learning, AI

Requirements:

• To be part of Digital Manufacturing and Industrie 4.0 projects across client group of companies
• Design and develop AI//ML models to be deployed across factories
• Knowledge on Hadoop, Apache Spark, MapReduce, Scala, Python programming, SQL and NoSQL databases is required
• Should be strong in statistics, data analysis, data modelling, machine learning techniques and Neural Networks
• Prior experience in developing AI and ML models is required
• Experience with data from the Manufacturing Industry would be a plus

Roles and Responsibilities:

• Develop AI and ML models for the Manufacturing Industry with a focus on Energy, Asset Performance Optimization and Logistics
• Multitasking, good communication necessary
• Entrepreneurial attitude

• Job Location:                      Chennai
• Reporting to:                      Team Leader, Energy Management System

Client  An IT Services Major, hiring for a leading insurance player.

Position: SENIOR CONSULTANT

Job Description:

• Azure admin- senior consultant with HD Insights(Big data)

Skills and Experience

• Microsoft Azure Administrator certification
• Bigdata project experience in Azure HDInsight Stack. big data processing frameworks such as Spark, Hadoop, Hive, Kafka or Hbase.
• Preferred: Insurance or BFSI domain experience
• 5 to 5 years of experience is required.

We are looking for an outstanding Big Data Engineer with experience setting up and maintaining Data Warehouse and Data Lakes for an Organization. This role would closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Roles and Responsibilities:

• Develop and maintain scalable data pipelines and build out new integrations and processes required for optimal extraction, transformation, and loading of data from a wide variety of data sources using 'Big Data' technologies.
• Develop programs in Scala and Python as part of data cleaning and processing.
• Assemble large, complex data sets that meet functional / non-functional business requirements and fostering data-driven decision making across the organization.  
• Responsible to design and develop distributed, high volume, high velocity multi-threaded event processing systems.
• Implement processes and systems to validate data, monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
• Perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Provide high operational excellence guaranteeing high availability and platform stability.
• Closely collaborate with the Data Science team and assist the team build and deploy machine learning and deep learning models on big data analytics platforms.

Skills:

• Experience with Big Data pipeline, Big Data analytics, Data warehousing.
• Experience with SQL/No-SQL, schema design and dimensional data modeling.
• Strong understanding of Hadoop Architecture, HDFS ecosystem and eexperience with Big Data technology stack such as HBase, Hadoop, Hive, MapReduce.
• Experience in designing systems that process structured as well as unstructured data at large scale.
• Experience in AWS/Spark/Java/Scala/Python development.
• Should have Strong skills in PySpark (Python & SPARK). Ability to create, manage and manipulate Spark Dataframes. Expertise in Spark query tuning and performance optimization.
• Experience in developing efficient software code/frameworks for multiple use cases leveraging Python and big data technologies.
• Prior exposure to streaming data sources such as Kafka.
• Should have knowledge on Shell Scripting and Python scripting.
• High proficiency in database skills (e.g., Complex SQL), for data preparation, cleaning, and data wrangling/munging, with the ability to write advanced queries and create stored procedures.
• Experience with NoSQL databases such as Cassandra / MongoDB.
• Solid experience in all phases of Software Development Lifecycle - plan, design, develop, test, release, maintain and support, decommission.
• Experience with DevOps tools (GitHub, Travis CI, and JIRA) and methodologies (Lean, Agile, Scrum, Test Driven Development).
• Experience building and deploying applications on on-premise and cloud-based infrastructure.
• Having a good understanding of machine learning landscape and concepts. 

Qualifications and Experience:

Engineering and post graduate candidates, preferably in Computer Science, from premier institutions with proven work experience as a Big Data Engineer or a similar role for 3-5 years.

Certifications:

Good to have at least one of the Certifications listed here:

    AZ 900 - Azure Fundamentals

    DP 200, DP 201, DP 203, AZ 204 - Data Engineering

    AZ 400 - Devops Certification

Company Description

At Bungee Tech, we help retailers and brands meet customers everywhere and, on every occasion, they are in. We believe that accurate, high-quality data matched with compelling market insights empowers retailers and brands to keep their customers at the center of all innovation and value they are delivering. 

We provide a clear and complete omnichannel picture of their competitive landscape to retailers and brands. We collect billions of data points every day and multiple times in a day from publicly available sources. Using high-quality extraction, we uncover detailed information on products or services, which we automatically match, and then proactively track for price, promotion, and availability. Plus, anything we do not match helps to identify a new assortment opportunity.

Empowered with this unrivalled intelligence, we unlock compelling analytics and insights that once blended with verified partner data from trusted sources such as Nielsen, paints a complete, consolidated picture of the competitive landscape.

We are looking for a Big Data Engineer who will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them.

You will also be responsible for integrating them with the architecture used in the company.

We're working on the future. If you are seeking an environment where you can drive innovation, If you want to apply state-of-the-art software technologies to solve real world problems, If you want the satisfaction of providing visible benefit to end-users in an iterative fast paced environment, this is your opportunity.

Responsibilities

As an experienced member of the team, in this role, you will:

• Contribute to evolving the technical direction of analytical Systems and play a critical role their design and development

• You will research, design and code, troubleshoot and support. What you create is also what you own.

• Develop the next generation of automation tools for monitoring and measuring data quality, with associated user interfaces.

• Be able to broaden your technical skills and work in an environment that thrives on creativity, efficient execution, and product innovation.

BASIC QUALIFICATIONS

• Bachelor’s degree or higher in an analytical area such as Computer Science, Physics, Mathematics, Statistics, Engineering or similar.
• 5+ years relevant professional experience in Data Engineering and Business Intelligence
• 5+ years in with Advanced SQL (analytical functions), ETL, Data Warehousing.
• Strong knowledge of data warehousing concepts, including data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures, data modeling and performance tuning.
• Ability to effectively communicate with both business and technical teams.
• Excellent coding skills in Java, Python, C++, or equivalent object-oriented programming language
• Understanding of relational and non-relational databases and basic SQL
• Proficiency with at least one of these scripting languages: Perl / Python / Ruby / shell script

PREFERRED QUALIFICATIONS

• Experience with building data pipelines from application databases.
• Experience with AWS services - S3, Redshift, Spectrum, EMR, Glue, Athena, ELK etc.
• Experience working with Data Lakes.
• Experience providing technical leadership and mentor other engineers for the best practices on the data engineering space
• Sharp problem solving skills and ability to resolve ambiguous requirements
• Experience on working with Big Data
• Knowledge and experience on working with Hive and the Hadoop ecosystem
• Knowledge of Spark
• Experience working with Data Science teams

Position: Big Data Engineer

What You'll Do

Punchh is seeking to hire Big Data Engineer at either a senior or tech lead level. Reporting to the Director of Big Data, he/she will play a critical role in leading Punchh’s big data innovations. By leveraging prior industrial experience in big data, he/she will help create cutting-edge data and analytics products for Punchh’s business partners.

This role requires close collaborations with data, engineering, and product organizations. His/her job functions include

• Work with large data sets and implement sophisticated data pipelines with both structured and structured data.
• Collaborate with stakeholders to design scalable solutions.
• Manage and optimize our internal data pipeline that supports marketing, customer success and data science to name a few.
• A technical leader of Punchh’s big data platform that supports AI and BI products.
• Work with infra and operations team to monitor and optimize existing infrastructure 
• Occasional business travels are required.

What You'll Need

• 5+ years of experience as a Big Data engineering professional, developing scalable big data solutions.
• Advanced degree in computer science, engineering or other related fields.
• Demonstrated strength in data modeling, data warehousing and SQL.
• Extensive knowledge with cloud technologies, e.g. AWS and Azure.
• Excellent software engineering background. High familiarity with software development life cycle. Familiarity with GitHub/Airflow.
• Advanced knowledge of big data technologies, such as programming language (Python, Java), relational (Postgres, mysql), NoSQL (Mongodb), Hadoop (EMR) and streaming (Kafka, Spark).
• Strong problem solving skills with demonstrated rigor in building and maintaining a complex data pipeline.
• Exceptional communication skills and ability to articulate a complex concept with thoughtful, actionable recommendations.

Role Summary/Purpose:

We are looking for a Developer/Senior Developers to be a part of building advanced analytical platform leveraging Big Data technologies and transform the legacy systems. This role is an exciting, fast-paced, constantly changing and challenging work environment, and will play an important role in resolving and influencing high-level decisions.

Requirements:

• The candidate must be a self-starter, who can work under general guidelines in a fast-spaced environment.
• Overall minimum of 4 to 8 year of software development experience and 2 years in Data Warehousing domain knowledge
• Must have 3 years of hands-on working knowledge on Big Data technologies such as Hadoop, Hive, Hbase, Spark, Kafka, Spark Streaming, SCALA etc…
• Excellent knowledge in SQL & Linux Shell scripting
• Bachelors/Master’s/Engineering Degree from a well-reputed university.
• Strong communication, Interpersonal, Learning and organizing skills matched with the ability to manage stress, Time, and People effectively
• Proven experience in co-ordination of many dependencies and multiple demanding stakeholders in a complex, large-scale deployment environment
• Ability to manage a diverse and challenging stakeholder community
• Diverse knowledge and experience of working on Agile Deliveries and Scrum teams.

Responsibilities

• Should works as a senior developer/individual contributor based on situations
• Should be part of SCRUM discussions and to take requirements
• Adhere to SCRUM timeline and deliver accordingly
• Participate in a team environment for the design, development and implementation
• Should take L3 activities on need basis
• Prepare Unit/SIT/UAT testcase and log the results
• Co-ordinate SIT and UAT Testing. Take feedbacks and provide necessary remediation/recommendation in time.
• Quality delivery and automation should be a top priority
• Co-ordinate change and deployment in time
• Should create healthy harmony within the team
• Owns interaction points with members of core team (e.g.BA team, Testing and business team) and any other relevant stakeholders