5+ Apache Spark Jobs in Chennai | Apache Spark Job openings in Chennai

Data Engineer- Senior

Cubera is a data company revolutionizing big data analytics and Adtech through data share value principles wherein the users entrust their data to us. We refine the art of understanding, processing, extracting, and evaluating the data that is entrusted to us. We are a gateway for brands to increase their lead efficiency as the world moves towards web3.

What are you going to do?

Design & Develop high performance and scalable solutions that meet the needs of our customers.

Closely work with the Product Management, Architects and cross functional teams.

Build and deploy large-scale systems in Java/Python.

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Create data tools for analytics and data scientist team members that assist them in building and optimizing their algorithms.

Follow best practices that can be adopted in Bigdata stack.

Use your engineering experience and technical skills to drive the features and mentor the engineers.

What are we looking for ( Competencies) :

Bachelor’s degree in computer science, computer engineering, or related technical discipline.

Overall 5 to 8 years of programming experience in Java, Python including object-oriented design.

Data handling frameworks: Should have a working knowledge of one or more data handling frameworks like- Hive, Spark, Storm, Flink, Beam, Airflow, Nifi etc.

Data Infrastructure: Should have experience in building, deploying and maintaining applications on popular cloud infrastructure like AWS, GCP etc.

Data Store: Must have expertise in one of general-purpose No-SQL data stores like Elasticsearch, MongoDB, Redis, RedShift, etc.

Strong sense of ownership, focus on quality, responsiveness, efficiency, and innovation.

Ability to work with distributed teams in a collaborative and productive manner.

Benefits:

Competitive Salary Packages and benefits.

Collaborative, lively and an upbeat work environment with young professionals.

Job Category: Development

Job Type: Full Time

Job Location: Bangalore

Location:  Chennai
Education: BE/BTech
Experience: Minimum 3+ years of experience as a Data Scientist/Data Engineer

Domain knowledge: Data cleaning, modelling, analytics, statistics, machine learning, AI

Requirements:

• To be part of Digital Manufacturing and Industrie 4.0 projects across client group of companies
• Design and develop AI//ML models to be deployed across factories
• Knowledge on Hadoop, Apache Spark, MapReduce, Scala, Python programming, SQL and NoSQL databases is required
• Should be strong in statistics, data analysis, data modelling, machine learning techniques and Neural Networks
• Prior experience in developing AI and ML models is required
• Experience with data from the Manufacturing Industry would be a plus

Roles and Responsibilities:

• Develop AI and ML models for the Manufacturing Industry with a focus on Energy, Asset Performance Optimization and Logistics
• Multitasking, good communication necessary
• Entrepreneurial attitude

• Job Location:                      Chennai
• Reporting to:                      Team Leader, Energy Management System

Should have Passion to learn and adapt new technologies, understanding,

solving/troubleshooting issues and risks, able to make informed decisions and ability to

lead the projects.

Your Qualifications

• 2-5 Years’ Experience with functional programming
• Experience with functional programming using Scala with Spark framework.
• Strong understanding of Object-oriented programming, data structures and algorithms
• Good experience in any of the cloud platforms (Azure, AWS, GCP) etc.,
• Experience with distributed (multi-tiered) systems, relational databases and NoSql storage solutions
• Desire to learn new technologies and languages
• Participation in software design, development, and code reviews
• High level of proficiency with Computer Science/Software Engineering knowledge and contribution to the technical skills growth of other team members

Your Responsibility

• Design, build and configure applications to meet business process and application requirements
• Proactively identify and communicate potential issues and concerns and recommend/implement alternative solutions as appropriate.
• Troubleshooting & Optimization of existing solution

• Must have the experience of leading teams and drive customer interactions
• Must have multiple successful deployments user stories
• Extensive hands on experience in Apache Spark along with HiveQL
• Sound knowledge in Amazon Web Services or any other Cloud environment.
• Experienced in data flow orchestration using Apache Airflow
• JSON, XML, CSV, Parquet file formats with snappy compression.
• File movements between HDFS and AWS S3
• Experience in shell scripting and scripting to automate report generation and migration of reports to AWS S3
• Worked in building a data pipeline using Pandas and Flask FrameworkGood Familiarity with Anaconda and Jupyternotebook