3 Big data Jobs in Kolkata | Big data Job openings in Kolkata

We are looking for a Big Data Engineer to build large-scale data processing systems.

Responsibilities

• Build batch and streaming pipelines with Spark and Kafka
• Manage data in the Hadoop ecosystem (HDFS, Hive)
• Write Spark jobs in Scala or PySpark
• Tune jobs for performance and cost

Requirements

• 2+ years of big data engineering
• Strong hands-on Spark experience
• Experience with Hadoop, Hive and Kafka

Skills and competencies:

Required:

·        Strong analytical skills in conducting sophisticated statistical analysis using bureau/vendor data, customer performance

Data and macro-economic data to solve business problems.

·        Working experience in languages PySpark & Scala to develop code to validate and implement models and codes in

Credit Risk/Banking

·        Experience with distributed systems such as Hadoop/MapReduce, Spark, streaming data processing, cloud architecture.

• Familiarity with machine learning frameworks and libraries (like scikit-learn, SparkML, tensorflow, pytorch etc.
• Experience in systems integration, web services, batch processing
• Experience in migrating codes to PySpark/Scala is big Plus
• The ability to act as liaison conveying information needs of the business to IT and data constraints to the business

applies equal conveyance regarding business strategy and IT strategy, business processes and work flow

·        Flexibility in approach and thought process

·        Attitude to learn and comprehend the periodical changes in the regulatory requirement as per FED

• Technical expertise in the area of development of Master Data Management, data extraction, transformation, and load (ETL) applications, big data using existing and emerging technology platforms and cloud architecture

• Functions as lead developer• Support System Analysis, Technical/Data design, development, unit testing, and oversee end-to-end data solution.

• Technical SME in Master Data Management application, ETL, big data and cloud technologies                                                                                               

• Collaborate with IT teams to ensure technical designs and implementations account for requirements, standards, and best practices                                                                                           

• Performance tuning of end-to-end MDM, database, ETL, Big data processes or in the source/target database endpoints as needed.                                                               

• Mentor and advise junior members of team to provide guidance.                                                  

• Perform a technical lead and solution lead role for a team of onshore and offshore developers