Data Engineer

Role - Senior Executive Analytics (BI)

Overview of job -

Our Client is the world’s largest media investment company and are a part of WPP. In fact, we are responsible for one in every three ads you see globally. We are currently looking for a Senior Executive Analytics to join us. In this role, you will be responsible for a massive opportunity to build and be a part of largest performance marketing setup APAC is committed to fostering a culture of diversity and inclusion. Our people are our strength, so we respect and nurture their individual talent and potential.

Reporting of the role - This role reports to the Director - Analytics.

3 best things about the job:

1. Responsible for data & analytics projects and developing data strategies by diving into data and extrapolating insights and providing guidance to clients

2. Build and be a part of a dynamic team

3. Being part of a global organisations with rapid growth opportunities

Responsibilities of the role:

• Design and build Visualization, Dashboard and reports for both Internal and external clients using Tableau, Power BI, Datorama or R Shiny/Python, SQL, Alteryx.

• Work with large data sets via hands-on data processing to produce structured data sets for analysis.

• Good to Have - Build Marketing-Mix and Multi-Touch attribution models using a range of tools, including free and paid.

What you will need:

• Degree in Mathematics, Statistics, Economics, Engineering, Data Science, Computer Science, or quantitative field.

• Proficiency in one or more coding languages – preferred languages: Python, R

• Proficiency in one or more Visualization Tools – Tableau, Datorama, Power BI

• Proficiency in using SQL.

• Minimum 3 years of experience in Marketing/Data Analytics or related field with hands-on experience in building Marketing-Mix and Attribution models is a plus.

1. Big data developer with 8+ years of professional IT experience with expertise in Hadoop ecosystem components in ingestion, Data modeling, querying, processing, storage, analysis, Data Integration and Implementing enterprise level systems spanning Big Data.
2. A skilled developer with strong problem solving, debugging and analytical capabilities, who actively engages in understanding customer requirements.
3. Expertise in Apache Hadoop ecosystem components like Spark, Hadoop Distributed File Systems(HDFS), HiveMapReduce, Hive, Sqoop, HBase, Zookeeper, YARN, Flume, Pig, Nifi, Scala and Oozie.
4. Hands on experience in creating real - time data streaming solutions using Apache Spark core, Spark SQL & DataFrames, Kafka, Spark streaming and Apache Storm.
5. Excellent knowledge of Hadoop architecture and daemons of Hadoop clusters, which include Name node,Data node, Resource manager, Node Manager and Job history server.
6. Worked on both Cloudera and Horton works in Hadoop Distributions. Experience in managing Hadoop clustersusing Cloudera Manager tool.
7. Well versed in installation, Configuration, Managing of Big Data and underlying infrastructure of Hadoop Cluster.
8. Hands on experience in coding MapReduce/Yarn Programs using Java, Scala and Python for analyzing Big Data.
9. Exposure to Cloudera development environment and management using Cloudera Manager.
10. Extensively worked on Spark using Scala on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL/Oracle .
11. Implemented Spark using PYTHON and utilizing Data frames and Spark SQL API for faster processing of data and handled importing data from different data sources into HDFS using Sqoop and performing transformations using Hive, MapReduce and then loading data into HDFS.
12. Used Spark Data Frames API over Cloudera platform to perform analytics on Hive data.
13. Hands on experience in MLlib from Spark which are used for predictive intelligence, customer segmentation and for smooth maintenance in Spark streaming.
14. Experience in using Flume to load log files into HDFS and Oozie for workflow design and scheduling.
15. Experience in optimizing MapReduce jobs to use HDFS efficiently by using various compression mechanisms.
16. Working on creating data pipeline for different events of ingestion, aggregation, and load consumer response data into Hive external tables in HDFS location to serve as feed for tableau dashboards.
17. Hands on experience in using Sqoop to import data into HDFS from RDBMS and vice-versa.
18. In-depth Understanding of Oozie to schedule all Hive/Sqoop/HBase jobs.
19. Hands on expertise in real time analytics with Apache Spark.
20. Experience in converting Hive/SQL queries into RDD transformations using Apache Spark, Scala and Python.
21. Extensive experience in working with different ETL tool environments like SSIS, Informatica and reporting tool environments like SQL Server Reporting Services (SSRS).
22. Experience in Microsoft cloud and setting cluster in Amazon EC2 & S3 including the automation of setting & extending the clusters in AWS Amazon cloud.
23. Extensively worked on Spark using Python on cluster for computational (analytics), installed it on top of Hadoop performed advanced analytical application by making use of Spark with Hive and SQL.
24. Strong experience and knowledge of real time data analytics using Spark Streaming, Kafka and Flume.
25. Knowledge in installation, configuration, supporting and managing Hadoop Clusters using Apache, Cloudera (CDH3, CDH4) distributions and on Amazon web services (AWS).
26. Experienced in writing Ad Hoc queries using Cloudera Impala, also used Impala analytical functions.
27. Experience in creating Data frames using PySpark and performing operation on the Data frames using Python.
28. In depth understanding/knowledge of Hadoop Architecture and various components such as HDFS and MapReduce Programming Paradigm, High Availability and YARN architecture.
29. Establishing multiple connections to different Redshift clusters (Bank Prod, Card Prod, SBBDA Cluster) and provide the access for pulling the information we need for analysis. 
30. Generated various kinds of knowledge reports using Power BI based on Business specification. 
31. Developed interactive Tableau dashboards to provide a clear understanding of industry specific KPIs using quick filters and parameters to handle them more efficiently.
32. Well Experience in projects using JIRA, Testing, Maven and Jenkins build tools.
33. Experienced in designing, built, and deploying and utilizing almost all the AWS stack (Including EC2, S3,), focusing on high-availability, fault tolerance, and auto-scaling.
34. Good experience with use-case development, with Software methodologies like Agile and Waterfall.
35. Working knowledge of Amazon's Elastic Cloud Compute( EC2 ) infrastructure for computational tasks and Simple Storage Service ( S3 ) as Storage mechanism.
36. Good working experience in importing data using Sqoop, SFTP from various sources like RDMS, Teradata, Mainframes, Oracle, Netezza to HDFS and performed transformations on it using Hive, Pig and Spark .
37. Extensive experience in Text Analytics, developing different Statistical Machine Learning solutions to various business problems and generating data visualizations using Python and R.
38. Proficient in NoSQL databases including HBase, Cassandra, MongoDB and its integration with Hadoop cluster.
39. Hands on experience in Hadoop Big data technology working on MapReduce, Pig, Hive as Analysis tool, Sqoop and Flume data import/export tools.

Work with the manager through the entire analytical and machine learning model life cycle:

⮚ Define the problem statement

⮚ Build and clean datasets

⮚ Exploratory data analysis

⮚ Feature engineering

⮚ Apply ML algorithms and assess the performance

⮚ Codify for deployment

⮚ Test and troubleshoot the code

⮚ Communicate analysis to stakeholders

Technical Skills

⮚ Proven experience in usage of Python and SQL

⮚ Excellent in programming and statistics

⮚ Working knowledge of tools and utilities - AWS, DevOps with Git, Selenium, Postman, Airflow, PySpark

Skills and requirements

• Experience analyzing complex and varied data in a commercial or academic setting.
• Desire to solve new and complex problems every day.
• Excellent ability to communicate scientific results to both technical and non-technical team members.

Desirable

• A degree in a numerically focused discipline such as, Maths, Physics, Chemistry, Engineering or Biological Sciences..
• Hands on experience on Python, Pyspark, SQL
• Hands on experience on building End to End Data Pipelines.
• Hands on Experience on Azure Data Factory, Azure Data Bricks, Data Lake - added advantage
• Hands on Experience in building data pipelines.
• Experience with Bigdata Tools, Hadoop, Hive, Sqoop, Spark, SparkSQL
• Experience with SQL or NoSQL databases for the purposes of data retrieval and management.
• Experience in data warehousing and business intelligence tools, techniques and technology, as well as experience in diving deep on data analysis or technical issues to come up with effective solutions.
• BS degree in math, statistics, computer science or equivalent technical field.
• Experience in data mining structured and unstructured data (SQL, ETL, data warehouse, Machine Learning etc.) in a business environment with large-scale, complex data sets.
• Proven ability to look at solutions in unconventional ways. Sees opportunities to innovate and can lead the way.
• Willing to learn and work on Data Science, ML, AI.

Do you have a passion for computer vision and deep learning problems? We are looking for someone who thrives on collaboration and wants to push the boundaries of what is possible today! Material Depot (materialdepot.in) is on a mission to be India’s largest tech company in the Architecture, Engineering and Construction space by democratizing the construction ecosystem and bringing stakeholders onto a common digital platform. Our engineering team is responsible for developing Computer Vision and Machine Learning tools to enable digitization across the construction ecosystem. The founding team includes people from top management consulting firms and top colleges in India (like BCG, IITB), and have worked extensively in the construction space globally and is funded by top Indian VCs.

Our team empowers Architectural and Design Businesses to effectively manage their day to day operations. We are seeking an experienced, talented Data Scientist to join our team. You’ll be bringing your talents and expertise to continue building and evolving our highly available and distributed platform.

Our solutions need complex problem solving in computer vision that require robust, efficient, well tested, and clean solutions. The ideal candidate will possess the self-motivation, curiosity, and initiative to achieve those goals. Analogously, the candidate is a lifelong learner who passionately seeks to improve themselves and the quality of their work. You will work together with similar minds in a unique team where your skills and expertise can be used to influence future user experiences that will be used by millions.

• Extensive knowledge in machine learning and deep learning techniques
• Solid background in image processing/computer vision
• Experience in building datasets for computer vision tasks
• Experience working with and creating data structures / architectures
• Proficiency in at least one major machine learning framework
• Experience visualizing data to stakeholders
• Ability to analyze and debug complex algorithms
• Good understanding and applied experience in classic 2D image processing and segmentation
• Robust semantic object detection under different lighting conditions
• Segmentation of non-rigid contours in challenging/low contrast scenarios
• Sub-pixel accurate refinement of contours and features
• Experience in image quality assessment
• Experience with in depth failure analysis of algorithms
• Highly skilled in at least one scripting language such as Python or Matlab and solid experience in C++
• Creativity and curiosity for solving highly complex problems
• Excellent communication and collaboration skills
• Mentor and support other technical team members in the organization
• Create, improve, and refine workflows and processes for delivering quality software on time and with carefully calculated debt
• Work closely with product managers, customer support representatives, and account executives to help the business move fast and efficiently through relentless automation.

• You’re part of an agile, multidisciplinary team.
• You bring your own unique skill set to the table and collaborate with others to accomplish your team’s goals.
• You prioritize your work with the team and its product owner, weighing both the business and technical value of each task.
• You experiment, test, try, fail, and learn continuously.
• You don’t do things just because they were always done that way, you bring your experience and expertise with you and help the team make the best decisions.

• Strong knowledge of and experience with the functional programming paradigm.
• Experience conducting code reviews, providing feedback to other engineers.
• Great communication skills and a proven ability to work as part of a tight-knit team.

What you will be doing:

As a part of the Global Credit Risk and Data Analytics team, this person will be responsible for carrying out analytical initiatives which will be as follows: -

• Dive into the data and identify patterns
• Development of end-to-end Credit models and credit policy for our existing credit products
• Leverage alternate data to develop best-in-class underwriting models
• Working on Big Data to develop risk analytical solutions
• Development of Fraud models and fraud rule engine
• Collaborate with various stakeholders (e.g. tech, product) to understand and design best solutions which can be implemented
• Working on cutting-edge techniques e.g. machine learning and deep learning models

Example of projects done in past:

• Lazypay Credit Risk model using CatBoost modelling technique ; end-to-end pipeline for feature engineering and model deployment in production using Python
• Fraud model development, deployment and rules for EMEA region

Basic Requirements:

• 1-3 years of work experience as a Data scientist (in Credit domain)
• 2016 or 2017 batch from a premium college (e.g B.Tech. from IITs, NITs, Economics from DSE/ISI etc)
• Strong problem solving and understand and execute complex analysis
• Experience in at least one of the languages - R/Python/SAS and SQL
• Experience in in Credit industry (Fintech/bank)
• Familiarity with the best practices of Data Science

Add-on Skills : 

• Experience in working with big data
• Solid coding practices
• Passion for building new tools/algorithms
• Experience in developing Machine Learning models