Data Engineer - AWS

Data Engineer 

Mandatory Requirements 

• Expertise in ETL , SNowFlake
• Experience in AWS ETL using AWS Glue, AWS Lambda
• Proficient in blob storage and data lake 
• Understanding of file-based ingestion best practices. 

CORE RESPONSIBILITIES

• Data ingestion from different data sources which exposes data using different technologies, such as: RDBMS, REST HTTP API, flat files, Streams, and Time series data based on various proprietary systems. Implement data ingestion and processing with the help of Big Data technologies 
• Data processing/transformation using various technologies such as Spark and Cloud Services. You will need to understand your part of business logic and implement it using the language supported by the base data platform 
• Develop automated data quality check to make sure right data enters the platform and verifying the results of the calculations 
• Develop an infrastructure to collect, transform, combine and publish/distribute customer data.
• Define process improvement opportunities to optimize data collection, insights and displays.
• Ensure data and results are accessible, scalable, efficient, accurate, complete and flexible 
• Identify and interpret trends and patterns from complex data sets 
• Construct a framework utilizing data visualization tools and techniques to present consolidated analytical and actionable results to relevant stakeholders. 
• Key participant in regular Scrum ceremonies with the agile teams  
• Proficient at developing queries, writing reports and presenting findings 
• Mentor junior members and bring best industry practices 

QUALIFICATIONS

• 5-7+ years’ experience as data engineer in consumer finance or manufacturing or Oil & Gas industry 
• Strong background in math, statistics, computer science, data science or related discipline
• Advanced knowledge one of language Python, R, C# 
• Production experience with: HDFS, YARN, Hive, Spark, Kafka, Azure, Docker / Kubernetes, SQL Server, Synapse, Snowflake,AWS
• Proficient with
  • Data mining/programming tools (e.g. SAS, SQL, R, Python)
  • Database technologies (e.g. MongoDB, PostgreSQL, Redshift, Snowflake. and Greenplum)
  • Data visualization (e.g. Tableau, PowerBI, QlikSense)
• Comfortable learning about and deploying new technologies and tools. 
• Organizational skills and the ability to handle multiple projects and priorities simultaneously and meet established deadlines. 
• Good written and oral communication skills and ability to present results to non-technical audiences 
• Knowledge of business intelligence and analytical tools, technologies and techniques.

Experience:  2 or above years of experience 

Responsibilities (but not limited to): 

• Create data staging, transformation layers
  
• Prepare model-ready-data 
  
• Create consumption layer of data/models by exposing them as service 
  
• Maintain/Monitor and ensure scalability 
  

Preferred Skills (but not limited to): 

• Strong background in handling data, writing efficient SQL, python scripts, optimizing a query, loops, designing dataflow jobs, identifying the bottlenecks in a code and optimizing them, data structures, and design 
  
• Strong background in deploying ML/Data as a service by writing APIs, monitoring, error handling, load balancing, access, and authentications 
  
• Conversant with using API developments ( like GCP APIgee, FastAPI, Spring boot ), 
  
• Have an understanding of Apache Airflow, Spark Streaming, SparkML 
  
• Familiarity with development of javascript, jquery UI, UX design while keeping in mind the optimized load balancing and other front-end aspects

Position: Data Scientist

Experience: 5+ Years

Senior Data Scientist-Job Description

The Senior Data Scientist role is a creative problem solver who utilizes statistical/mathematical principles and modelling skills to uncover new insights that will significantly and meaningfully impact business decisions and actions.  She/he applies their data science expertise in identifying, defining, and executing state-of-art techniques for academic opportunities and business objectives in collaboration with other Analytics team members. The Senior Data Scientist will execute analyses & outputs spanning test design and measurement, predictive analytics, multivariate analysis, data/text mining, pattern recognition, artificial intelligence, and machine learning.

Key Responsibilities:

• Perform the full range of data science activities including test design and measurement, predictive/advanced analytics, and data mining, and analytic dashboards.
• Extract, manipulate, analyse & interpret data from various corporate data sources developing advanced analytic solutions, deriving key observations, findings, insights, and formulating actionable recommendations.
• Generate clearly understood and intuitive data science / advanced analytics outputs.
• Provide thought leadership and recommendations on business process improvement, analytic solutions to complex problems.
• Participate in best practice sharing and communication platform for advancement of the data science discipline.
• Coach and collaborate with other data scientists and data analysts.
• Present impact, insights, outcomes & recommendations to key business partners and stakeholders.
• Comply with established Service Level Agreements to ensure timely, high quality deliverables with value-add recommendations, clearly articulated key findings and observations.

Qualification:

• Bachelor's Degree (B.A./B.S.) or Master’s Degree (M.A./M.S.) in Computer Science, Statistics, Mathematics, Machine Learning, Physics, or similar degree
• 5+ years of experience in data science in a digitally advanced industry focusing on strategic initiatives, marketing and/or operations.
• Advanced knowledge of best-in-class analytic software tools and languages: Python, SQL, R, SAS, Tableau, Excel, PowerPoint.
• Expertise in statistical methods, statistical analysis, data visualization, and data mining techniques.
• Experience in Test design, Design of Experiments, A/B Testing, Measurement Science Strong influencing skills to drive a robust testing agenda and data driven decision making for process improvements
• Strong Critical thinking skills to track down complex data and engineering issues, evaluate different algorithmic approaches, and analyse data to solve problems.
• Experience in partnering with IT, marketing operations & business operations to deploy predictive analytic solutions.
• Ability to translate/communicate complex analytical/statistical/mathematical concepts with non-technical audience.
• Strong written and verbal communications skills, as well as presentation skills.

• Use statistical methods to analyze data and generate useful business reports and insights
• Analyze Publisher and Demand side data and provide actionable insights to improve monetisation to operations team and implement the strategies
• Provide support for ad hoc data requests from the Operations teams and Management
• Use 3rd party API's, web scraping, csv report processing to build dashboards in Google Data Studio
• Provide support for Analytics Processes monitoring and troubleshooting
• Support in creating reports, dashboards and models
• Independently determine the appropriate approach for new assignments

• Inquisitive and having great problem-solving skills
• Ability to own projects and work independently once given a direction
• Experience working directly with business users to build reports, dashboards, models and solving business questions with data
• Tools Expertise - Relational Databases -SQL is a must along with Python
• Familiarity with AWS Athena, Redshift a plus

Experience

• 2-7 years

Education

• UG - B.Tech/B.E.; PG - M.Tech/ MSc, Computer Science, Statistics, Maths, Data Science/ Data Analytics

Data Scientist - Product Development

Employment Type: Full Time, Permanent

Experience: 3-5 Years as a Full Time Data Scientist

Job Description:

We are looking for an exceptional Data Scientist who is passionate about data and motivated to build large scale machine learning solutions to shine our data products. This person will be contributing to the analytics of data for insight discovery and development of machine learning pipeline to support modeling of terabytes (TB) of daily data for various use cases.

Location: Pune (Currently remote up till pandemic, later you need to relocate)

About the Organization: A funded product development company, headquarter in Singapore and offices in Australia, United States, Germany, United Kingdom and India. You will gain work experience in a global environment. Qualifications:

Candidate Profile:

• 3+ years relevant working experience
• Master / Bachelor’s in computer science or engineering
• Working knowledge of Python, Spark / Pyspark, SQL
• Experience working with large-scale data
• Experience in data manipulation, analytics, visualization, model building, model deployment
• Proficiency of various ML algorithms for supervised and unsupervised learning
• Experience working in Agile/Lean model
• Exposure to building large-scale ML models using one or more of modern tools and libraries such as AWS Sagemaker, Spark ML-Lib, Tensorflow, PyTorch, Keras, GCP ML Stack
• Exposure to MLOps tools such as MLflow, Airflow
• Exposure to modern Big Data tech such as Cassandra/Scylla, Snowflake, Kafka, Ceph, Hadoop
• Exposure to IAAS platforms such as AWS, GCP, Azure
• Experience with Java and Golang is a plus
• Experience with BI toolkit such as Superset, Tableau, Quicksight, etc is a plus

****** Looking for someone who can join immediately / within a month and carries experience with product development companies and dealt with streaming data. Experience working in a product development team is desirable. AWS experience is a must. Strong experience in Python and its related library is required.

• 6+ years of recent hands-on Java development
• Developing data pipelines in AWS or Google Cloud
• Java, Python, JavaScript programming languages
• Great understanding of designing for performance, scalability, and reliability of data intensive application
• Hadoop MapReduce, Spark, Pig. Understanding of database fundamentals and advanced SQL knowledge.
• In-depth understanding of object oriented programming concepts and design patterns
• Ability to communicate clearly to technical and non-technical audiences, verbally and in writing
• Understanding of full software development life cycle, agile development and continuous integration
• Experience in Agile methodologies including Scrum and Kanban