Data Engineer - GCP

1. Minimum 2.5 years of experience as a Python Developer.
2. Minimum 2.5 years of experience in any framework like Django/Flask/Fast API
3. Minimum 2.5 years of experience in SQL/ Postgress
4. Minimum 2.5 years of experience in Git/Gitlab/Bit-Bucket
5. Minimum 2+ years of experience in deployment (CICD with Jenkins)
6. Minimum 2.5 years of experience in any cloud like AWS/GCP/Azure

TOP 3 SKILLS

Python (Language)

Spark Framework

Spark Streaming

Docker/Jenkins/ Spinakar

AWS

Hive Queries

He/She should be good coder.

Preff: - Airflow

Must have experience: -

Python

Spark framework and streaming

exposure to Machine Learning Lifecycle is mandatory.

Project:

This is searching domain project. Any searching activity which is happening on website this team create the model for the same, they create sorting/scored model for any search. This is done by the data

scientist This team is working more on the streaming side of data, the candidate would work extensively on Spark streaming and there will be a lot of work in Machine Learning.

INTERVIEW INFORMATION

3-4 rounds.

1st round based on data engineering batching experience.

2nd round based on data engineering streaming experience.

3rd round based on ML lifecycle (3rd round can be a techno-functional round based on previous

feedbacks otherwise 4th round will be a functional round if required.

About CarWale: CarWale's mission is to bring delight in car buying, we offer a bouquet of reliable tools and services to help car consumers decide on buying the right car, at the right price and from the right partner. CarWale has always strived to serve car buyers and owners in the most comprehensive and convenient way possible. We provide a platform where car buyers and owners can research, buy, sell and come together to discuss and talk about their cars.We aim to empower Indian consumers to make informed car buying and ownership decisions with exhaustive and un-biased information on cars through our expert reviews, owner reviews, detailed specifications and comparisons. We understand that a car is by and large the second-most expensive asset a consumer associates his lifestyle with! Together with CarTrade & BikeWale, we are the market leaders in the personal mobility media space.About the Team:We are a bunch of enthusiastic analysts assisting all business functions with their data needs. We deal with huge but diverse datasets to find relationships, patterns and meaningful insights. Our goal is to help drive growth across the organization by creating a data-driven culture.

We are looking for an experienced Data Scientist who likes to explore opportunities and know their way around data to build world class solutions making a real impact on the business. 

Skills / Requirements –

• 3-5 years of experience working on Data Science projects
• Experience doing statistical modelling of big data sets
• Expert in Python, R language with deep knowledge of ML packages
• Expert in fetching data from SQL
• Ability to present and explain data to management
• Knowledge of AWS would be beneficial
• Demonstrate Structural and Analytical thinking
• Ability to structure and execute data science project end to end

Education –

Bachelor’s degree in a quantitative field (Maths, Statistics, Computer Science). Masters will be preferred.

Work Experience : 0-2 years

Responsibilities :

- Design and implement mathematical models for fundamental valuation of securities. The person will need to understand latest research in quantitative finance and implement the same.

- Design, back-testing and implementation of high-frequency trading strategies on international exchanges. Work as part of the market-making team to determine the signals and trading strategies to go live with.

- Conduct performance attribution of live portfolios.

Required Skills :

- Strong candidates should have 0-2 years of work experience and successful track record in quantitative analysis preferably in the capital markets domain.

- Post-Graduate degree in statistics, finance, mathematics, engineering (Computer Science preferred) or other quantitative or computational disciplines

- Experience in using some or all of the following packages: R, MATLAB, SPSS, CART, C# .Net, Python

- Good written and oral communication skills.

- Strong experience working both independently and in a team-oriented collaborative environment.

- Entrepreneurial, self-motivated individual - high energy, high activity levels - passion for working with an innovative, small but rapidly growing company.

Must Have Skills:

• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skills

Technology Skills (Good to Have):

• Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
• Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. 
• Designing and implementing data engineering, ingestion, and transformation functions
• Azure Synapse or Azure SQL data warehouse
• Spark on Azure is available in HD insights and data bricks

Job Overview

                         We are looking for a ML Architect to help us discover the information hidden in vast amounts of data, and help us make smarter decisions to deliver even better products. Your primary focus will be in applying data mining techniques, doing statistical analysis, and building high quality prediction systems integrated with our products. They must have strong experience using variety of data mining and data analysis methods, building and implementing models, using/creating algorithm’s and creating/running simulations.  They must be comfortable working with a wide range of stakeholders and functional teams. The right candidate will have a passion for discovering solutions hidden in large data sets and working with stakeholders to improve business outcomes. Automating to identify the textual data with their properties and structure form various type of document.

Responsibilities

• Selecting features, building and optimizing classifiers using machine learning techniques
• Data mining using state-of-the-art methods
• Enhancing data collection procedures to include information that is relevant for building analytic systems
• Processing, cleansing, and verifying the integrity of data used for analysis
• Creating automated anomaly detection systems and constant tracking of its performance
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Secure and manage when needed GPU cluster resources for events
• Write comprehensive internal feedback reports and find opportunities for improvements
• Manage GPU instances/machines to increase the performance and efficiency of the ML/DL model.

Skills and Qualifications

• Strong Hands-on experience in Python Programming
• Working experience with Computer Vision models - Object Detection Model, Image Classification
• Good experience in feature extraction, feature selection techniques and transfer learning
• Working Experience in building deep learning NLP Models for text classification, image analytics-CNN,RNN,LSTM.
• Working Experience in any of the AWS/GCP cloud platforms, exposure in fetching data from various sources.
• Good experience in exploratory data analysis, data visualisation, and other data preprocessing techniques.
• Knowledge in any one of the DL frameworks like Tensorflow, Pytorch, Keras, Caffe
• Good knowledge in statistics,distribution of data and in supervised and unsupervised machine learning algorithms.
• Exposure to OpenCV Familiarity with GPUs + CUDA
• Experience with NVIDIA software for cluster management and provisioning such as nvsm, dcgm and DeepOps.
• We are looking for a candidate with 14+ years of experience, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with AWS cloud services: EC2, RDS, AWS-Sagemaker(Added advantage)
• Experience with object-oriented/object function scripting languages in any: Python, Java, C++, Scala, etc.

What you will be doing:

As a part of the Global Credit Risk and Data Analytics team, this person will be responsible for carrying out analytical initiatives which will be as follows: -

• Dive into the data and identify patterns
• Development of end-to-end Credit models and credit policy for our existing credit products
• Leverage alternate data to develop best-in-class underwriting models
• Working on Big Data to develop risk analytical solutions
• Development of Fraud models and fraud rule engine
• Collaborate with various stakeholders (e.g. tech, product) to understand and design best solutions which can be implemented
• Working on cutting-edge techniques e.g. machine learning and deep learning models

Example of projects done in past:

• Lazypay Credit Risk model using CatBoost modelling technique ; end-to-end pipeline for feature engineering and model deployment in production using Python
• Fraud model development, deployment and rules for EMEA region

Basic Requirements:

• 1-3 years of work experience as a Data scientist (in Credit domain)
• 2016 or 2017 batch from a premium college (e.g B.Tech. from IITs, NITs, Economics from DSE/ISI etc)
• Strong problem solving and understand and execute complex analysis
• Experience in at least one of the languages - R/Python/SAS and SQL
• Experience in in Credit industry (Fintech/bank)
• Familiarity with the best practices of Data Science

Add-on Skills : 

• Experience in working with big data
• Solid coding practices
• Passion for building new tools/algorithms
• Experience in developing Machine Learning models