Big Data Engineer

Responsibilities

● Able to contribute to the gathering of functional requirements, developing technical
specifications, and test case planning
● Demonstrating technical expertise, and solving challenging programming and design
problems
● 60% hands-on coding with architecture ownership of one or more products
● Ability to articulate architectural and design options, and educate development teams and
business users
● Resolve defects/bugs during QA testing, pre-production, production, and post-release
patches
● Mentor and guide team members
● Work cross-functionally with various bidgely teams including product management, QA/QE,
various product lines, and/or business units to drive forward results

Requirements
● BS/MS in computer science or equivalent work experience
● 8-12 years’ experience designing and developing applications in Data Engineering
● Hands-on experience with Big data EcoSystems.
● Past experience with Hadoop,Hdfs,Map Reduce,YARN,AWS Cloud, EMR, S3, Spark, Cassandra,
Kafka, Zookeeper
● Expertise with any of the following Object-Oriented Languages (OOD): Java/J2EE,Scala,
Python
● Ability to lead and mentor technical team members
● Expertise with the entire Software Development Life Cycle (SDLC)
● Excellent communication skills: Demonstrated ability to explain complex technical issues to
both technical and non-technical audiences
● Expertise in the Software design/architecture process
● Expertise with unit testing & Test-Driven Development (TDD)
● Business Acumen - strategic thinking & strategy development
● Experience on Cloud or AWS is preferable
● Have a good understanding and ability to develop software, prototypes, or proofs of
concepts (POC's) for various Data Engineering requirements.
● Experience with Agile Development, SCRUM, or Extreme Programming methodologies

Job Description: Be responsible for scaling our analytics capability across all internal disciplines and guide our strategic direction in regards to analytics Organize and analyze large, diverse data sets across multiple platforms Identify key insights and leverage them to inform and influence product strategy Technical Interactions with vendor or partners in technical capacity for scope/ approach & deliverables. Develops proof of concept to prove or disprove validity of concept. Working with all parts of the business to identify analytical requirements and formalize an approach for reliable, relevant, accurate, efficientreporting on those requirements Designing and implementing advanced statistical testing for customized problem solving Deliver concise verbal and written explanations of analyses to senior management that elevate findings into strategic recommendations Desired Candidate Profile: MTech / BE / BTech / MSc in CS or Stats or Maths, Operation Research, Statistics, Econometrics or in any quantitative field Experience in using Python, R, SAS Experience in working with large data sets and big data systems (SQL, Hadoop, Hive, etc.) Keen aptitude for large-scale data analysis with a passion for identifying key insights from data Expert working knowledge in various machine learning algorithms such XGBoost, SVM Etc. We are looking candidates from the following: Experience in Unsecured Loans & SME Loans analytics (cards, installment loans) - risk based pricing analytics Experience in Differential pricing / selection analytics (retail, airlines / travel etc). Experience in Digital product companies or Digital eCommerce with Product mindset and experience Experience in Fraud / Risk from Banks, NBFC / Fintech / Credit Bureau Experience in Online media with knowledge of media, online ads & sales (agencies) - Knowledge of DMP, DFP, Adobe/Omniture tools, Cloud Experience in Consumer Durable Loans lending companies (Experience in Credit Cards, Personal Loan - optional) Experience in Tractor Loans lending companies (Experience in Farm) Experience in Recovery, Collections analytics Experience in Marketing Analytics with Digital Marketing, Market Mix modelling, Advertising Technology

Employee Strength : around 600 in all over India

1. Create and maintain optimal data pipeline architecture
2. Assemble large, complex data sets that meet business requirements
3. Identify, design, and implement internal process improvements
4. Optimize data delivery and re-design infrastructure for greater scalability
5. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS technologies
6. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
7. Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs
8. Create data tools for analytics and data scientist team members
 
Skills Required:
 
1. Working knowledge of ETL on any cloud (Azure / AWS / GCP)
2. Proficient in Python (Programming / Scripting)
3. Good understanding of any of the data warehousing concepts (Snowflake / AWS Redshift / Azure Synapse Analytics / Google Big Query / Hive)
4. In-depth understanding of principles of database structure
5.  Good understanding of any of the ETL technologies (Informatica PowerCenter / AWS Glue / Data Factory / SSIS / Spark / Matillion / Talend / Azure)
6. Proficient in SQL (query solving)
7. Knowledge in Change case Management / Version Control – (VSS / DevOps / TFS / GitHub, Bit bucket, CICD Jenkin)

Role Summary - The position holder will be responsible for supporting various aspects of organization's Analytical & BI activities. As a member of team, candidate will collaborate with a multi-disciplinary team of experts and SMT group on a wide range of problems which will give him opportunities to solve critical business problems by using Analytical & Statistical techniques. Essential/Key Responsibilities - By analyzing data/reports to identify early warning signals (unusual trends, patterns, Process gaps etc.) and proactively providing feedback in order to take corrective actions by finding continuous improvement in process (improvement in performance, reducing cost, technological improvement etc.) - Will also be responsible for creating/defining BI (Business Intelligence) and AI (Analytical Intelligence) standards for Home Credit - Being a part of BICC team, expecting high level of Business Intelligence support (Regular Reports, weekly presentations etc.) to top management - Will ensure automation & centralization of BI activities for better utilization of resources - Will be responsible for supporting data driven ADHOC's & critical requirements Qualifications/Requirements: - MBA/ M. Tech / B-Tech or Bachelors in a quantitative discipline such as Computer Science, Engineering, Mathematics, Statistics, Operations Research, Economics from premier /Tier 1 Colleges with a minimum of 3 years of experience in Analytics/ Business Intelligence - Highly numerate/ Statistical knowledge - able to work with numbers and can understand the data trend - Ability to work with both business and technical communities - Good to know, financial analysis / modeling to support the various teams on specific analysis projects. Skills/ Desired Characteristics - Able to think analytically, use a systematics and logical approach to analyze data, problems and situations. - Good Database skills with exposure to Oracle (11g) systems and tools - Highly skilled in Excel, SQL, R/Python or Power BI /Tableau or VBA - Ability to manage multiple deliverables with minimum guidance and pro-actively set up communication processes with stakeholders - Willing to working in IC (Individual Contributor) role - Excellent communication skills in English - written, verbal - Good knowledge in Project Management and Program management. Who should join us - If you are willing to face new challenges and want to apply your data knowledge for growth / future of company, then Home Credit can give you this opportunity. Home Credit can provide you platform to show your skills & suggest valuable ideas to company. - Will get opportunity to work on company level platform & will be part of BI platform of company. - Opportunity to work in a team of enthusiastic professionals.

Overall experience of 4 – 8 years of experience in DW / BI technologies.
Minimum 2 years of work experience on Snowflake and Azure storage.
Minimum 3 years of development experience in ETL Tool Experience.
Strong SQL database skills in other databases like Oracle, SQL Server, DB2 and Teradata
Good to have Hadoop and Spark experience.
Good conceptual knowledge on Data-Warehouse and various methodologies.
Working knowledge in any of the scripting like UNIX / Shell
Good Presentation and communication skills.
Should be flexible with the overlapping working hours.
Should be able to work independently and be proactive.
Good understanding of Agile development cycle.

Responsibilities:

• Exploring and visualizing data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world.
• Verifying data quality, and/or ensuring it via data cleaning.
• Able to adapt and work fast in producing the output which upgrades the decision making of stakeholders using ML.
• To design and develop Machine Learning systems and schemes. 
• To perform statistical analysis and fine-tune models using test results.
• To train and retrain ML systems and models as and when necessary. 
• To deploy ML models in production and maintain the cost of cloud infrastructure.
• To develop Machine Learning apps according to client and data scientist requirements.
• To analyze the problem-solving capabilities and use-cases of ML algorithms and rank them by how successful they are in meeting the objective.

Technical Knowledge:

• Worked with real time problems, solved them using ML and deep learning models deployed in real time and should have some awesome projects under his belt to showcase. 
• Proficiency in Python and experience with working with Jupyter Framework, Google collab and cloud hosted notebooks such as AWS sagemaker, DataBricks etc.
• Proficiency in working with libraries Sklearn, Tensorflow, Open CV2, Pyspark,  Pandas, Numpy and related libraries.
• Expert in visualising and manipulating complex datasets.
• Proficiency in working with visualisation libraries such as seaborn, plotly, matplotlib etc.
• Proficiency in Linear Algebra, statistics and probability required for Machine Learning.
• Proficiency in ML Based algorithms for example, Gradient boosting, stacked Machine learning, classification algorithms and deep learning algorithms. Need to have experience in hypertuning various models and comparing the results of algorithm performance.
• Big data Technologies such as Hadoop stack and Spark. 
• Basic use of clouds (VM’s example EC2).
• Brownie points for Kubernetes and Task Queues.      
• Strong written and verbal communications.
• Experience working in an Agile environment.

Data Architect who leads a team of 5 numbers. Required skills : Spark ,Scala , hadoop