Data Engineer

Data Scientist – Program Embedded 

Job Description:   

We are seeking a highly skilled and motivated senior data scientist to support a big data program. The successful candidate will play a pivotal role in supporting multiple projects in this program covering traditional tasks from revenue management, demand forecasting, improving customer experience to testing/using new tools/platforms such as Copilot Fabric for different purpose. The expected candidate would have deep expertise in machine learning methodology and applications. And he/she should have completed multiple large scale data science projects (full cycle from ideation to BAU). Beyond technical expertise, problem solving in complex set-up will be key to the success for this role. This is a data science role directly embedded into the program/projects, stake holder management and collaborations with patterner are crucial to the success on this role (on top of the deep expertise). 

What we are looking for: 

1. Highly efficient in Python/Pyspark/R. 
2. Understand MLOps concepts, working experience in product industrialization (from Data Science point of view). Experience in building product for live deployment, and continuous development and continuous integration. 
3. Familiar with cloud platforms such as Azure, GCP, and the data management systems on such platform. Familiar with Databricks and product deployment on Databricks. 
4. Experience in ML projects involving techniques: Regression, Time Series, Clustering, Classification, Dimension Reduction, Anomaly detection with traditional ML approaches and DL approaches. 
5. Solid background in statistics, probability distributions, A/B testing validation, univariate/multivariate analysis, hypothesis test for different purpose, data augmentation etc. 
6. Familiar with designing testing framework for different modelling practice/projects based on business needs. 
7. Exposure to Gen AI tools and enthusiastic about experimenting and have new ideas on what can be done. 
8. If they have improved an internal company process using an AI tool, that would be great (e.g. process simplification, manual task automation, auto emails) 
9. Ideally, 10+ years of experience, and have been on independent business facing roles. 
10. CPG or retail as a data scientist would be nice, but not number one priority, especially for those who have navigated through multiple industries. 
11. Being proactive and collaborative would be essential. 

Some projects examples within the program: 

1. Test new tools/platforms such as Copilo, Fabric for commercial reporting. Testing, validation and build trust. 
2. Building algorithms for predicting trend in category, consumptions to support dashboards. 
3. Revenue Growth Management, create/understand the algorithms behind the tools (can be built by 3rd parties) we need to maintain or choose to improve. Able to prioritize and build product roadmap. Able to design new solutions and articulate/quantify the limitation of the solutions. 
4. Demand forecasting, create localized forecasts to improve in store availability. Proper model monitoring for early detection of potential issues in the forecast focusing particularly on improving the end user experience. 

Job Title: Data Analyst with Python

Experience: 4+ years

Location: Mumbai

Working Mode: Onsite

Primary Skills: Python, Data Analysis of RDs, FDs, Saving Accounts, Banking domain, Risk Consultant, Car Loan Model, Cross-sell model

Qualification: Any graduation

Job Description

1. With 4 to 5 years of banking analytics experience

2. Data Analyst profile with good domain understanding

3. Good with Python and worked on Liabilities (Mandatory)

4. For Liabilities Should have worked on (Saving A/c, FD,RD) (Mandatory)

5. Good with stakeholder management and requirement gathering

6. Only from banking industry

Job Responsibilities

• Design, build & test ETL processes using Python & SQL for the corporate data warehouse
• Inform, influence, support, and execute our product decisions
• Maintain advertising data integrity by working closely with R&D to organize and store data in a format that provides accurate data and allows the business to quickly identify issues.
• Evaluate and prototype new technologies in the area of data processing
• Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams
• High energy level, strong team player and good work ethic
• Data analysis, understanding of business requirements and translation into logical pipelines & processes
• Identification, analysis & resolution of production & development bugs
• Support the release process including completing & reviewing documentation
• Configure data mappings & transformations to orchestrate data integration & validation
• Provide subject matter expertise
• Document solutions, tools & processes
• Create & support test plans with hands-on testing
• Peer reviews of work developed by other data engineers within the team
• Establish good working relationships & communication channels with relevant departments

Skills and Qualifications we look for

• University degree 2.1 or higher (or equivalent) in a relevant subject. Master’s degree in any data subject will be a strong advantage.
• 4 - 6 years experience with data engineering.
• Strong coding ability and software development experience in Python.
• Strong hands-on experience with SQL and Data Processing.
• Google cloud platform (Cloud composer, Dataflow, Cloud function, Bigquery, Cloud storage, dataproc)
• Good working experience in any one of the ETL tools (Airflow would be preferable).
• Should possess strong analytical and problem solving skills.
• Good to have skills - Apache pyspark, CircleCI, Terraform
• Motivated, self-directed, able to work with ambiguity and interested in emerging technologies, agile and collaborative processes.
• Understanding & experience of agile / scrum delivery methodology

A Business Transformation Organization that partners with businesses to co–create customer-centric hyper-personalized solutions to achieve exponential growth. Invente offers platforms and services that enable businesses to provide human-free customer experience, Business Process Automation.

Location: Hyderabad (WFO)

Budget: Open

Position: Azure Data Engineer

Experience: 5+ years of commercial experience

Responsibilities

●     Design and implement Azure data solutions using ADLS Gen 2.0, Azure Data Factory, Synapse, Databricks, SQL, and Power BI

●     Build and maintain data pipelines and ETL processes to ensure efficient data ingestion and processing

●     Develop and manage data warehouses and data lakes

●     Ensure data quality, integrity, and security

●     Implement from existing use cases required by the AI and analytics teams.

●     Collaborate with other teams to integrate data solutions with other systems and applications

●     Stay up-to-date with emerging data technologies and recommend new solutions to improve our data infrastructure

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

• Proficient in R and Python
• Work experience 1+ years with at least 6 months working with Python
• Prior experience with building ML models
• Prior experience with SQL
• Knowledge of statistical techniques
• Experience with working on Spatial Data will be an added advantage

Role: Data Analytics Lead / Manager

1. Lead cross-functional projects using advanced data modelling and analysis techniques to discover insights that will guide strategic decisions and uncover optimization opportunities.
2. Build, develop and maintain data models, reporting systems, data automation systems, dashboards and performance metrics support that support key business decisions.
3. Coordinate with different teams to determine requirements for data warehousing, reporting and analytical solutions, and ensure customer satisfaction on deliverables
4. Oversee analytics projects to extract, manage, and analyse data from multiple applications, ensuring that deadlines are met.
5. Apply statistics and data modelling to gain actionable business insights and boost customer productivity and revenue.
6. Enforce company policies and procedures to ensure quality and prevent discrepancies.
7. Communicate and track key performance metrics across departments.
8. Keep abreast of industry best practices and policies.
9. Research Latest trends, analyse data, identify opportunities and incorporate changes into business strategies.
10. Manage and optimize processes for data intake, validation, mining and engineering as well as modelling, visualization and communication deliverables.
11. Examine, interpret and report results of analytical initiatives to stakeholders in leadership, technology, sales, marketing and product teams.
12. Oversee the data/report requests process: tracking requests submitted, prioritization, approval, etc.
13. Develop and implement MLDevOps, quality controls and departmental standards to ensure quality standards, organizational expectations, and regulatory requirements.
14. Anticipate future demands of initiatives related to people, technology, budget and business within your department and design/implement solutions to meet these needs.
15. Organize and drive successful completion of data insight initiatives through effective management of data analyst and effective collaboration with stakeholders.

Essential Skills

1. Working knowledge of data mining principles: predictive analytics, mapping, collecting data from multiple data systems on premises and cloud-based data sources.
2. Strong SQL skills, ability to perform effective querying involving multiple tables and subqueries.
3. Understanding of and experience using analytical concepts and statistical techniques: hypothesis development, designing tests/experiments, analyzing data, drawing conclusions, and developing actionable recommendations for business units.
4. Experience and knowledge of statistical modelling techniques: multiple regression, logistic regression, log-linear regression, variable selection, etc.

 We’re looking for someone with at least 10+ years of experience in a position monitoring, managing, transforming and drawing insights from data, and someone with at least 5 years of experience leading a data analyst team.

Experience in Azure is preferred