ETL Engineer - Data Pipeline

at DataToBiz

DP
Posted by PS Dhillon
icon
Chandigarh, Delhi, Gurugram, Noida
icon
2 - 6 yrs
icon
₹7L - ₹15L / yr
icon
Full time
Skills
ETL
Amazon Web Services (AWS)
Amazon Redshift
Python
Job Responsibilities : - Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth.
Pipelines should be optimised to handle both real time data, batch update data and historical data.
Establish scalable, efficient, automated processes for complex, large scale data analysis.
Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse.
Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques.
Participate in data pipelines health monitoring and performance optimisations as well as quality documentation.
Interact with end users/clients and translate business language into technical requirements.
Acts independently to expose and resolve problems.

Job Requirements :-
2+ years experience working in software development & data pipeline development for enterprise analytics.
2+ years of working with Python with exposure to various warehousing tools
In-depth working with any of commercial tools like AWS Glue, Ta-lend, Informatica, Data-stage, etc.
Experience with various relational databases like MySQL, MSSql, Oracle etc. is a must.
Experience with analytics and reporting tools (Tableau, Power BI, SSRS, SSAS).
Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement.
Strong verbal and written communication skills with other developers and business client.
Knowledge of Logistics and/or Transportation Domain is a plus.
Hands-on with traditional databases and ERP systems like Sybase and People-soft.
Read more

About DataToBiz

Advanced data analytics consulting company offering customer analytics, spatial & marketing analytics, supply chain analytics & computer vision solutions
Read more
Founded
2018
Type
Services
Size
20-100 employees
Stage
Bootstrapped
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Data Analyst

at Vahak

Founded 2016  •  Product  •  20-100 employees  •  Raised funding
Data Analytics
Data Analyst
MS-Excel
SQL
Tableau
Python
PowerBI
icon
Bengaluru (Bangalore)
icon
4 - 10 yrs
icon
₹10L - ₹15L / yr

Who Are We?

Vahak (https://www.vahak.in) is India’s largest & most trusted online transport marketplace & directory for road transport businesses and individual commercial vehicle (Trucks, Trailers, Containers, Hyva, LCVs) owners for online truck and load booking, transport business branding and transport business network expansion. Lorry owners can find intercity and intracity loads from all over India and connect with other businesses to find trusted transporters and best deals in the Indian logistics services market. With the Vahak app, users can book loads and lorries from a live transport marketplace with over 5 Lakh + Transporters and Lorry owners in over 10,000+ locations for daily transport requirements.

 

Vahak has raised a capital of $5 Million in a “Pre Series A” round from RTP Global along with participation from Luxor Capital and Leo Capital. The other marquee angel investors include Kunal Shah, Founder and CEO, CRED; Jitendra Gupta, Founder and CEO, Jupiter; Vidit Aatrey and Sanjeev Barnwal, Co-founders, Meesho; Mohd Farid, Co-founder, Sharechat; Amrish Rau, CEO, Pine Labs; Harsimarbir Singh, Co-founder, Pristyn Care; Rohit and Kunal Bahl, Co-founders, Snapdeal; and Ravish Naresh, Co-founder and CEO, Khatabook.


Responsibilities for Data Analyst:

  • Undertake preprocessing of structured and unstructured data
  • Propose solutions and strategies to business challenges
  • Present information using data visualization techniques
  • Identify valuable data sources and automate collection processes
  • Analyze large amounts of information to discover trends and patterns
  • Mine and analyze data from company databases to drive optimization and improvement of product development, marketing techniques and business strategies.
  • Proactively analyze data to answer key questions from stakeholders or out of self-initiated curiosity with an eye for what drives business performance.

Qualifications for Data Analyst

  • Experience using business intelligence tools (e.g. Tableau, Power BI – not mandatory)
  • Strong SQL or Excel skills with the ability to learn other analytic tools
  • Conceptual understanding of various modelling techniques, pros and cons of each technique
  • Strong problem solving skills with an emphasis on product development.
  • Programming advanced computing, Developing algorithms and predictive modeling experience
  • Experience using statistical computer languages (R, Python, SQL, etc.) to manipulate data and draw insights from large data sets.
  • Advantage - Knowledge of a variety of machine learning techniques (clustering, decision tree learning, artificial neural networks, etc.) and their real-world advantages/drawbacks.
  • Demonstrated experience applying data analysis methods to real-world data problems
Read more
Job posted by
Vahak Talent

Machine Learning Engineer

at E-Commerce Aggregator Platform

Agency job
via Qrata
Machine Learning (ML)
Natural Language Processing (NLP)
Python
Algorithms
Deep Learning
icon
Remote only
icon
4 - 8 yrs
icon
₹14L - ₹22L / yr
6+ years of applied machine learning experience with a focus on natural language processing. Some of our current projects require knowledge of natural language
generation.
o 3+ years of software engineering experience.
o Advanced knowledge of Python, with 2+ years in a production environment.
o Experience with practical applications of deep learning.
o Experience with agile, test-driven development, continuous integration, and automated testing.
o Experience with productionizing machine learning models and integrating into web- services.
o Experience with the full software development life cycle, including requirements collection, design, implementation, testing, and operational support.
o Excellent verbal and written communication, teamwork, decision making and influencing
skills.
o Hustle. Thrives in an evolving, fast paced, ambiguous work environment.
Read more
Job posted by
Blessy Fernandes

Data Analyst

at A modern ayurvedic nutrition brand

Agency job
via Jobdost
Data Analytics
Data Analyst
MS-Excel
SQL
Python
R Language
icon
Bengaluru (Bangalore)
icon
1.5 - 3 yrs
icon
₹5L - ₹5L / yr
About the role:
We are looking for a motivated data analyst with sound experience in handling web/ digital analytics, to join us as part of the Kapiva D2C Business Team. This team is primarily responsible for driving sales and customer engagement on our website. This channel has grown 5x in revenue over the last 12 months and is poised to grow another 5x over the next six. It represents a high-growth, important part of our overall e-commerce growth strategy.
The mandate here is to run an end-to-end sustainable e-commerce business, boost sales through marketing campaigns, and build a cutting edge product (website) that optimizes the customer’s journey as well as increases customer lifetime value.
The Data Analyst will support the business heads by providing data-backed insights in order to drive customer growth, retention and engagement. They will be required to set-up and manage reports, test various hypotheses and coordinate with various stakeholders on a day-to-day basis.


Job Responsibilities:
Strategy and planning:
● Work with the D2C functional leads and support analytics planning on a quarterly/ annual basis
● Identify reports and analytics needed to be conducted on a daily/ weekly/ monthly frequency
● Drive planning for hypothesis-led testing of key metrics across the customer funnel
Analytics:
● Interpret data, analyze results using statistical techniques and provide ongoing reports
● Analyze large amounts of information to discover trends and patterns
● Work with business teams to prioritize business and information needs
● Collaborate with engineering and product development teams to setup data infrastructure as needed

Reporting and communication:
● Prepare reports / presentations to present actionable insights that can drive business objectives
● Setup live dashboards reporting key cross-functional metrics
● Coordinate with various stakeholders to collect useful and required data
● Present findings to business stakeholders to drive action across the organization
● Propose solutions and strategies to business challenges

Requirements sought:
Must haves:
● Bachelor’s/ Masters in Mathematics, Economics, Computer Science, Information Management, Statistics or related field
● High proficiency in MS Excel and SQL
● Knowledge of one or more programming languages like Python/ R. Adept at queries, report writing and presenting findings
● Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy - working knowledge of statistics and statistical methods
● Ability to work in a highly dynamic environment across cross-functional teams; good at
coordinating with different departments and managing timelines
● Exceptional English written/verbal communication
● A penchant for understanding consumer traits and behavior and a keen eye to detail

Good to have:
● Hands-on experience with one or more web analytics tools like Google Analytics, Mixpanel, Kissmetrics, Heap, Adobe Analytics, etc.
● Experience in using business intelligence tools like Metabase, Tableau, Power BI is a plus
● Experience in developing predictive models and machine learning algorithms
Read more
Job posted by
Sathish Kumar

Opening for Lead Data Architect with Searce

at Searce Inc

Founded 2004  •  Products & Services  •  100-1000 employees  •  Profitable
PySpark
Data engineering
Big Data
Hadoop
Spark
Amazon Web Services (AWS)
Google Cloud Platform (GCP)
Technical Architecture
Scala
Architecture
icon
Mumbai, Pune
icon
8 - 14 yrs
icon
₹10L - ₹15L / yr
Job Responsibilities
1. Understand the business problem and translate these to data services and
engineering outcomes.
2. Expertise in working on cloud application designs, cloud approval plans, and
systems required to manage cloud storage.
3. Explore new technologies and learn new techniques to solve business problems
creatively
4. Collaborate with different teams - engineering and business, to build better data
products
5. Regularly evaluate cloud applications, hardware, and software.
6. Respond to technical issues in a professional and timely manner.
7. Identify the top cloud architecture solutions to successfully meet the strategic
needs of the company.
8. Offer guidance in infrastructure movement techniques including bulk application
transfers into the cloud.
9. Manage team and handle delivery of 2-3 projects
JD | Data Architect 24-Aug-2021
Solving for better 3 of 7

Qualifications
Is Education overrated? Yes. We believe so. But there is no way to locate you
otherwise. So we might look for at least a computer science, computer engineering,
information technology, or relevant field along with:

1. Over 4-6 years of experience in Data handling
2. Hands-on experience of any one programming language (Python, Java, Scala)
3. Understanding of SQL is must
4. Big data (Hadoop, Hive, Yarn, Sqoop)
5. MPP platforms (Spark, Presto)
6. Data-pipeline & scheduler tool (Ozzie, Airflow, Nifi)
7. Streaming engines (Kafka, Storm, Spark Streaming)
8. Any Relational database or DW experience
9. Any ETL tool experience
10. Hands-on experience in pipeline design, ETL and application development
11. Hands-on experience in cloud platforms like AWS, GCP etc.
12. Good communication skills and strong analytical skills
13. Experience in team handling and project delivery
Read more
Job posted by
Reena Bandekar

Data Scientist

at Analytics Consulting Company | REMOTE

Agency job
via Unnati
Data Science
R Programming
Python
MongoDB
SQL
Predictive analytics
Regression analysis
Deep Learning
icon
Remote, Bengaluru (Bangalore)
icon
2 - 4 yrs
icon
₹18L - ₹20L / yr
Do you want your software skills to contribute meaningfully into finding technology driven solutions for various businesses and alongside grow your career, then read on.
 
Our client provides data-based process optimization and analytics solutions to businesses worldwide. Their innovative algorithms and customized IT solutions cater to complex problems related to every field or industry, through tools that are non standard and are backed-up by extensive research. They serve startups as well as large, medium and small enterprises, a majority of their clients being industry leaders.
 
With registered offices in India, USA and UAE, their projects support various sectors and functions like logistics, IT, Retail, Ecommerce, Healthcare industry among others, across Asia, America and Europe. The founder holds a Master’s degree from IIT and a PhD in Operations Research from USA, with rich experience in Optimization and Analytics for various industries. His team of top scientists and pedagogy experts are focusing on innovative revenue generation ideas with minimum operational costs.
 
As a Data Scientist, you will apply expertise in machine-learning, data mining and statistical methods to design, prototype, and build the next-generation analytics engines and services.
 
What you will do:
  • Conducting advanced statistical analysis to provide actionable insights, identify trends, and measure performance
  • Performing data exploration, cleaning, preparation and feature engineering; in addition to executing tasks such as building a POC, validation/ AB testing
  • Collaborating with data engineers & architects to implement and deploy scalable solutions
  • Communicating results to diverse audiences with effective writing and visualizations
  • Identifying and executing on high impact projects, triage external requests, and ensure timely completion for the results to be useful
  • Providing thought leadership by researching best practices, conducting experiments, and collaborating with industry leaders

 

 

What you need to have:
  • 2-4 year experience in machine learning algorithms, predictive analytics, demand forecasting in real-world projects
  • Strong statistical background in descriptive and inferential statistics, regression, forecasting techniques.
  • Strong Programming background in Python (including packages like Tensorflow), R, D3.js , Tableau, Spark, SQL, MongoDB.
  • Preferred exposure to Optimization & Meta-heuristic algorithm and related applications
  • Background in a highly quantitative field like Data Science, Computer Science, Statistics, Applied Mathematics,Operations Research, Industrial Engineering, or similar fields.
  • Should have 2-4 years of experience in Data Science algorithm design and implementation, data analysis in different applied problems.
  • DS Mandatory skills : Python, R, SQL, Deep learning, predictive analysis, applied statistics
Read more
Job posted by
Veena Salian

Data Engineer

at AI enabled SAAS organisation

Data engineering
Data Engineer
AWS Lambda
Microservices
ETL
Python
MongoDB
MySQL
Cassandra
Docker
GitHub
Tableau
airflow
icon
Bengaluru (Bangalore)
icon
1 - 8 yrs
icon
₹5L - ₹40L / yr
Required Skills & Experience:
• 2+ years of experience in data engineering & strong understanding of data engineering principles using big data technologies
• Excellent programming skills in Python is mandatory
• Expertise in relational databases (MSSQL/MySQL/Postgres) and expertise in SQL. Exposure to NoSQL such as Cassandra. MongoDB will be a plus.
• Exposure to deploying ETL pipelines such as AirFlow, Docker containers & Lambda functions
• Experience in AWS loud services such as AWS CLI, Glue, Kinesis etc
• Experience using Tableau for data visualization is a plus
• Ability to demonstrate a portfolio of projects (GitHub, papers, etc.) is a plus
• Motivated, can-do attitude and desire to make a change is a must
• Excellent communication skills
Read more
Job posted by
Kalindi Maheshwari

Machine Learning Engineer

at CES IT

Founded 1996  •  Services  •  1000-5000 employees  •  Profitable
Machine Learning (ML)
Deep Learning
Python
Data modeling
icon
Hyderabad
icon
7 - 12 yrs
icon
₹5L - ₹15L / yr
o Critical thinking mind who likes to solve complex problems, loves programming, and cherishes to work in a fast-paced environment.
o Strong Python development skills, with 7+ yrs. experience with SQL.
o A bachelor or master’s degree in Computer Science or related areas
o 5+ years of experience in data integration and pipeline development
o Experience in Implementing Databricks Delta lake and data lake
o Expertise designing and implementing data pipelines using modern data engineering approach and tools: SQL, Python, Delta Lake, Databricks, Snowflake Spark
o Experience in working with multiple file formats (Parque, Avro, Delta Lake) & API
o experience with AWS Cloud on data integration with S3.
o Hands on Development experience with Python and/or Scala.
o Experience with SQL and NoSQL databases.
o Experience in using data modeling techniques and tools (focused on Dimensional design)
o Experience with micro-service architecture using Docker and Kubernetes
o Have experience working with one or more of the public cloud providers i.e. AWS, Azure or GCP
o Experience in effectively presenting and summarizing complex data to diverse audiences through visualizations and other means
o Excellent verbal and written communications skills and strong leadership capabilities

Skills:
ML
MOdelling
Python
SQL
Azure Data Lake, dataFactory, Databricks, Delta Lake
Read more
Job posted by
Yash Rathod

Python developer

at Gauge Data Solutions Pvt Ltd

Founded 2014  •  Products & Services  •  20-100 employees  • 
Data Science
Machine Learning (ML)
Natural Language Processing (NLP)
Computer Vision
Artificial Intelligence (AI)
Python
OOAD
Data storage
recommendation algorithm
icon
Noida
icon
0 - 4 yrs
icon
₹3L - ₹8L / yr

Essential Skills :

- Develop, enhance and maintain Python related projects, data services, platforms and processes.

- Apply and maintain data quality checks to ensure data integrity and completeness.

- Able to integrate multiple data sources and databases.

- Collaborate with cross-functional teams across, Decision Sciences, Search, Database Management. To design innovative solutions, capture requirements and drive a common future vision.

Technical Skills/Capabilities :

- Hands on experience in Python programming language.

- Understanding and proven application of Computer Science fundamentals in object oriented design, data structures, algorithm design, Regular expressions, data storage procedures, problem solving, and complexity analysis.

- Understanding of natural language processing and basic ML algorithms will be a plus.

- Good troubleshooting and debugging skills.

- Strong individual contributor, self-motivated, and a proven team player.

- Eager to learn and develop new experience and skills.

- Good communication and interpersonal skills.

About Company Profile :

Gauge Data Solutions Pvt Ltd :

- We are a leading company into Data Science, Machine learning and Artificial Intelligence.

- Within Gauge data we have a competitive environment for the Developers and Engineers.

- We at Gauge create potential solutions for the real world problems. One such example of our engineering is Casemine.

- Casemine is a legal research platform powered by Artificial Intelligence. It helps lawyers, judges and law researchers in their day to day life.

- Casemine provides exhaustive case results to its users with the use of cutting edge technologies.

- It is developed with the efforts of great engineers at Gauge Data.

- One such opportunity is now open for you. We at Gauge Data invites application for competitive, self motivated Python Developer.

Purpose of the Role :

- This position will play a central role in developing new features and enhancements for the products and services at Gauge Data.

- To know more about what we do and how we do it, feel free to read these articles:

- https://bit.ly/2YfVAsv

- https://bit.ly/2rQArJc

- You can also visit us at https://www.casemine.com/.

- For more information visit us at: - www.gaugeanalytics.com

- Join us on LinkedIn, Twitter & Facebook
Read more
Job posted by
Deeksha Dewal

Data Scientist

at liquiloans

Founded 2018  •  Products & Services  •  20-100 employees  •  Bootstrapped
Data Science
Machine Learning (ML)
Python
Data Analytics
icon
Mumbai
icon
1 - 7 yrs
icon
₹6L - ₹14L / yr
Work on the cutting edge FinTech landscape on problems of prediction and analytics.This position will work on internal and external data to draw insights into the bottomline for improving customer experience, credit decisioning and predictive maintainance of the platform.
Read more
Job posted by
Vipin Kumar

Technical Architect

at Nitor Infotech

Founded 2006  •  Products & Services  •  100-1000 employees  •  Profitable
Amazon Web Services (AWS)
Big Data
Business Intelligence (BI)
icon
Pune
icon
9 - 100 yrs
icon
₹13L - ₹25L / yr
The hunt is for a AWS BigData /DWH Architect with the ability to manage effective relationships with a wide range of stakeholders (customers & team members alike). Incumbent will demonstrate personal commitment and accountability to ensure standards are continuously sustained and improved both within the internal teams, and with partner organizations and suppliers. We at Nitor Infotech a Product Engineering Services company are always on hunt for some best talents in the IT industry & keeping with our trend of What next in IT. We are scouting for result oriented resources with passion for product, technology services, and creating great customer experiences. Someone who can take our current expertise & footprint of Nitor Infotech Inc., to an altogether different dimension & level in tune with the emerging market trends and ensure Brilliance @ Work continues to prevail in whatever we do. Nitor Infotech works with global ISVs to help them build and accelerate their product development. Nitor is able to do so because of the fact that product development is its DNA. This DNA is enriched by its 10 years of expertise, best practices and frameworks & Accelerators. Because of this ability Nitor Infotech has been able to build business relationships with product companies having revenues from $50 Million to $1 Billion. • 7-12+ years of relevant experience of working in Database, BI and Analytics space with over 0-2 yrs of architecting and designing data warehouse experience including 2 to 3 yrs in Big Data ecosystem • Experience in data warehouse design in AWS • Strong architecting, programming, design skills and proven track record of architecting and building large scale, distributed big data solutions • Professional and technical advice on Big Data concepts and technologies, in particular highlighting the business potential through real-time analysis • Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, etc. NoSQL stores like Mongodb, Cassandra, HBase etc.) • Performance tuning of Hadoop clusters and Hadoop MapReduce routines. • Evaluate and recommend Big Data technology stack for the platform • Drive significant technology initiatives end to end and across multiple layers of architecture • Should have breadth of BI knowledge which includes:  MSBI, Database design, new visualization tools like Tableau, Qlik View, Power BI  Understand internals and intricacies of Old and New DB platform which includes:  Strong RDMS DB Fundamentals either of it SQL Server/ MySQL/ Oracle  DB and DWH design  Designing Semantic Model using OLAP and Tabular model using MS and Non MS tools  No SQL DBs including Document, Graph, Search and Columnar DBs • Excellent communication skills and strong ability to build good rapport with prospect and existing customers • Be a Mentor and go to person for Jr. team members in the team Qualification & Experience: · Educational qualification: BE/ME/B.Tech/M.Tech, BCA/MCA/BCS/MCS, any other degree with relevant IT qualification.
Read more
Job posted by
Balakumar Mohan
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at DataToBiz?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort