i

ETL Engineer - Data Pipeline

i
Posted by PS Dhillon
Apply to this job
i
Chandigarh, NCR (Delhi | Gurgaon | Noida)
i
- yrs
i
₹7L - ₹15L / yr
Skills
ETL
Amazon Web Services (AWS)
Amazon Redshift
Python
Job description
Job Responsibilities : - Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth.
Pipelines should be optimised to handle both real time data, batch update data and historical data.
Establish scalable, efficient, automated processes for complex, large scale data analysis.
Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse.
Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques.
Participate in data pipelines health monitoring and performance optimisations as well as quality documentation.
Interact with end users/clients and translate business language into technical requirements.
Acts independently to expose and resolve problems.

Job Requirements :-
2+ years experience working in software development & data pipeline development for enterprise analytics.
2+ years of working with Python with exposure to various warehousing tools
In-depth working with any of commercial tools like AWS Glue, Ta-lend, Informatica, Data-stage, etc.
Experience with various relational databases like MySQL, MSSql, Oracle etc. is a must.
Experience with analytics and reporting tools (Tableau, Power BI, SSRS, SSAS).
Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement.
Strong verbal and written communication skills with other developers and business client.
Knowledge of Logistics and/or Transportation Domain is a plus.
Hands-on with traditional databases and ERP systems like Sybase and People-soft.
About DataToBiz
Advanced data analytics consulting company offering customer analytics, spatial & marketing analytics, supply chain analytics & computer vision solutions
Founded
2018
Type
Services
Size
20-100 employees
Stage
Bootstrapped
Why apply to jobs via CutShort
i
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
i
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
i
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
6212
Companies hiring
Similar jobs
i
at service-based company
Agency job
Python
Hadoop
Machine Learning (ML)
Data mining
Statistical Analysis
Algorithms
i
Bengaluru (Bangalore)
i
5 - 10 yrs
i
₹1L - ₹15L / yr
Postion : Data Scientist
Location: Bangalore
Notice: 1 week to max 1-month

Experience : 5-10yrs
Key Skills: ML, python, Hadoop
ML+ Python+ Hadoop Querying
Proven record of performing data mining / building machine learning solutions.
Knowledge in statistics and ML algorithms and their usage
Read more
Job posted by
i
Jyotsna Singh
Apply for job
i
at Provider of Digital Retail Banking Services.
Agency job
via Qrata
Data Science
Python
Scala
Java
Artificial Intelligence (AI)
Machine Learning (ML)
Software Development
i
Bengaluru (Bangalore)
i
3 - 7 yrs
i
₹30L - ₹35L / yr
We are building a new age personalized mobile-first, AI-based digital bank. We are a fast-paced tech startup that relentlessly innovates each day to make banking more accessible and transparent for our users.To help us grow, we are looking at awesome team-players to join us and contribute towards building a community first digital bank.Join us if you wish to build the bank of the future!

Responsibilities

    • Participate, Own & Influence in architecting & designing of systems
    • Collaborate with other engineers, data scientists, product managers
    • Build intelligent systems that drive decisions
    • Build systems that enable us to perform experiments and iterate quickly
    • Build platforms that enable scientists to train, deploy and monitor models at scale
    • Build analytical systems that drives better decision making

Required Skills

    • Engineer with strong computer science fundamentals
    • Excellent problem solving skills
    • Strong programming Skills with strong focus on re-usability and low operational overhead
    • Excellent judgement on architecture and design
    • Strong work ethics like sense of collaboration and ownership, result orientation, being a team player
    • Ability to excel in a fast-paced, startup-like environment.
    • 3 to 7 year experienced in building systems with high quality
    • BTech/MTech in Computer Science or equivalent fields, with strong programming skills in Java/Scala/Python
Read more
Job posted by
i
Blessy Fernandes
Apply for job
i
Founded 2016  •  Products & Services  •  20-100 employees  •  Bootstrapped
Data Science
Python
Machine Learning (ML)
Amazon Web Services (AWS)
Mathematics
Algorithms
i
Remote only
i
5 - 20 yrs
i
₹10L - ₹30L / yr

Introduction

The Biostrap platform extracts many metrics related to health, sleep, and activity.  Many algorithms are designed through research and often based on scientific literature, and in some cases they are augmented with or entirely designed using machine learning techniques.  Biostrap is seeking a Data Scientist to design, develop, and implement algorithms to improve existing metrics and measure new ones. 

Job Description

As a Data Scientist at Biostrap, you will take on projects to improve or develop algorithms to measure health metrics, including:

  • Research: search literature for starting points of the algorithm
  • Design: decide on the general idea of the algorithm, in particular whether to use machine learning, mathematical techniques, or something else.
  • Implement: program the algorithm in Python, and help deploy it.  

The algorithms and their implementation will have to be accurate, efficient, and well-documented.

Requirements

  • A Master’s degree in a computational field, with a strong mathematical background. 
  • Strong knowledge of, and experience with, different machine learning techniques, including their theoretical background.  
  • Strong experience with Python
  • Experience with Keras/TensorFlow, and preferably also with RNNs
  • Experience with AWS or similar services for data pipelining and machine learning.  
  • Ability and drive to work independently on an open problem.
  • Fluency in English.
Read more
Job posted by
i
Reinier van Mourik
Apply for job
i
at Product based Company
Agency job
ETL
i
Coimbatore
i
4 - 15 yrs
i
₹5L - ₹25L / yr
Hi Professionals,
We are looking for ETL Developer for Reputed Client @ Coimbatore Permanent role
Work Location : Coimbatore
Experience : 4+ Years
Skills ;
  •  Talend (or)Strong experience in any of the ETL Tools like (Informatica/Datastage/Talend)
  • DB preference (Teradata /Oracle /Sql server )
  • Supporting Tools (JIRA/SVN)
Notice Period : Immediate to 30 Days
Read more
Job posted by
i
Gowtham V
Apply for job
i
Founded 2004  •  Products & Services  •  100-1000 employees  •  Profitable
Data architecture
Big Data
Hadoop
Google Cloud Platform (GCP)
Python
Java
Scala
Data engineering
i
Mumbai
i
5 - 9 yrs
i
₹15L - ₹22L / yr
JD of Data Architect
As a Data Architect, you work with business leads, analysts and data scientists to understand the business domain and manage data engineers to build data products that empower better decision making. You are passionate about data quality of our business metrics and flexibility of your solution that scales to respond to broader business questions.
If you love to solve problems using your skills, then come join the Team Searce. We have a
casual and fun office environment that actively steers clear of rigid "corporate" culture, focuses on productivity and creativity, and allows you to be part of a world-class team while still being yourself.

What You’ll Do
● Understand the business problem and translate these to data services and engineering
outcomes
● Explore new technologies and learn new techniques to solve business problems
creatively
● Collaborate with many teams - engineering and business, to build better data products
● Manage team and handle delivery of 2-3 projects

What We’re Looking For
● Over 4-6 years of experience with
○ Hands-on experience of any one programming language (Python, Java, Scala)
○ Understanding of SQL is must
○ Big data (Hadoop, Hive, Yarn, Sqoop)
○ MPP platforms (Spark, Presto)
○ Data-pipeline & scheduler tool (Ozzie, Airflow, Nifi)
○ Streaming engines (Kafka, Storm, Spark Streaming)
○ Any Relational database or DW experience
○ Any ETL tool experience
● Hands-on experience in pipeline design, ETL and application development
● Hands-on experience in cloud platforms like AWS, GCP etc.
● Good communication skills and strong analytical skills
● Experience in team handling and project delivery
Read more
Job posted by
i
Reena Bandekar
Apply for job
i
Founded 1998  •  Product  •  5000+ employees  •  Raised funding
Big Data
Hadoop
Apache Hive
Spark
Amazon Web Services (AWS)
PySpark
Scala
MapReduce
MySQL
Data engineering
i
NCR (Delhi | Gurgaon | Noida)
i
5 - 9 yrs
i
₹18L - ₹40L / yr
Overview:
This position is for a Big Data Engineer specialized in Hadoop and Spark technologies.
Roles & Responsibility:
  • For this role, we require someone with strong product design sense. The position requires
  • one to work on complex technical projects and closely work with peers in an innovative and
  • fast-paced environment.
  • Grow our analytics capabilities with faster, more reliable data pipelines, and better
  • tools, handling petabytes of data every day.
  • Brainstorm and create new platforms that can help in our quest to make data
  • available to cluster users in all shapes and forms, with low latency and horizontal
  • scalability.
  • Make changes to our data platform, refactoring/redesigning as needed and
  • diagnosing any problems across the entire technical stack.
  • Design and develop a real-time events pipeline for Data ingestion for real-time dash-
  • boarding.
  • Develop complex and efficient functions to transform raw data sources into powerful,
  • reliable components of our data lake.
  • Design & implement new components and various emerging technologies in Hadoop
  • Eco System, and successful execution of various projects.
  • Optimize and improve existing features or data processes for performance and
  • stability.
  • Conduct peer design and code reviews.
  • Write unit tests and support continuous integration.
  • Be obsessed about quality and ensure minimal production downtimes.
  • Mentor peers, share information and knowledge and help build a great team.
  • Monitor job performances, file system/disk-space management, cluster & database
  • connectivity, log files, management of backup/security and troubleshooting various
  • user issues.
  • Collaborate with various cross-functional teams: infrastructure, network, database.
Desired Skills
  •  Fluent with data structures, algorithms and design patterns.
  •  Strong hands-on experience with Hadoop, MapReduce, Hive, Spark.
  •  Excellent programming/debugging skills in Java/Scala.
  •  Experience with any scripting language such as Python, Bash etc.
  •  Good to have experience of working with noSQL databases like Hbase, Cassandra.
  •  Hands on programming experience with multithreaded applications.
  •  Good to have experience in Database, SQL, messaging queues like Kafka.
  •  Good to have experience in developing streaming applications eg Spark Streaming,
  • Flink, Storm, etc.
  •  Good to have experience with AWS and cloud technologies such as S3
  •  Experience with caching architectures like Redis, Memcached etc.
  •  Memory optimization and GC tuning.
  •  Experience with profiling and performance optimizations.
  •  Experience with agile development methodologies and DevOps action.
Read more
Job posted by
i
Sagar Das
Apply for job
i
Founded 2017  •  Product  •  20-100 employees  •  Raised funding
Natural Language Processing (NLP)
Artificial Intelligence (AI)
Deep Learning
Machine Learning (ML)
Python
Java
Scala
Natural Language Toolkit (NLTK)
i
Bengaluru (Bangalore)
i
- yrs
i
₹5L - ₹10L / yr
We at artivatic are seeking passionate, talented and research focused natural processing language engineer with strong machine learning and mathematics background to help build industry-leading technology. - The ideal candidate will have research/implementation experience modeling and developing NLP tools and experience working with machine learning/deep learning algorithms.Qualifications :- Bachelors or Master degree in Computer Science, Mathematics or related field with specialization in natural language processing, Machine Learning or Deep Learning.- Publication record in conferences/journals is a plus.- 2+ years of working/research experience building NLP based solutions is preferred.Required Skills :- Hands-on Experience building NLP models using different NLP libraries ad toolkit like NLTK, Stanford NLP etc.- Good understanding of Rule-based, Statistical and probabilistic NLP techniques.- Good knowledge of NLP approaches and concepts like topic modeling, text summarization, semantic modeling, Named Entity recognition etc.- Good understanding of Machine learning and Deep learning algorithms.- Good knowledge of Data Structures and Algorithms.- Strong programming skills in Python/Java/Scala/C/C++.- Strong problem solving and logical skills.- A go-getter kind of attitude with a willingness to learn new technologies.- Well versed with software design paradigms and good development practices.Responsibilities :- Developing novel algorithms and modeling techniques to advance the state of the art in Natural Language Processing.- Developing NLP based tools and solutions end to end.
Read more
Job posted by
i
Layak Singh
Apply for job
i
Founded 2012  •  Products & Services  •  100-1000 employees  •  Bootstrapped
Machine Learning (ML)
Deep Learning
Python
TensorFlow
Keras
Natural Language Processing (NLP)
i
Mumbai
i
- yrs
i
₹4L - ₹15L / yr
1. The candidate should be passionate about machine learning and deep learning.
2. Should understand the importance and know-how of taking the machine-learning-based solution to the consumer.
3. Hands-on experience with statistical, machine-learning tools and techniques
4. Good exposure to Deep learning libraries like Tensorflow, PyTorch.
5. Experience in implementing Deep Learning techniques, Computer Vision and NLP. The candidate should be able to develop the solution from scratch with Github codes exposed.
6. Should be able to read research papers and pick ideas to quickly reproduce research in the most comfortable Deep Learning library.
7. Should be strong in data structures and algorithms. Should be able to do code complexity analysis/optimization for smooth delivery to production.
8. Expert level coding experience in Python.
9. Technologies: Backend - Python (Programming Language)
10. Should have the ability to think long term solutions, modularity, and reusability of the components.
11. Should be able to work in a collaborative way. Should be open to learning from peers as well as constantly bring new ideas to the table.
12. Self-driven missile. Open to peer criticism, feedback and should be able to take it positively. Ready to be held accountable for the responsibilities undertaken.
Read more
Job posted by
i
Anwar Shaikh
Apply for job
i
Founded 2017  •  Services  •  20-100 employees  •  Raised funding
Data Science
R Programming
Python
i
NCR (Delhi | Gurgaon | Noida)
i
- yrs
i
₹9L - ₹20L / yr
What you will do:
As a Data Science Lead, you will be working on creating industry first analytical and propensity models to
help discover the information hidden in vast amounts of data, and make smarter decisions to deliver
even better customer experience. Your primary focus will be in applying data mining techniques, doing
statistical analysis, and building high quality prediction systems integrated with our products.

➢ Working with business and leadership teams to gathering and analyse structured and unstructured data
➢ Data mining using state-of-the-art methods
➢ Enhancing data collection procedures to include information that is relevant for building analytic
systems
➢ Processing, cleansing, and verifying the integrity of data used for analysis
➢ Doing ad-hoc analysis and presenting results in a clear manner
➢ Creating automated anomaly detection systems and constant tracking of its performance
➢ Creation and evolution of an efficient BI pipeline into a multi-faceted pipeline to support various
modelling needs.

What we are looking for:

➢ 5-8 years of relevant experience, preferably in financial services industry.
➢ A bachelors / master’s degree in the field of Statistics, Mathematics, Computer Science or
Management from Tier 1 Institutes.
➢ Data warehousing experience will be a plus.
➢ Good conceptual understanding of statistics and probability.
➢ Experience in developing dashboards and reports using BI tools.
Read more
Job posted by
i
Deepika Toppo
Apply for job
i
Founded 2018  •  Products & Services  •  0-20 employees  •  Bootstrapped
Data Science
R Programming
Python
i
Hyderabad
i
- yrs
i
₹3L - ₹6L / yr
We're an early stage film-tech startup with a mission to empower filmmakers and independent content creators with data-driven decision-making tools. We're looking for a data person to join the core team. Please get in touch if you would be excited to join us on this super exciting journey of disrupting the film production and distribution business. We are currently collaborating with Rana Daggubatt's Suresh Productions, and work out of their studio in Hyderabad - so exposure and opportunities to work on real issues faced by the media industry will be in plenty.
Read more
Job posted by
i
Athul Krishnan
Apply for job
Did not find a job you were looking for?
i
Search
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on CutShort.
iiiii
Want to apply for this role at DataToBiz?
i
Apply for this job
Why apply via CutShort?
Connect with actual hiring teams and get their fast response. No spam.