Data Engineer

at Shiprocket

DP
Posted by sunil kumar
icon
Delhi, Gurugram, Noida
icon
4 - 6 yrs
icon
₹14L - ₹18L / yr
icon
Full time
Skills
SQL
Data engineering
Hadoop
Apache Kafka
Spark
Data Analytics
Data-flow analysis
Big Data
Data warehouse architecture
Datawarehousing

                                                Sr. Data Engineer

 

Company Profile:

 

Bigfoot Retail Solutions [Shiprocket] is a logistics platform which connects Indian eCommerce SMBs with logistics players to enable end to end solutions.

Our  innovative  data backed  platform  drives  logistics  efficiency,  helps  reduce  cost,  increases  sales throughput by reducing RTO and improves post order customer engagement and experience.

Our vision is to power all logistics for the direct commerce market in India

including first mile, linehaul,  last mile, warehousing, cross border and O2O.

 

Position: Sr.Data Engineer

Team : Business Intelligence

Location:  New Delhi

 

Job Description:

We are looking for a savvy Data Engineer to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

 

Key Responsibilities:

  • Create and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
  • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
  • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
  • Keep our data separated and secure across national boundaries through multiple data centres and AWS regions.
  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
  • Work with data and analytics experts to strive for greater functionality in our data systems.

 

 

Qualifications for Data Engineer

  • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • A successful history of manipulating, processing and extracting value from large disconnected datasets.
  • Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
  • Strong project management and organizational skills.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
    • Experience with big data tools: Hadoop, Spark, Kafka, etc.
    • Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
    • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
    • Experience with AWS cloud services: EC2, EMR, RDS, Redshift
    • Experience with stream-processing systems: Storm, Spark-Streaming, etc.
    • Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

 

 

 

 

 

Read more

About Shiprocket

ShipRocket is India's most used eCommerce logistics and shipping software solution. Avail features like COD (Cash on Delivery), prepaid delivery, automated shipping, multiple couriers, rate calculator, etc. Ship your products to customers to 26,000+ pin codes in India.
Read more
Founded
2012
Type
Product
Size
500-1000 employees
Stage
Profitable
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Senior Manager- Data Intelligence|

at Data Intelligence Platform For Decision-Makers

Agency job
via Unnati
Data Analytics
Quantitative analyst
data intelligence
Analytical Skills
icon
Delhi, Gurugram, Noida
icon
5 - 8 yrs
icon
₹12L - ₹15L / yr

Supporting today’s data driven business world, our client acts as a full-stack data intelligence platform which leverages granular and deep data from various sources, thus helping the decision-makers at the executive level. Their solutions include supply chain optimization, building footprints, track construction hotspots, real estate, and lots more.


Their work embeds geospatial analytics, location intelligence and predictive modelling in the foundations of economic modelling and evaluation theory to build data intelligence layers for their clients which include governments, multilateral institutions, and private organizations.

 

Headquartered in New Delhi, our client works as a team of economists, data scientists, geo-spatial analysts, etc. Their decision-support system includes Big-Data, predictive modeling, forecasting, socio economic dataset and many more.


As a Senior Manager– Data Intelligence, you will be responsible for contributing to all stages of projects– conceptualizing, preparing work plans, overseeing analytical work, driving teams to meet targets and ensuring quality standards.

What you will do:

  • Thoroughly understanding the data processing pipeline and troubleshooting/problem-solving the technical team
  • Acting as SPOC for client communications across the portfolio of projects undertaken by the organization
  • Contributing to different aspects of organizational growth– team building, process building, strategy and business development

 

Desired Candidate Profile

What you need to have:

  • Post-graduate degree in relevant subjects such as Economics/ Engineering/ Quantitative Social Sciences/ Management etc
  • At least 5 years of relevant work experience
  • Vast experience in managing multiple client projects
  • Strong data/quantitative analysis skills are a must
  • Prior experience working in data analytics teams
  • Credible experience of data platforms & languages in the past    
Read more
Job posted by
Sarika Tamhane

Machine Learning Engineer - NLP

at Leena AI

Founded 2015  •  Product  •  20-100 employees  •  Raised funding
Natural Language Processing (NLP)
Machine Learning (ML)
Data Science
Data Analytics
icon
Remote only
icon
2 - 8 yrs
icon
₹25L - ₹40L / yr

Responsibilities: 

  • Improve robustness of Leena AI current NLP stack 
  • Increase zero shot learning capability of Leena AI current NLP stack 
  • Opportunity to add/build new NLP architectures based on requirements 
  • Manage End to End lifecycle of the data in the system till it achieves more than 90% accuracy 
  • Manage a NLP team 

Page BreakRequirements: 

  • Strong understanding of linear algebra, optimisation, probability, statistics 
  • Experience in the data science methodology from exploratory data analysis, feature engineering, model selection, deployment of the model at scale and model evaluation 
  • Experience in deploying NLP architectures in production 
  • Understanding of latest NLP architectures like transformers is good to have 
  • Experience in adversarial attacks/robustness of DNN is good to have 
  • Experience with Python Web Framework (Django), Analytics and Machine Learning frameworks like Tensorflow/Keras/Pytorch. 
Read more
Job posted by
Preethi Gothandam

Data Engineer

at surusha technology Pvt Ltd

Founded 2016  •  Products & Services  •  20-100 employees  •  Profitable
C#
SQL Azure
ETL
OLAP
SQL
Data engineering
icon
Remote only
icon
3 - 6 yrs
icon
₹3L - ₹6L / yr
Role - Data Engineer

Skillsets-Azure, Olap, Etl, sql, python, c#

exp range - 3+ to 4 years

salary-best in industry

notice period - Currently serving notice period (Immediate joiners are preferred)

location- remote work

job type -permanent role

it is full time and totally remote based


Note: For the interview 3 rounds are there -technical round, manager/client round, hr round
Read more
Job posted by
subham kumar

Data Engineer (PySpark+SQL)

at Fragma Data Systems

Founded 2015  •  Products & Services  •  employees  •  Profitable
PySpark
Data engineering
Data Warehouse (DWH)
SQL
Spark
PowerBI
icon
Remote, Bengaluru (Bangalore)
icon
3.5 - 8 yrs
icon
₹5L - ₹18L / yr
Must-Have Skills:
• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skill
 
 
Technology Skills (Good to Have):
  • Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
  • Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. 
  • Designing and implementing data engineering, ingestion, and transformation functions
  • Azure Synapse or Azure SQL data warehouse
  • Spark on Azure is available in HD insights and data bricks
 
Good to Have: 
  • Experience with Azure Analysis Services
  • Experience in Power BI
  • Experience with third-party solutions like Attunity/Stream sets, Informatica
  • Experience with PreSales activities (Responding to RFPs, Executing Quick POCs)
  • Capacity Planning and Performance Tuning on Azure Stack and Spark.
Read more
Job posted by
Evelyn Charles
PySpark
Big Data
Spark
Data Warehouse (DWH)
SQL
Data engineering
Apache Synapse
icon
Abu Dhabi, Dubai
icon
6 - 12 yrs
icon
₹18L - ₹25L / yr
Must-Have Skills:
• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skill
 
 
Technology Skills (Good to Have):
  • Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
  • Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. 
  • Designing and implementing data engineering, ingestion, and transformation functions
  • Azure Synapse or Azure SQL data warehouse
  • Spark on Azure is available in HD insights and data bricks
Read more
Job posted by
Evelyn Charles

Data Analyst

at Aza Fashions

Founded 2005  •  Products & Services  •  100-1000 employees  •  Profitable
Data Analytics
Analytical Skills
Google Analytics
Marketing analytics
Business Analysis
B2C
icon
Prabhadevi, Mumbai
icon
5 - 10 yrs
icon
₹5L - ₹15L / yr

Job Responsibilities

  • In this role, you will work closely with business leaders, subject matter experts to design/define, measure, and track suitable metrics for efficiency and effectiveness of business.
  • Influence product roadmap for website and mobile through high quality digital analytical insights and recommendations.
  • Drive customer conversions and increase customer delight via system interventions designed based on data driven insights.
  • You will be responsible for modeling complex problems and processes using various analytical tools and methods.
  • Effectively carry out the translation of business requirements into functional specifications to satisfy the business needs and necessary system modifications.
  • Compile and analyse data related to business' issues.
  • Develop clear visualizations to convey complicated data in a straightforward fashion

Job Specification:

  • Excellent analytical abilities: Data driven and understanding of key analytical techniques • Hands on: Web and app analytics tools (GA, apps flyer, Clever tap etc), Excel, SQL.
  • Experience of quantifying impact of product features for sales conversion improvement, A/B testing, attribution modeling, campaign analysis etc.
  • Champion of user behaviour understanding on digital platforms (website, mobile applications etc.)
  • Effectively analyse and monitor the services, market trends, and customer requirements.
  • Effectively undertake the requirement elicitation, document and analysis, solution design, testing definition and execution.
  • Knowledge about the different functionalities and features of the main web browsers
  • Ability to write clear reports and maintain the important records.
  • Effective organizational and management skills.
  • Expert in time management and the ability for achieving the given tasks within the allocated time frame .
  • Atleast 3 to 6 years of experience with a growing e-commerce company or with analytics will be very important .
  • Capable of taking initiatives for process betterments and flow improvements.
Read more
Job posted by
Sejal Shah

Bigdata Lead Architecture

at Datametica Solutions Private Limited

Founded 2013  •  Products & Services  •  100-1000 employees  •  Profitable
Big Data
Hadoop
Spark
Apache Spark
Apache Hive
PySpark
Python
Data engineering
icon
Pune, Hyderabad
icon
7 - 12 yrs
icon
₹12L - ₹33L / yr

Job description

Role : Lead Architecture (Spark, Scala, Big Data/Hadoop, Java)

Primary Location : India-Pune, Hyderabad

Experience : 7 - 12 Years

Management Level: 7

Joining Time: Immediate Joiners are preferred


  • Attend requirements gathering workshops, estimation discussions, design meetings and status review meetings
  • Experience of Solution Design and Solution Architecture for the data engineer model to build and implement Big Data Projects on-premises and on cloud.
  • Align architecture with business requirements and stabilizing the developed solution
  • Ability to build prototypes to demonstrate the technical feasibility of your vision
  • Professional experience facilitating and leading solution design, architecture and delivery planning activities for data intensive and high throughput platforms and applications
  • To be able to benchmark systems, analyses system bottlenecks and propose solutions to eliminate them
  • Able to help programmers and project managers in the design, planning and governance of implementing projects of any kind.
  • Develop, construct, test and maintain architectures and run Sprints for development and rollout of functionalities
  • Data Analysis, Code development experience, ideally in Big Data Spark, Hive, Hadoop, Java, Python, PySpark,
  • Execute projects of various types i.e. Design, development, Implementation and migration of functional analytics Models/Business logic across architecture approaches
  • Work closely with Business Analysts to understand the core business problems and deliver efficient IT solutions of the product
  • Deployment sophisticated analytics program of code using any of cloud application.


Perks and Benefits we Provide!


  • Working with Highly Technical and Passionate, mission-driven people
  • Subsidized Meals & Snacks
  • Flexible Schedule
  • Approachable leadership
  • Access to various learning tools and programs
  • Pet Friendly
  • Certification Reimbursement Policy
  • Check out more about us on our website below!

www.datametica.com

Read more
Job posted by
Nikita Aher

Senior Data Engineer

at Data Team

Agency job
via Oceanworld
Big Data
Data engineering
Hadoop
data engineer
Apache Hive
Apache Kafka
icon
Remote only
icon
8 - 12 yrs
icon
₹10L - ₹20L / yr
Senior Data Engineer (SDE)

(Hadoop, HDFS, Kafka, Spark, Hive)

Overall Experience - 8 to 12 years

Relevant exp on Big data - 3+ years in above

Salary: Max up-to 20LPA 

Job location - Chennai / Bangalore / 

Notice Period - Immediate joiner / 15-to-20-day Max 

The Responsibilities of The Senior Data Engineer Are:

- Requirements gathering and assessment

- Breakdown complexity and translate requirements to specification artifacts and story boards to build towards, using a test-driven approach

- Engineer scalable data pipelines using big data technologies including but not limited to Hadoop, HDFS, Kafka, HBase, Elastic

- Implement the pipelines using execution frameworks including but not limited to MapReduce, Spark, Hive, using Java/Scala/Python for application design.

- Mentoring juniors in a dynamic team setting

- Manage stakeholders with proactive communication upholding TheDataTeam's brand and values

A Candidate Must Have the Following Skills:

- Strong problem-solving ability

- Excellent software design and implementation ability

- Exposure and commitment to agile methodologies

- Detail oriented with willingness to proactively own software tasks as well as management tasks, and see them to completion with minimal guidance

- Minimum 8 years of experience

- Should have experience in full life-cycle of one big data application

- Strong understanding of various storage formats (ORC/Parquet/Avro)

- Should have hands on experience in one of the Hadoop distributions (Hortoworks/Cloudera/MapR)

- Experience in at least one cloud environment (GCP/AWS/Azure)

- Should be well versed with at least one database (MySQL/Oracle/MongoDB/Postgres)

- Bachelor's in Computer Science, and preferably, a Masters as well - Should have good code review and debugging skills

Additional skills (Good to have):

- Experience in Containerization (docker/Heroku)

- Exposure to microservices

- Exposure to DevOps practices - Experience in Performance tuning of big data applications
Read more
Job posted by
Chandan J

Data analytics Manager

at FMCG sector

Agency job
via CIEL HR Services
Data Analytics
Advanced analytics
Data mining
Statistical Modeling
Analytics
icon
Remote only
icon
9 - 16 yrs
icon
₹20L - ₹30L / yr

Role: Data Analytics Lead / Manager

  1. Lead cross-functional projects using advanced data modelling and analysis techniques to discover insights that will guide strategic decisions and uncover optimization opportunities.
  2. Build, develop and maintain data models, reporting systems, data automation systems, dashboards and performance metrics support that support key business decisions.
  3. Coordinate with different teams to determine requirements for data warehousing, reporting and analytical solutions, and ensure customer satisfaction on deliverables
  4. Oversee analytics projects to extract, manage, and analyse data from multiple applications, ensuring that deadlines are met.
  5. Apply statistics and data modelling to gain actionable business insights and boost customer productivity and revenue.
  6. Enforce company policies and procedures to ensure quality and prevent discrepancies.
  7. Communicate and track key performance metrics across departments.
  8. Keep abreast of industry best practices and policies.
  9. Research Latest trends, analyse data, identify opportunities and incorporate changes into business strategies.
  10. Manage and optimize processes for data intake, validation, mining and engineering as well as modelling, visualization and communication deliverables.
  11. Examine, interpret and report results of analytical initiatives to stakeholders in leadership, technology, sales, marketing and product teams.
  12. Oversee the data/report requests process: tracking requests submitted, prioritization, approval, etc.
  13. Develop and implement MLDevOps, quality controls and departmental standards to ensure quality standards, organizational expectations, and regulatory requirements.
  14. Anticipate future demands of initiatives related to people, technology, budget and business within your department and design/implement solutions to meet these needs.
  15. Organize and drive successful completion of data insight initiatives through effective management of data analyst and effective collaboration with stakeholders.

Essential Skills

  1. Working knowledge of data mining principles: predictive analytics, mapping, collecting data from multiple data systems on premises and cloud-based data sources.
  2. Strong SQL skills, ability to perform effective querying involving multiple tables and subqueries.
  3. Understanding of and experience using analytical concepts and statistical techniques: hypothesis development, designing tests/experiments, analyzing data, drawing conclusions, and developing actionable recommendations for business units.
  4. Experience and knowledge of statistical modelling techniques: multiple regression, logistic regression, log-linear regression, variable selection, etc.

 We’re looking for someone with at least 10+ years of experience in a position monitoring, managing, transforming and drawing insights from data, and someone with at least 5 years of experience leading a data analyst team.

Experience in Azure is preferred

Read more
Job posted by
Riya Tharakan

Technical Architect

at Nitor Infotech

Founded 2006  •  Products & Services  •  100-1000 employees  •  Profitable
Amazon Web Services (AWS)
Big Data
Business Intelligence (BI)
icon
Pune
icon
9 - 100 yrs
icon
₹13L - ₹25L / yr
The hunt is for a AWS BigData /DWH Architect with the ability to manage effective relationships with a wide range of stakeholders (customers & team members alike). Incumbent will demonstrate personal commitment and accountability to ensure standards are continuously sustained and improved both within the internal teams, and with partner organizations and suppliers. We at Nitor Infotech a Product Engineering Services company are always on hunt for some best talents in the IT industry & keeping with our trend of What next in IT. We are scouting for result oriented resources with passion for product, technology services, and creating great customer experiences. Someone who can take our current expertise & footprint of Nitor Infotech Inc., to an altogether different dimension & level in tune with the emerging market trends and ensure Brilliance @ Work continues to prevail in whatever we do. Nitor Infotech works with global ISVs to help them build and accelerate their product development. Nitor is able to do so because of the fact that product development is its DNA. This DNA is enriched by its 10 years of expertise, best practices and frameworks & Accelerators. Because of this ability Nitor Infotech has been able to build business relationships with product companies having revenues from $50 Million to $1 Billion. • 7-12+ years of relevant experience of working in Database, BI and Analytics space with over 0-2 yrs of architecting and designing data warehouse experience including 2 to 3 yrs in Big Data ecosystem • Experience in data warehouse design in AWS • Strong architecting, programming, design skills and proven track record of architecting and building large scale, distributed big data solutions • Professional and technical advice on Big Data concepts and technologies, in particular highlighting the business potential through real-time analysis • Provides technical leadership in Big Data space (Hadoop Stack like M/R, HDFS, Pig, Hive, HBase, Flume, Sqoop, etc. NoSQL stores like Mongodb, Cassandra, HBase etc.) • Performance tuning of Hadoop clusters and Hadoop MapReduce routines. • Evaluate and recommend Big Data technology stack for the platform • Drive significant technology initiatives end to end and across multiple layers of architecture • Should have breadth of BI knowledge which includes:  MSBI, Database design, new visualization tools like Tableau, Qlik View, Power BI  Understand internals and intricacies of Old and New DB platform which includes:  Strong RDMS DB Fundamentals either of it SQL Server/ MySQL/ Oracle  DB and DWH design  Designing Semantic Model using OLAP and Tabular model using MS and Non MS tools  No SQL DBs including Document, Graph, Search and Columnar DBs • Excellent communication skills and strong ability to build good rapport with prospect and existing customers • Be a Mentor and go to person for Jr. team members in the team Qualification & Experience: · Educational qualification: BE/ME/B.Tech/M.Tech, BCA/MCA/BCS/MCS, any other degree with relevant IT qualification.
Read more
Job posted by
Balakumar Mohan
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Shiprocket?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort