Data Engineer

at Product / Internet / Media Companies

Agency job
icon
Bengaluru (Bangalore)
icon
4 - 9 yrs
icon
₹15L - ₹30L / yr
icon
Full time
Skills
Big Data
Hadoop
Data processing
Python
Data engineering
HDFS
Spark
Data lake

REQUIREMENT:

  •  Previous experience of working in large scale data engineering
  •  4+ years of experience working in data engineering and/or backend technologies with cloud experience (any) is mandatory.
  •  Previous experience of architecting and designing backend for large scale data processing.
  •  Familiarity and experience of working in different technologies related to data engineering – different database technologies, Hadoop, spark, storm, hive etc.
  •  Hands-on and have the ability to contribute a key portion of data engineering backend.
  •  Self-inspired and motivated to drive for exceptional results.
  •  Familiarity and experience working with different stages of data engineering – data acquisition, data refining, large scale data processing, efficient data storage for business analysis.
  •  Familiarity and experience working with different DB technologies and how to scale them.

RESPONSIBILITY:

  •  End to end responsibility to come up with data engineering architecture, design, development and then implementation of it.
  •  Build data engineering workflow for large scale data processing.
  •  Discover opportunities in data acquisition.
  •  Bring industry best practices for data engineering workflow.
  •  Develop data set processes for data modelling, mining and production.
  •  Take additional tech responsibilities for driving an initiative to completion
  •  Recommend ways to improve data reliability, efficiency and quality
  •  Goes out of their way to reduce complexity.
  •  Humble and outgoing - engineering cheerleaders.
Read more
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

AI/ML Developer

at An US based firm offering permanent WFH

Agency job
via Jobdost
Deep Learning
Computer Vision
PyTorch
TensorFlow
Python
OpenCV
pandas
matplotlib
RestNet
VGG
MobileNet
Pix2pix
icon
Remote only
icon
3 - 6 yrs
icon
₹12L - ₹23L / yr
This person MUST have:
- B.E Computer Science or equivalent.
- In-depth knowledge of machine learning algorithms and their applications including
practical experience with and theoretical understanding of algorithms for classification,
regression and clustering.
- Hands-on experience in computer vision and deep learning projects to solve real world
problems involving vision tasks such as object detection, Object tracking, instance
segmentation, activity detection, depth estimation, optical flow, multi-view geometry,
domain adaptation etc.
- Strong understanding of modern and traditional Computer Vision Algorithms.
- Experience in one of the Deep Learning Frameworks / Networks: PyTorch, TensorFlow,
Darknet (YOLO v4 v5), U-Net, Mask R-CNN, EfficientDet, BERT etc.
- Proficiency with CNN architectures such as ResNet, VGG, UNet, MobileNet, pix2pix,
and Cycle GAN.
- Experienced user of libraries such as OpenCV, scikit-learn, matplotlib and pandas.
- Ability to transform research articles into working solutions to solve real-world problems.
- High proficiency in Python programming knowledge.
- Familiar with software development practices/pipelines (DevOps- Kubernetes, docker
containers, CI/CD tools).
- Strong communication skills.
Read more
Job posted by
Mamatha A
Deep Learning
Machine Learning (ML)
Data Science
Natural Language Processing (NLP)
Computer Vision
Python
NumPy
pandas
TensorFlow
MongoDB
recommendation algorithm
icon
Remote only
icon
6 - 9 yrs
icon
₹12L - ₹15L / yr
We are hiring for the Machine Learning Lead in our Data Science Team.

  • Role: Machine Learning Lead
  • Experience: 5+ Years
  • Employee strength: 80+
  • Remuneration: Most competitive in the market


Programming Language:

• Advance knowledge of Python.

• Object Oriented Programming skills.

 

Conceptual:

• Mathematical understanding of machine learning and deep learning algorithms.

• Thorough grasp on statistical terminologies.

 

Applied:

• Libraries: Tensorflow, Keras, Pytorch, Statsmodels, Scikit-learn, SciPy, Numpy, Pandas, Matplotlib, Seaborn, Plotly

• Algorithms: Ensemble Algorithms, Artificial Neural Networks and Deep Learning, Clustering Algorithms, Decision Tree Algorithms, Dimensionality Reduction Algorithms, etc.

• MySQL, MongoDB, ElasticSearch or other NoSQL database implementations.

If interested kindly share your cv at tanya @tigihr. com

 
 
Candidates preview
Read more
Job posted by
Grenisha Patel

Data Scientist

at Blue Sky Analytics

Founded 2018  •  Product  •  20-100 employees  •  Raised funding
NumPy
SciPy
Data Science
Python
pandas
Git
GitHub
SQL
Amazon S3
Amazon EC2
GIS analysis
GDAL
QGIS
icon
Remote only
icon
1 - 5 yrs
icon
Best in industry

About the Company

Blue Sky Analytics is a Climate Tech startup that combines the power of AI & Satellite data to aid in the creation of a global environmental data stack. Our funders include Beenext and Rainmatter. Over the next 12 months, we aim to expand to 10 environmental data-sets spanning water, land, heat, and more!


We are looking for a data scientist to join its growing team. This position will require you to think and act on the geospatial architecture and data needs (specifically geospatial data) of the company. This position is strategic and will also require you to collaborate closely with data engineers, data scientists, software developers and even colleagues from other business functions. Come save the planet with us!


Your Role

Manage: It goes without saying that you will be handling large amounts of image and location datasets. You will develop dataframes and automated pipelines of data from multiple sources. You are expected to know how to visualize them and use machine learning algorithms to be able to make predictions. You will be working across teams to get the job done.

Analyze: You will curate and analyze vast amounts of geospatial datasets like satellite imagery, elevation data, meteorological datasets, openstreetmaps, demographic data, socio-econometric data and topography to extract useful insights about the events happening on our planet.

Develop: You will be required to develop processes and tools to monitor and analyze data and its accuracy. You will develop innovative algorithms which will be useful in tracking global environmental problems like depleting water levels, illegal tree logging, and even tracking of oil-spills.

Demonstrate: A familiarity with working in geospatial libraries such as GDAL/Rasterio for reading/writing of data, and use of QGIS in making visualizations. This will also extend to using advanced statistical techniques and applying concepts like regression, properties of distribution, and conduct other statistical tests.

Produce: With all the hard work being put into data creation and management, it has to be used! You will be able to produce maps showing (but not limited to) spatial distribution of various kinds of data, including emission statistics and pollution hotspots. In addition, you will produce reports that contain maps, visualizations and other resources developed over the course of managing these datasets.

Requirements

These are must have skill-sets that we are looking for:

  • Excellent coding skills in Python (including deep familiarity with NumPy, SciPy, pandas).
  • Significant experience with git, GitHub, SQL, AWS (S3 and EC2).
  • Worked on GIS and is familiar with geospatial libraries such as GDAL and rasterio to read/write the data, a GIS software such as QGIS for visualisation and query, and basic machine learning algorithms to make predictions.
  • Demonstrable experience implementing efficient neural network models and deploying them in a production environment.
  • Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests and proper usage, etc.) and experience with applications.
  • Capable of writing clear and lucid reports and demystifying data for the rest of us.
  • Be curious and care about the planet!
  • Minimum 2 years of demonstrable industry experience working with large and noisy datasets.

Benefits

  • Work from anywhere: Work by the beach or from the mountains.
  • Open source at heart: We are building a community where you can use, contribute and collaborate on.
  • Own a slice of the pie: Possibility of becoming an owner by investing in ESOPs.
  • Flexible timings: Fit your work around your lifestyle.
  • Comprehensive health cover: Health cover for you and your dependents to keep you tension free.
  • Work Machine of choice: Buy a device and own it after completing a year at BSA.
  • Quarterly Retreats: Yes there's work-but then there's all the non-work+fun aspect aka the retreat!
  • Yearly vacations: Take time off to rest and get ready for the next big assignment by availing the paid leaves.
Read more
Job posted by
Balahun Khonglanoh

Data Analyst

at MedCords

Founded 2016  •  Product  •  20-100 employees  •  Raised funding
Data Analytics
Data Analyst
R Language
Python
icon
Kota
icon
0 - 1 yrs
icon
₹1L - ₹2.5L / yr

Required Python ,R

work in handling large-scale data engineering pipelines.
Excellent verbal and written communication skills.
Proficient in PowerPoint or other presentation tools.
Ability to work quickly and accurately on multiple projects.

Read more
Job posted by
kavita jain
PySpark
Python
Spark
icon
Bengaluru (Bangalore)
icon
3 - 7 yrs
icon
₹8L - ₹16L / yr
Roles and Responsibilities:

• Responsible for developing and maintaining applications with PySpark 
• Contribute to the overall design and architecture of the application developed and deployed.
• Performance Tuning wrt to executor sizing and other environmental parameters, code optimization, partitions tuning, etc.
• Interact with business users to understand requirements and troubleshoot issues.
• Implement Projects based on functional specifications.

Must-Have Skills:

• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ETL architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skills
Read more
Job posted by
Priyanka U

Data Engineer

at Srijan Technologies

Founded 2002  •  Products & Services  •  100-1000 employees  •  Profitable
Big Data
Apache Kafka
Hadoop
Spark
Data engineering
Python
Scala
Kafka
icon
Remote only
icon
2 - 5 yrs
icon
₹5L - ₹15L / yr
Job Description:-
We are looking for a Data Engineer, responsibilities include creating machine learning models and retraining systems. To do this job successfully, you need exceptional skills in statistics and programming. If you also have knowledge of data science and software engineering, your ultimate goal will be to shape and build efficient self-learning applications.


Technical Knowledge (Must Have)

  • Strong experience in SQL / HiveQL/ AWS Athena,
  • Strong expertise in the development of data pipelines (snaplogic is preferred).
  • Design, Development, Deployment and administration of data processing applications.
  • Good Exposure towards AWS and Azure Cloud computing environments.
  • Knowledge around BigData, AWS Cloud Architecture, Best practices, Securities, Governance, Metadata Management, Data Quality etc.
  • Data extraction through various firm sources (RDBMS, Unstructured Data Sources) and load to datalake with all best practices.
  • Knowledge in Python
  • Good knowledge in NoSQL technologies (Neo4J/ MongoDB)
  • Experience/knowledge in SnapLogic (ETL Technologies)
  • Working knowledge on Unix (AIX, Linux), shell scripting
  • Experience/knowledge in Data Modeling. Database Development
  • Experience/knowledge creation of reports and dashboards in Tableau/ PowerBI
Read more
Job posted by
Srijan Technologies

Director | Applied AI

at Searce Inc

Founded 2004  •  Products & Services  •  100-1000 employees  •  Profitable
Artificial Intelligence (AI)
Natural Language Processing (NLP)
Machine Learning (ML)
Deep Learning
Research and development
TensorFlow
Spark
Hadoop
Data Analytics
Data Science
Engineering Management
Django
HTML/CSS
Flask
Google Cloud Platform (GCP)
icon
Pune
icon
10 - 14 yrs
icon
₹30L - ₹35L / yr

Director - Applied AI


Who we are?

Searce is  a niche’ Cloud Consulting business with futuristic tech DNA. We do new-age tech to realise the “Next” in the “Now” for our Clients. We specialise in Cloud Data Engineering, AI/Machine Learning and Advanced Cloud infra tech such as Anthos and Kubernetes. We are one of the top & the fastest growing partners for Google Cloud and AWS globally with over 2,500 clients successfully moved to cloud.


What do we believe?

  • Best practices are overrated
      • Implementing best practices can only make one an average .
  • Honesty and Transparency
      • We believe in naked truth. We do what we tell and tell what we do.
  • Client Partnership
    • Client - Vendor relationship: No. We partner with clients instead. 
    • And our sales team comprises 100% of our clients.

How do we work ?

It’s all about being Happier first. And rest follows. Searce work culture is defined by HAPPIER.

  • Humble: Happy people don’t carry ego around. We listen to understand; not to respond.
  • Adaptable: We are comfortable with uncertainty. And we accept changes well. As that’s what life's about.
  • Positive: We are super positive about work & life in general. We love to forget and forgive. We don’t hold grudges. We don’t have time or adequate space for it.
  • Passionate: We are as passionate about the great street-food vendor across the street as about Tesla’s new model and so on. Passion is what drives us to work and makes us deliver the quality we deliver.
  • Innovative: Innovate or Die. We love to challenge the status quo.
  • Experimental: We encourage curiosity & making mistakes.
  • Responsible: Driven. Self motivated. Self governing teams. We own it.

So, what are we hunting for ?

  1. To devise strategy through the delivery of sustainable intelligent solutions, strategic customer engagements, and research and development
  2. To enable and lead our data and analytics team and develop machine learning and AI paths across strategic programs, solution implementation, and customer relationships
  3. To manage existing customers and realize new opportunities and capabilities of growth
  4. To collaborate with different stakeholders for delivering automated, high availability and secure solutions
  5. To develop talent and skills to create a high performance team that delivers superior products
  6. To communicate effectively across the organization to ensure that the team is completely aligned to business objectives
  7. To build strong interpersonal relationships with peers and other key stakeholders that will contribute to your team's success

Your bucket of Undertakings :

  1. Develop an AI roadmap aligned to client needs and vision
  2. Develop a Go-To-Market strategy of AI solutions for customers
  3. Build a diverse cross-functional team to identify and prioritize key areas of the business across AI, NLP and other cognitive solutions that will drive significant business benefit
  4. Lead AI R&D initiatives to include prototypes and minimum viable products
  5. Work closely with multiple teams on projects like Visual quality inspection, ML Ops, Conversational banking, Demand forecasting, Anomaly detection etc. 
  6. Build reusable and scalable solutions for use across the customer base
  7. Create AI white papers and enable strategic partnerships with industry leaders
  8. Align, mentor, and manage, team(s) around strategic initiatives
  9. Prototype and demonstrate AI related products and solutions for customers
  10. Establish processes, operations, measurement, and controls for end-to-end life-cycle management of the digital workforce (intelligent systems)
  11. Lead AI tech challenges and proposals with team members
  12. Assist business development teams in the expansion and enhancement of a pipeline to support short- and long-range growth plans
  13. Identify new business opportunities and prioritize pursuits for AI 

Education & Experience : 

  1. Advanced or basic degree (PhD with few years experience, or MS / BS (with many years experience)) in a quantitative field such as CS, EE, Information sciences, Statistics, Mathematics, Economics, Operations Research, or related, with focus on applied and foundational Machine Learning , AI , NLP and/or / data-driven statistical analysis & modelling
  2. 10+ years of Experience majorly in applying AI/ML/ NLP / deep learning / data-driven statistical analysis & modelling solutions to multiple domains, including financial engineering, financial processes a plus
  3. Strong, proven programming skills and with machine learning and deep learning and Big data frameworks including TensorFlow, Caffe, Spark, Hadoop. Experience with writing complex programs and implementing custom algorithms in these and other environments
  4. Experience beyond using open source tools as-is, and writing custom code on top of, or in addition to, existing open source frameworks
  5. Proven capability in demonstrating successful advanced technology solutions (either prototypes, POCs, well-cited research publications, and/or products) using ML/AI/NLP/data science in one or more domains
  6. Experience in data management, data analytics middleware, platforms and infrastructure, cloud and fog computing is a plus
  7. Excellent communication skills (oral and written) to explain complex algorithms, solutions to stakeholders across multiple disciplines, and ability to work in a diverse team
Read more
Job posted by
Mishita Juneja

Data Engineer

at They provide both wholesale and retail funding. PM1

Agency job
via Multi Recruit
Spark
Big Data
Data Engineer
Hadoop
Apache Kafka
Data Warehousing
Data Modeling
Hive
Kafka
icon
Bengaluru (Bangalore)
icon
1 - 5 yrs
icon
₹15L - ₹20L / yr
  • 1-5 years of experience in building and maintaining robust data pipelines, enriching data, low-latency/highly-performance data analytics applications.
  • Experience handling complex, high volume, multi-dimensional data and architecting data products in streaming, serverless, and microservices-based Architecture and platform.
  • Experience in Data warehousing, Data modeling, and Data architecture.
  • Expert level proficiency with the relational and NoSQL databases.
  • Expert level proficiency in Python, and PySpark.
  • Familiarity with Big Data technologies and utilities (Spark, Hive, Kafka, Airflow).
  • Familiarity with cloud services (preferable AWS)
  • Familiarity with MLOps processes such as data labeling, model deployment, data-model feedback loop, data drift.

Key Roles/Responsibilities:

  • Act as a technical leader for resolving problems, with both technical and non-technical audiences.
  • Identifying and solving issues with data pipelines regarding consistency, integrity, and completeness.
  • Lead data initiatives, architecture design discussions, and implementation of next-generation BI solutions.
  • Partner with data scientists, tech architects to build advanced, scalable, efficient self-service BI infrastructure.
  • Provide thought leadership and mentor data engineers in information presentation and delivery.

 

 

Read more
Job posted by
Kavitha S

Data Engineer

at Yulu Bikes Pvt Ltd

Founded 2017  •  Products & Services  •  20-100 employees  •  Raised funding
Big Data
Spark
Scala
Hadoop
Apache Kafka
ETL
Datawarehousing
Distributed computing
MongoDB
Amazon Web Services (AWS)
icon
Bengaluru (Bangalore)
icon
2 - 5 yrs
icon
₹15L - ₹28L / yr
Job Description
We are looking for a Data Engineer that will be responsible for collecting, storing, processing, and analyzing huge sets of data that is coming from different sources.

Responsibilities
Working with Big Data tools and frameworks to provide requested capabilities Identify development needs in order to improve and streamline operations Develop and manage BI solutions Implementing ETL process and Data Warehousing Monitoring performance and managing infrastructure

Skills 
Proficient understanding of distributed computing principles Proficiency with Hadoop and Spark Experience with building stream-processing systems, using solutions such as Kafka and Spark-Streaming Good knowledge of Data querying tools SQL and Hive Knowledge of various ETL techniques and frameworks Experience with Python/Java/Scala (at least one) Experience with cloud services such as AWS or GCP Experience with NoSQL databases, such as DynamoDB,MongoDB will be an advantage Excellent written and verbal communication skills
Read more
Job posted by
Keerthana k

Big Data Developer

at MediaMelon Inc

Founded 2008  •  Product  •  20-100 employees  •  Raised funding
Scala
Spark Streaming
Aero spike
Cassandra
Apache Kafka
Big Data
Elastic Search
icon
Bengaluru (Bangalore), Bengaluru (Bangalore)
icon
1 - 7 yrs
icon
₹0L / yr
Develop analytic tools, working on BigData and Distributed systems. - Provide technical leadership on developing our core Analytic platform - Lead development efforts on product features using Scala/Java -Demonstrable excellence in innovation, problem solving, analytical skills, data structures and design patterns - Expert in building applications using Spark and Spark Streaming -Exposure to NoSQL: HBase/Cassandra, Hive and Pig -Latin, Mahout -Extensive experience with Hadoop and Machine learning algorithms
Read more
Job posted by
Katreddi Kiran Kumar
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Product / Internet / Media Companies?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort