data engineer

at Information Solution Provider Company

Agency job
icon
Delhi
icon
3 - 5 yrs
icon
₹3L - ₹10L / yr
icon
Full time
Skills
Spark
Scala
Hadoop
Big Data
Data engineering
PySpark
Machine Learning (ML)

Data Engineer 

Responsibilities:

 

  • Designing and implementing fine-tuned production ready data/ML pipelines in Hadoop platform.
  • Driving optimization, testing and tooling to improve quality.
  • Reviewing and approving high level & amp; detailed design to ensure that the solution delivers to the business needs and aligns to the data & analytics architecture principles and roadmap.
  • Understanding business requirements and solution design to develop and implement solutions that adhere to big data architectural guidelines and address business requirements.
  • Following proper SDLC (Code review, sprint process).
  • Identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, etc.
  • Building robust and scalable data infrastructure (both batch processing and real-time) to support needs from internal and external users.
  • Understanding various data security standards and using secure data security tools to apply and adhere to the required data controls for user access in the Hadoop platform.
  • Supporting and contributing to development guidelines and standards for data ingestion.
  • Working with a data scientist and business analytics team to assist in data ingestion and data related technical issues.
  • Designing and documenting the development & deployment flow.

 

Requirements:

 

  • Experience in developing rest API services using one of the Scala frameworks.
  • Ability to troubleshoot and optimize complex queries on the Spark platform
  • Expert in building and optimizing ‘big data’ data/ML pipelines, architectures and data sets.
  • Knowledge in modelling unstructured to structured data design.
  • Experience in Big Data access and storage techniques.
  • Experience in doing cost estimation based on the design and development.
  • Excellent debugging skills for the technical stack mentioned above which even includes analyzing server logs and application logs.
  • Highly organized, self-motivated, proactive, and ability to propose best design solutions.
  • Good time management and multitasking skills to work to deadlines by working independently and as a part of a team.

 

Read more
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Luxury e-commerce platform well-established organisation
Agency job
via The Hub by Sridevi Viswanathan
Hyderabad
3 - 8 yrs
₹23L - ₹28L / yr
Machine Learning (ML)
Algorithms
Data mining
Pattern recognition
Digital Signal Processing
Algorithm Engineer
Experience 3 to 8 Years

Skill Set
  • experience in algorithm development with a focus on signal processing, pattern recognition, machine learning, classification, data mining, and other areas of machine intelligence.
  • Ability to analyse data streams from multiple sensors and develop algorithms to extract accurate and meaningful sport metrics.
  • Should have a deeper understanding of IMU sensors and Biosensors like HRM, ECG
  • A good understanding on Power and memory management on embedded platform
Responsibilities
  •  Expertise in the design of multitasking, event-driven, real-time firmware using C and understanding of RTOS concepts
  • Knowledge of Machine learning, Analytical and methodical approaches to data analysis and verification and Python
  • Prior experience on fitness algorithm development using IMU sensor
  •  Interest in fitness activities and knowledge of human body anatomy
Read more
Agency job
via zyoin by RAKESH RANJAN
Remote, Bengaluru (Bangalore)
5 - 8 yrs
₹20L - ₹38L / yr
Big Data
Spark
Hadoop
Apache Kafka
Apache Hive
+4 more

Company Overview:

Rakuten, Inc. (TSE's first section: 4755) is the largest ecommerce company in Japan, and third largest eCommerce marketplace company worldwide. Rakuten provides a variety of consumer and business-focused services including e-commerce, e-reading, travel, banking, securities, credit card, e-money, portal and media, online marketing and professional sports. The company is expanding globally and currently has operations throughout Asia, Western Europe, and the Americas. Founded in 1997, Rakuten is headquartered in Tokyo, with over 17,000 employees and partner staff worldwide. Rakuten's 2018 revenues were 1101.48 billions yen.   -In Japanese, Rakuten stands for ‘optimism.’ -It means we believe in the future. -It’s an understanding that, with the right mind-set, -we can make the future better by what we do today. Today, our 70+ businesses span e-commerce, digital content, communications and FinTech, bringing the joy of discovery to more than 1.2 billion members across the world.


Website
: https://www.rakuten.com/

Crunchbase : Rakuten  has raised a total of  $42.4M  in funding over  2  rounds

Companysize : 10,001 + Employees

Founded : 1997

Headquarters : Tokyo, Japan

Work location : Bangalore (M.G.Road)


Please find below Job Description.


Role Description – Data Engineer for AN group (Location - India)

 

Key responsibilities include:

 

We are looking for engineering candidate in our Autonomous Networking Team. The ideal candidate must have following abilities –

 

  • Hands- on experience in big data computation technologies (at least one and potentially several of the following: Spark and Spark Streaming, Hadoop, Storm, Kafka Streaming, Flink, etc)
  • Familiar with other related big data technologies, such as big data storage technologies (e.g., Phoenix/HBase, Redshift, Presto/Athena, Hive, Spark SQL, BigTable, BigQuery, Clickhouse, etc), messaging layer (Kafka, Kinesis, etc), Cloud and container- based deployments (Docker, Kubernetes etc), Scala, Akka, SocketIO, ElasticSearch, RabbitMQ, Redis, Couchbase, JAVA, Go lang.
  • Partner with product management and delivery teams to align and prioritize current and future new product development initiatives in support of our business objectives
  • Work with cross functional engineering teams including QA, Platform Delivery and DevOps
  • Evaluate current state solutions to identify areas to improve standards, simplify, and enhance functionality and/or transition to effective solutions to improve supportability and time to market
  • Not afraid of refactoring existing system and guiding the team about same.
  • Experience with Event driven Architecture, Complex Event Processing
  • Extensive experience building and owning large- scale distributed backend systems.
Read more
Global SaaS product built to help revenue teams. (TP1)
Agency job
via Multi Recruit by Kavitha S
Bengaluru (Bangalore)
2 - 5 yrs
₹30L - ₹40L / yr
Data Scientist
Data Science
Machine Learning (ML)
  • You'd have to set up your own shop, work with design customers to find generalizable use cases, and build them out.
  • Ability to collaborate with cross-functional teams to build and ship new features
  • At least 2-5 years of experience
  • Predictive Analytics – Machine Learning Algorithms, Logistics & Linear Regression, Decision Tree, Clustering. 
  • Exploratory Data Analysis – Data Preparation, Data Exploration, and Data Visualization. 
  • Analytics Tools – R, Python, SQL, Power BI, MS Excel.
Read more
Leading Multinational Co
Agency job
via SIlverPeople Consulting by Richa Awasthi
Mumbai
2 - 9 yrs
₹8L - ₹27L / yr
Machine Learning (ML)

ML Engineer-Analyst/ Senior Analyst

Job purpose:

To design and develop machine learning and deep learning systems. Run machine learning tests andexperiments and implementing appropriate ML algorithms. Works cross-functionally with the Data Scientists, Software application developers and business groups for the development of innovative ML models. Use Agile experience to work collaboratively with other Managers/Owners in geographically distributed teams.

Accountabilities:

  • Work with Data Scientists and Business Analysts to frame problems in a business context. Assist all the processes from data collection, cleaning, and preprocessing, to training models and deploying them to production.
  • Understand business objectives and developing models that help to achieve them, along with metrics to track their progress.
  • Explore and visualize data to gain an understanding of it, then identify differences in data distribution that could affect performance when deploying the model in the real world.
  • Define validation strategies, preprocess or feature engineering to be done on a given dataset and data augmentation pipelines.
  • Analyze the errors of the model and design strategies to overcome them.
  • Collaborate with data engineers to build data and model pipelines, manage the infrastructure and data pipelines needed to bring code to production and demonstrate end-to-end understanding of applications (including, but not limited to, the machine learning algorithms) being created.

Qualifications & Specifications

  • Bachelor's degree in Engineering /Computer Science/ Math/ Statistics or equivalent. Master's degree in relevant specification will be first preference
  • Experience of machine learning algorithms and libraries
  • Understanding of data structures, data modeling and software architecture.
  • Deep knowledge of math, probability, statistics and algorithms
  • Experience with machine learning platforms such as Microsoft Azure, Google Cloud, IBM Watson, and Amazon
  • Big data environment: Hadoop, Spark
  • Programming languages: Python, R, PySpark
  • Supervised & Unsupervised machine learning: linear regression, logistic regression, k-means

clustering, ensemble models, random forest, svm, gradient boosting

  • Sampling data: bagging & boosting, bootstrapping
  • Neural networks: ANN, CNN, RNN related topics
  • Deep learning: Keras, Tensorflow
  • Experience with AWS Sagemaker deployment and agile methodology

 

 

Read more
App-based lending platform. ( AF1)
Agency job
via Multi Recruit by Ayub Pasha
Bengaluru (Bangalore)
2 - 5 yrs
₹25L - ₹28L / yr
Data Science
Machine Learning (ML)
Data Scientist
Python
Logistic regression
+2 more
  • Use data to develop machine learning models that optimize decision making in Credit Risk, Fraud, Marketing, and Operations
  • Implement data pipelines, new features, and algorithms that are critical to our production models
  • Create scalable strategies to deploy and execute your models
  • Write well designed, testable, efficient code
  • Identify valuable data sources and automate collection processes.
  • Undertake to preprocess of structured and unstructured data.
  • Analyze large amounts of information to discover trends and patterns.

 

Requirements:

  • 2+ years of experience in applied data science or engineering with a focus on machine learning
  • Python expertise with good knowledge of machine learning libraries, tools, techniques, and frameworks (e.g. pandas, sklearn, xgboost, lightgbm, logistic regression, random forest classifier, gradient boosting regressor, etc)
  • strong quantitative and programming skills with a product-driven sensibility

 

 

Read more
DP
Posted by Aniket Manhar Nanjee
Bengaluru (Bangalore)
2 - 5 yrs
₹6L - ₹10L / yr
Data Science
R Programming
Python
Scala
Tableau
+1 more
Simplilearn.com is the world’s largest professional certifications company and an Onalytica Top 20 influential brand. With a library of 400+ courses, we've helped 500,000+ professionals advance their careers, delivering $5 billion in pay raises. Simplilearn has over 6500 employees worldwide and our customers include Fortune 1000 companies, top universities, leading agencies and hundreds of thousands of working professionals. We are growing over 200% year on year and having fun doing it. Description We are looking for candidates with strong technical skills and proven track record in building predictive solutions for enterprises. This is a very challenging role and provides an opportunity to work on developing insights based Ed-Tech software products used by large set of customers across globe. It provides an exciting opportunity to work across various advanced analytics & data science problem statement using cutting-edge modern technologies collaborating with product, marketing & sales teams. Responsibilities • Work on enterprise level advanced reporting requirements & data analysis. • Solve various data science problems customer engagement, dynamic pricing, lead scoring, NPS improvement, optimization, chatbots etc. • Work on data engineering problems utilizing our tech stack - S3 Datalake, Spark, Redshift, Presto, Druid, Airflow etc. • Collect relevant data from source systems/Use crawling and parsing infrastructure to put together data sets. • Craft, conduct and analyse A/B experiments to evaluate machine learning models/algorithms. • Communicate findings and take algorithms/models to production with ownership. Desired Skills • BE/BTech/MSc/MS in Computer Science or related technical field. • 2-5 years of experience in advanced analytics discipline with solid data engineering & visualization skills. • Strong SQL skills and BI skills using Tableau & ability to perform various complex analytics in data. • Ability to propose hypothesis and design experiments in the context of specific problems using statistics & ML algorithms. • Good overlap with Modern Data processing framework such as AWS-lambda, Spark using Scala or Python. • Dedication and diligence in understanding the application domain, collecting/cleaning data and conducting various A/B experiments. • Bachelor Degree in Statistics or, prior experience with Ed-Tech is a plus
Read more
Bengaluru (Bangalore)
3 - 7 yrs
₹4L - ₹20L / yr
Machine Learning (ML)
Natural Language Processing (NLP)
Artificial Intelligence (AI)
chat bot
Role Description: The Chatbot Developer will develop software and system architecture while ensuring alignment with enterprise technology standards (e.g., solution patterns, application frameworks).

Responsibilities:

- Develop REST/JSON API’s Design code for high scale/availability/resiliency.

- Develop responsive web apps and integrate APIs using NodeJS.

- Presenting Chat efficiency reports to higher Management

- Develop system flow diagrams to automate a business function and identify impacted systems; metrics to depict the cost benefit analysis of the solutions developed.

- Work closely with business operations to convert requirements into system solutions and collaborate with development teams to ensure delivery of highly scalable and available systems.

- Using tools to classify/categorize the chat based on intents and coming up with F1 score for Chat Analysis

- Experience in analyzing real agents Chat conversation with agent to train the Chatbot.

- Developing Conversational Flows in the chatbot

- Calculating Chat efficiency reports.

Good to Have:

- Monitors performance and quality control plans to identify performance. 

- Works on problems of moderate and varied complexity where analysis of data may require adaptation of standardized practices. 

- Works with management to prioritize business and information needs.

- Experience in analyzing real agents Chat conversation with agent to train the Chatbot.

- Identifies, analyzes, and interprets trends or patterns in complex data sets.

- Ability to manage multiple assignments.

- Understanding of ChatBot Architecture.

- Experience of Chatbot training
Read more
DP
Posted by Juzar Malubhoy
Pune
3 - 7 yrs
₹7L - ₹15L / yr
Machine Learning (ML)
Python
Data Science
We are looking for a Machine Learning Engineer with 3+ years of experience with a background in Statistics and hands-on experience in the Python ecosystem, using sound  Software Engineering practices. Skills & Knowledge: - Formal knowledge of fundamentals of probability & statistics along with the ability to apply basic statistical analysis methods like hypothesis testing, t-tests, ANOVA etc. - Hands-on knowledge of data formats, data extraction, loading, wrangling, transformation, pre-processing and analysis. - Thorough understanding of data-modeling and machine-learning concepts - Complete understanding and ability to apply, implement and adapt standard implementations of machine learning algorithms - Good understanding and ability to apply and adapt Neural Networks and Deep Learning, including common high-level Deep Learning architectures like CNNs and RNNs - Fundamentals of computer science & programming, especially Data structures (like multi-dimensional arrays, trees, and graphs) and Algorithms (like searching, sorting, and dynamic programming) - Fundamentals of software engineering and system design, such as requirements analysis, REST APIs, database queries, system and library calls, version control, etc. Languages and Libraries: - Hands-on experience with Python and Python Libraries for data analysis and machine learning, especially Scikit-learn, Tensorflow, Pandas, Numpy, Statsmodels, and Scipy. - Experience with R and its ecosystem is a plus - Knowledge of other open source machine learning and data modeling frameworks like Spark MLlib, H2O, etc. is a plus
Read more
DP
Posted by Tanu Mehra
Anywhere
8 - 11 yrs
₹5L - ₹10L / yr
nano electronics
vehicle dynamics
computational dynamics
Android Development
Big Data
+3 more
We are a team with a mission, A mission to create and deliver great learning experiences to engineering students through various workshops and courses. If you are an industry professional and :- See great scope of improvement in higher technical education across the country and connect with our purpose of impacting it for good. Keen on sharing your technical expertise to enhance the practical learning of students. You are innovative in your ways of creating content and delivering them. You don’t mind earning few extra bucks while doing this in your free time. Buzz us at [email protected] and let us discuss how together we can take technological education in the country to new heights.
Read more
DP
Posted by Sarabjeet Singh
Pune
10 - 16 yrs
₹10L - ₹20L / yr
Machine Learning (ML)
Python
Artificial Intelligence (AI)
Deep Learning
Natural Language Processing (NLP)
+3 more

Job Description:

Roles & Responsibilities:

· You will be involved in every part of the project lifecycle, right from identifying the business problem and proposing a solution, to data collection, cleaning, and preprocessing, to training and optimizing ML/DL models and deploying them to production.

· You will often be required to design and execute proof-of-concept projects that can demonstrate business value and build confidence with CloudMoyo’s clients.

· You will be involved in designing and delivering data visualizations that utilize the ML models to generate insights and intuitively deliver business value to CXOs.


Desired Skill Set:

· Candidates should have strong Python coding skills and be comfortable working with various ML/DL frameworks and libraries.

· Hands-on skills and industry experience in one or more of the following areas is necessary:

1)      Deep Learning (CNNs/RNNs, Reinforcement Learning, VAEs/GANs)

2)      Machine Learning (Regression, Random Forests, SVMs, K-means, ensemble methods)

3)      Natural Language Processing

4)      Graph Databases (Neo4j, Apache Giraph)

5)      Azure Bot Service

6)      Azure ML Studio / Azure Cognitive Services

7)      Log Analytics with NLP/ML/DL

· Previous experience with data visualization, C# or Azure Cloud platform and services will be a plus.

· Candidates should have excellent communication skills and be highly technical, with the ability to discuss ideas at any level from executive to developer.

· Creative problem-solving, unconventional approaches and a hacker mindset is highly desired.

Read more
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Information Solution Provider Company?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort