Data Engineer

at INSOFE

DP
Posted by Nitika Bist
icon
Hyderabad, Bengaluru (Bangalore)
icon
7 - 10 yrs
icon
₹12L - ₹18L / yr
icon
Full time
Skills
Big Data
Data engineering
Apache Hive
Apache Spark
Hadoop
Amazon Web Services (AWS)
Java
SQL
Python
Roles & Responsibilities:
  • Total Experience of 7-10 years and should be interested in teaching and research
  • 3+ years’ experience in data engineering which includes data ingestion, preparation, provisioning, automated testing, and quality checks.
  • 3+ Hands-on experience in Big Data cloud platforms like AWS and GCP, Data Lakes and Data Warehouses
  • 3+ years of Big Data and Analytics Technologies. Experience in SQL, writing code in spark engine using python, scala or java Language. Experience in Spark, Scala
  • Experience in designing, building, and maintaining ETL systems
  • Experience in data pipeline and workflow management tools like Airflow
  • Application Development background along with knowledge of Analytics libraries, opensource Natural Language Processing, statistical and big data computing libraries
  • Familiarity with Visualization and Reporting Tools like Tableau, Kibana.
  • Should be good at storytelling in Technology
Please note that candidates should be interested in teaching and research work.

Qualification: B.Tech / BE / M.Sc / MBA / B.Sc, Having Certifications in Big Data Technologies and Cloud platforms like AWS, Azure and GCP will be preferred
Primary Skills: Big Data + Python + Spark + Hive + Cloud Computing
Secondary Skills: NoSQL+ SQL + ETL + Scala + Tableau
Selection Process: 1 Hackathon, 1 Technical round and 1 HR round
Benefit: Free of cost training on Data Science from top notch professors
Read more

About INSOFE

Founded
2011
Type
Services
Size
100-1000 employees
Stage
Profitable
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Data Architect

at EnterpriseMinds

Founded 2017  •  Products & Services  •  100-1000 employees  •  Profitable
Spark
Hadoop
Big Data
Data engineering
PySpark
Python
Java
Scala
icon
Pune
icon
9 - 14 yrs
icon
₹20L - ₹40L / yr
Hi,

Enterprise Minds is looking for Data Architect for Pune Location.

Req Skills:
Python,Pyspark,Hadoop,Java,Scala
Read more
Job posted by
phani kalyan

DevOps Engineer

at Panamax InfoTech Ltd.

Founded 2004  •  Products & Services  •  100-1000 employees  •  Profitable
Data Warehouse (DWH)
Informatica
ETL
DevOps
Python
Perl
Java
.NET
Shell Scripting
Bash
Terraform
SVN
Maven
Git
Docker
Kubernetes
Chef
Ansible
Puppet
Splunk
Gradle
Software deployment
helm
icon
Remote only
icon
3 - 4 yrs
icon
₹5L - ₹10L / yr
1. Excellent understanding of at least any one of the programming language .NET ,Python, Perl, and Java
2. Good understanding and hands on experience in Shell/Bash scripting, sonarqube, Terraform,
3. Experience with Continuous Integration and Continuous Deployment Pipelines
4. Experience in SVN, Maven, Git and Git workflows
5. Should be able to develop overall strategy for Build & Release management
6. Experience in working with container orchestration tools such as Docker and Kubernetes
7. Good knowledge in Devops Automation Tools like Chef, Ansible, Puppet, helm, splunk, maven, gradle & XL Deploy.etc
8. Managing stakeholders and external interfaces and Setting up tools and required infrastructure
9. Encouraging and building automated processes wherever possible
10. Awareness of critical concepts in DevOps and Agile principles
11. Experience in Cloud infrastructure like AWS, GCP or Azure. In AWS understanding on EC2, S3 & cloud
12. Strong knowledge and hands on experience in unix OS
13.Experience in network, server, application status monitoring and troubleshooting, Security.
14.Design, develop automation suite and integrate with continuous integration process through Jenkins
15. Possess good problem solving and debugging skills. Troubleshoot issues and coordinate with development team to streamline code deployment to generate build
Read more
Job posted by
Bhavani P

Data Scientist

at CarWale

Founded  •   •  employees  • 
Data Science
Data Scientist
R Programming
Python
Machine Learning (ML)
Amazon Web Services (AWS)
icon
Navi Mumbai, Mumbai
icon
3 - 5 yrs
icon
₹10L - ₹15L / yr

About CarWale: CarWale's mission is to bring delight in car buying, we offer a bouquet of reliable tools and services to help car consumers decide on buying the right car, at the right price and from the right partner. CarWale has always strived to serve car buyers and owners in the most comprehensive and convenient way possible. We provide a platform where car buyers and owners can research, buy, sell and come together to discuss and talk about their cars.We aim to empower Indian consumers to make informed car buying and ownership decisions with exhaustive and un-biased information on cars through our expert reviews, owner reviews, detailed specifications and comparisons. We understand that a car is by and large the second-most expensive asset a consumer associates his lifestyle with! Together with CarTrade & BikeWale, we are the market leaders in the personal mobility media space.About the Team:We are a bunch of enthusiastic analysts assisting all business functions with their data needs. We deal with huge but diverse datasets to find relationships, patterns and meaningful insights. Our goal is to help drive growth across the organization by creating a data-driven culture.

We are looking for an experienced Data Scientist who likes to explore opportunities and know their way around data to build world class solutions making a real impact on the business. 

 

Skills / Requirements –

  • 3-5 years of experience working on Data Science projects
  • Experience doing statistical modelling of big data sets
  • Expert in Python, R language with deep knowledge of ML packages
  • Expert in fetching data from SQL
  • Ability to present and explain data to management
  • Knowledge of AWS would be beneficial
  • Demonstrate Structural and Analytical thinking
  • Ability to structure and execute data science project end to end

 

Education –

Bachelor’s degree in a quantitative field (Maths, Statistics, Computer Science). Masters will be preferred.

 

Read more
Job posted by
Vanita Acharya

Data Scientist

at LodgIQ

Founded 2015  •  Product  •  20-100 employees  •  Raised funding
Data Science
Machine Learning (ML)
Data Scientist
Python
MongoDB
Amazon Web Services (AWS)
icon
Remote, Bengaluru (Bangalore)
icon
3 - 12 yrs
icon
₹10L - ₹30L / yr

About LodgIQ

LodgIQ is led by a team of experienced hospitality technology experts, data scientists and product domain experts. Seed funded by Highgate Ventures, a venture capital platform focused on early stage technology investments in the hospitality industry and Trilantic Capital Partners, a global private equity firm, LodgIQ has made a significant investment in advanced machine learning platforms and data science.

Title : Data Scientist


Job Description:

  • Apply Data Science and Machine Learning to a REAL-LIFE  problem  - “Predict Guest Arrivals and Determine Best Prices for Hotels
  • Apply advanced analytics in a BIG Data Environment – AWS, MongoDB, SKLearn
  • Help scale up the product in a global offering across 100+  global markets

Qualifications:

  • Minimum 3 years of experience with advanced data analytic techniques, including data mining, machine learning, statistical analysis, and optimization.    Student projects are acceptable.
  • At least 1 year of experience with Python / Numpy / Pandas / Scipy/ MatPlotLib / Scikit-Learn
  •   Experience in working with massive data sets, including structured and unstructured with at least 1 prior engagement involving data gathering, data cleaning, data mining, and data visualization
  •   Solid grasp over  optimization techniques
  •   Master's  or PhD degree in Business Analytics.  Data science, Statistics or Mathematics
  •   Ability to show a track record of solving large, complex problems
Read more
Job posted by
Sougata Chatterjee

Data Engineer_Scala

at Ganit Business Solutions

Founded 2017  •  Products & Services  •  100-1000 employees  •  Bootstrapped
ETL
Informatica
Data Warehouse (DWH)
Big Data
Scala
Hadoop
Apache Hive
PySpark
Spark
icon
Remote only
icon
4 - 7 yrs
icon
₹10L - ₹30L / yr

Job Description:

We are looking for a Big Data Engineer who have worked across the entire ETL stack. Someone who has ingested data in a batch and live stream format, transformed large volumes of daily and built Data-warehouse to store the transformed data and has integrated different visualization dashboards and applications with the data stores.    The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them.

Responsibilities:

  • Develop, test, and implement data solutions based on functional / non-functional business requirements.
  • You would be required to code in Scala and PySpark daily on Cloud as well as on-prem infrastructure
  • Build Data Models to store the data in a most optimized manner
  • Identify, design, and implement process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Implementing the ETL process and optimal data pipeline architecture
  • Monitoring performance and advising any necessary infrastructure changes.
  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
  • Work with data and analytics experts to strive for greater functionality in our data systems.
  • Proactively identify potential production issues and recommend and implement solutions
  • Must be able to write quality code and build secure, highly available systems.
  • Create design documents that describe the functionality, capacity, architecture, and process.
  • Review peer-codes and pipelines before deploying to Production for optimization issues and code standards

Skill Sets:

  • Good understanding of optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and ‘big data’ technologies.
  • Proficient understanding of distributed computing principles
  • Experience in working with batch processing/ real-time systems using various open-source technologies like NoSQL, Spark, Pig, Hive, Apache Airflow.
  • Implemented complex projects dealing with the considerable data size (PB).
  • Optimization techniques (performance, scalability, monitoring, etc.)
  • Experience with integration of data from multiple data sources
  • Experience with NoSQL databases, such as HBase, Cassandra, MongoDB, etc.,
  • Knowledge of various ETL techniques and frameworks, such as Flume
  • Experience with various messaging systems, such as Kafka or RabbitMQ
  • Creation of DAGs for data engineering
  • Expert at Python /Scala programming, especially for data engineering/ ETL purposes

 

 

 

Read more
Job posted by
Vijitha VS

Event & Unstructured Data

at They provide both wholesale and retail funding. PM1

Agency job
via Multi Recruit
AWS KINESYS
Data engineering
AWS Lambda
DynamoDB
data pipeline
Data governance
Data processing
Amazon Web Services (AWS)
athena
Audio
Linux/Unix
Python
SQL
WebLogic
KINESYS
Lambda
icon
Mumbai
icon
5 - 7 yrs
icon
₹20L - ₹25L / yr
  • Key responsibility is to design & develop a data pipeline for real-time data integration, processing, executing of the model (if required), and exposing output via MQ / API / No-SQL DB for consumption
  • Provide technical expertise to design efficient data ingestion solutions to store & process unstructured data, such as Documents, audio, images, weblogs, etc
  • Developing API services to provide data as a service
  • Prototyping Solutions for complex data processing problems using AWS cloud-native solutions
  • Implementing automated Audit & Quality assurance Checks in Data Pipeline
  • Document & maintain data lineage from various sources to enable data governance
  • Coordination with BIU, IT, and other stakeholders to provide best-in-class data pipeline solutions, exposing data via APIs, loading in down streams, No-SQL Databases, etc

Skills

  • Programming experience using Python & SQL
  • Extensive working experience in Data Engineering projects, using AWS Kinesys,  AWS S3, DynamoDB, EMR, Lambda, Athena, etc for event processing
  • Experience & expertise in implementing complex data pipeline
  • Strong Familiarity with AWS Toolset for Storage & Processing. Able to recommend the right tools/solutions available to address specific data processing problems
  • Hands-on experience in Unstructured (Audio, Image, Documents, Weblogs, etc) Data processing.
  • Good analytical skills with the ability to synthesize data to design and deliver meaningful information
  • Know-how on any No-SQL DB (DynamoDB, MongoDB, CosmosDB, etc) will be an advantage.
  • Ability to understand business functionality, processes, and flows
  • Good combination of technical and interpersonal skills with strong written and verbal communication; detail-oriented with the ability to work independently

Functional knowledge

  • Real-time Event Processing
  • Data Governance & Quality assurance
  • Containerized deployment
  • Linux
  • Unstructured Data Processing
  • AWS Toolsets for Storage & Processing
  • Data Security

 

Read more
Job posted by
Sapna Deb

Senior Big Data Engineer

at Banyan Data Services

Founded 2018  •  Product  •  20-100 employees  •  Bootstrapped
Data Science
Data Scientist
MongoDB
Java
Big Data
Apache Kafka
Python
SQL
Deep Learning
RF
Generalized linear model
k-means clustering
Hadoop
Spring
Apache HBase
Cassandra
DevOps
Docker
Kubernetes
icon
Bengaluru (Bangalore)
icon
3 - 15 yrs
icon
₹6L - ₹20L / yr

Senior Big Data Engineer 

Note:   Notice Period : 45 days 

Banyan Data Services (BDS) is a US-based data-focused Company that specializes in comprehensive data solutions and services, headquartered in San Jose, California, USA. 

 

We are looking for a Senior Hadoop Bigdata Engineer who has expertise in solving complex data problems across a big data platform. You will be a part of our development team based out of Bangalore. This team focuses on the most innovative and emerging data infrastructure software and services to support highly scalable and available infrastructure. 

 

It's a once-in-a-lifetime opportunity to join our rocket ship startup run by a world-class executive team. We are looking for candidates that aspire to be a part of the cutting-edge solutions and services we offer that address next-gen data evolution challenges. 

 

 

Key Qualifications

 

·   5+ years of experience working with Java and Spring technologies

· At least 3 years of programming experience working with Spark on big data; including experience with data profiling and building transformations

· Knowledge of microservices architecture is plus 

· Experience with any NoSQL databases such as HBase, MongoDB, or Cassandra

· Experience with Kafka or any streaming tools

· Knowledge of Scala would be preferable

· Experience with agile application development 

· Exposure of any Cloud Technologies including containers and Kubernetes 

· Demonstrated experience of performing DevOps for platforms 

· Strong Skillsets in Data Structures & Algorithm in using efficient way of code complexity

· Exposure to Graph databases

· Passion for learning new technologies and the ability to do so quickly 

· A Bachelor's degree in a computer-related field or equivalent professional experience is required

 

Key Responsibilities

 

· Scope and deliver solutions with the ability to design solutions independently based on high-level architecture

· Design and develop the big data-focused micro-Services

· Involve in big data infrastructure, distributed systems, data modeling, and query processing

· Build software with cutting-edge technologies on cloud

· Willing to learn new technologies and research-orientated projects 

· Proven interpersonal skills while contributing to team effort by accomplishing related results as needed 

Read more
Job posted by
Sathish Kumar
PySpark
Big Data
Spark
Data Warehouse (DWH)
SQL
Data engineering
Apache Synapse
icon
Abu Dhabi, Dubai
icon
6 - 12 yrs
icon
₹18L - ₹25L / yr
Must-Have Skills:
• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skill
 
 
Technology Skills (Good to Have):
  • Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
  • Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. 
  • Designing and implementing data engineering, ingestion, and transformation functions
  • Azure Synapse or Azure SQL data warehouse
  • Spark on Azure is available in HD insights and data bricks
Read more
Job posted by
Evelyn Charles

Data Engineer

at Big revolution in the e-gaming industry. (GK1)

Agency job
via Multi Recruit
Python
Scala
Hadoop
Spark
Data Engineer
Kafka
Luigi
Airflow
Nosql
icon
Bengaluru (Bangalore)
icon
2 - 3 yrs
icon
₹15L - ₹20L / yr
  • We are looking for a Data Engineer to build the next-generation mobile applications for our world-class fintech product.
  • The candidate will be responsible for expanding and optimising our data and data pipeline architecture, as well as optimising data flow and collection for cross-functional teams.
  • The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimising data systems and building them from the ground up.
  • Looking for a person with a strong ability to analyse and provide valuable insights to the product and business team to solve daily business problems.
  • You should be able to work in a high-volume environment, have outstanding planning and organisational skills.

 

Qualifications for Data Engineer

 

  • Working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
  • Experience building and optimising ‘big data’ data pipelines, architectures, and data sets.
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Looking for a candidate with 2-3 years of experience in a Data Engineer role, who is a CS graduate or has an equivalent experience.

 

What we're looking for?

 

  • Experience with big data tools: Hadoop, Spark, Kafka and other alternate tools.
  • Experience with relational SQL and NoSQL databases, including MySql/Postgres and Mongodb.
  • Experience with data pipeline and workflow management tools: Luigi, Airflow.
  • Experience with AWS cloud services: EC2, EMR, RDS, Redshift.
  • Experience with stream-processing systems: Storm, Spark-Streaming.
  • Experience with object-oriented/object function scripting languages: Python, Java, Scala.
Read more
Job posted by
Ayub Pasha

Internship- JAVA / Python / AI / ML

at Wise Source

Founded 2014  •  Product  •  20-100 employees  •  Profitable
Artificial Intelligence (AI)
Machine Learning (ML)
Internship
Java
Python
icon
Remote, Guindy
icon
0 - 2 yrs
icon
₹1L - ₹1.5L / yr
Looking out for Internship Candidates . Designation:- Intern/ Trainee Technology : .NET/JAVA/ Python/ AI/ ML Duration : 2-3 Months Job Location :Online Internship Joining :Immediately Job Type :Internship Job Description - MCA/M.Tech/ B.Tech/ BE who need 2-6 months internship project to be done. - Should be available to join us immediately. - Should be flexible to work on any Skills/ Technologies. - Ready to work in long working hours. - Must possess excellent analytical and logical skills. - Internship experience is provided from experts - Internship Certificate will be provided at the end of training. - The requirement is strictly for internship and not a permanent job - Stipend will be provided only based on the performance.
Read more
Job posted by
Wise HR
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at INSOFE?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort