Data Engineer

at Our client company is into Analytics. (RF1)

Agency job
icon
Bengaluru (Bangalore)
icon
3 - 5 yrs
icon
₹12L - ₹14L / yr
icon
Full time
Skills
Data Engineer
Big Data
Python
Amazon Web Services (AWS)
SQL
Java
ETL
  •  We are looking for a Data Engineer with 3-5 years experience in Python, SQL, AWS (EC2, S3, Elastic Beanstalk, API Gateway), and Java.
  • The applicant must be able to perform Data Mapping (data type conversion, schema harmonization) using Python, SQL, and Java.
  • The applicant must be familiar with and have programmed ETL interfaces (OAUTH, REST API, ODBC) using the same languages.
  • The company is looking for someone who shows an eagerness to learn and who asks concise questions when communicating with teammates.
Read more
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

icon
Remote only
icon
5 - 10 yrs
icon
₹30L - ₹50L / yr
PySpark
Data engineering
Big Data
Hadoop
Spark
+5 more

Please note - This is a 100% remote opportunity and you can work from any location.

 

About the team:

You will be a part of Cactus Labs which is the R&D Cell of Cactus Communications. Cactus Labs is a high impact cell that works to solve complex technical and business problems that help keep us strategically competitive in the industry. We are a multi-cultural team spread across multiple countries. We work in the domain of AI/ML especially with Text (NLP - Natural Language Processing), Language Understanding, Explainable AI, Big Data, AR/VR etc.

 

 

The opportunity: Within Cactus Labs you will work with the Big Data team. This team manages Terabytes of data coming from different sources. We are re-orchestrating data pipelines to handle this data at scale and improve visibility and robustness. We operate across all the three Cloud Platforms and leverage the best of them.

 

In this role, you will get to own a component end to end. You will also get to work on could platform and learn to design distributed data processing systems to operate at scale.

 

Responsibilities:

  • Build and maintain robust data processing pipelines at scale
  • Collaborate with a team of Big Data Engineers, Big Data and Cloud Architects and Domain SMEs to drive the product ahead
  • Help junior team members in designing solutions and split their user stories
  • Review team members’ code make sure standards are followed, verify unit test coverage
  • Follow best practices in building and optimize existing processes
  • Stay up to date with the progress in the domain since we work on cutting-edge technologies and are constantly trying new things out
  • Build solutions for massive scale. This requires extensive benchmarking to pick the right approach
  • Understand the data in and out and make sense of it. You will at times need to draw conclusions and present it to the business users
  • Be independent, self-driven and highly motivated. While you will have the best people to learn from and access to various courses or training materials, we expect you to take charge of your growth and learning.

 

Expectations from you:

  • 5-8 Years of relevant experience in Big Data preferable with pyspark
  • Highly proficient in distributed computing and Big Data Ecosystem - Hadoop, HDFS, Apache Spark
  • Good understanding of data lake and their importance in a Big Data Ecosystem
  • Being able to mentor junior team members and review their code
  • Experience in working in a Cloud Environment (AWS, Azure or GCP)
  • You like to work without a lot of supervision or micromanagement.
  • Above all, you get excited by data. You like to dive deep, mine patterns and draw conclusions. You believe in making data driven decisions and helping the team look for the pattern as well.

 

Preferred skills:

  • Familiarity with search engines like Elasticsearch and Bigdata warehouses systems like AWS Athena, Google Big Query etc
  • Building data pipelines using Airflow
  • Experience of working in AWS Cloud Environment
  • Knowledge of NLP and ML
Read more
Top 3 Fintech Startup
Agency job
via Jobdost by Sathish Kumar
icon
Bengaluru (Bangalore)
icon
6 - 9 yrs
icon
₹20L - ₹30L / yr
Amazon Web Services (AWS)
PySpark
SQL
Apache Spark
Python

We are looking for an exceptionally talented Lead data engineer who has exposure in implementing AWS services to build data pipelines, api integration and designing data warehouse. Candidate with both hands-on and leadership capabilities will be ideal for this position.

 

Qualification: At least a bachelor’s degree in Science, Engineering, Applied Mathematics. Preferred Masters degree

 

Job Responsibilities:

• Total 6+ years of experience as a Data Engineer and 2+ years of experience in managing a team

• Have minimum 3 years of AWS Cloud experience.

• Well versed in languages such as Python, PySpark, SQL, NodeJS etc

• Has extensive experience in Spark ecosystem and has worked on both real time and batch processing

• Have experience in AWS Glue, EMR, DMS, Lambda, S3, DynamoDB, Step functions, Airflow, RDS, Aurora etc.

• Experience with modern Database systems such as Redshift, Presto, Hive etc.

• Worked on building data lakes in the past on S3 or Apache Hudi

• Solid understanding of Data Warehousing Concepts

• Good to have experience on tools such as Kafka or Kinesis

• Good to have AWS Developer Associate or Solutions Architect Associate Certification

• Have experience in managing a team

Read more
DP
Posted by Michael Hoppitt
icon
Bengaluru (Bangalore)
icon
3 - 15 yrs
icon
Best in industry
Data Science
R Programming
Python

Kwalee is one of the world’s leading multiplatform game publishers and developers, with well over 750 million downloads worldwide for mobile hits such as Draw It, Teacher Simulator, Let’s Be Cops 3D, Traffic Cop 3D and Makeover Studio 3D. Alongside this, we also have a growing PC and Console team of incredible pedigree that is on the hunt for great new titles to join TENS!, Eternal Hope and Die by the Blade. 

With a team of talented people collaborating daily between our studios in Leamington Spa, Bangalore and Beijing, or on a remote basis from Turkey, Brazil, the Philippines and many more places, we have a truly global team making games for a global audience. And it’s paying off: Kwalee games have been downloaded in every country on earth! If you think you’re a good fit for one of our remote vacancies, we want to hear from you wherever you are based.

Founded in 2011 by David Darling CBE, a key architect of the UK games industry who previously co-founded and led Codemasters for many years, our team also includes legends such as Andrew Graham (creator of Micro Machines series) and Jason Falcus (programmer of classics including NBA Jam) alongside a growing and diverse team of global gaming experts. Everyone contributes creatively to Kwalee’s success, with all employees eligible to pitch their own game ideas on Creative Wednesdays, and we’re proud to have built our success on this inclusive principle. Could your idea be the next global hit?

What’s the job?

As a Data Scientist you will help utilise masses of data generated by Kwalee players all over the world to solve complex problems using cutting edge techniques.   

What you tell your friends you do 

"My models optimise the performance of Kwalee games and advertising every day!”

What you will really be doing 

  • Building intelligent systems which generate value from the data which our players and marketing activities produce.
  • Leveraging statistical modelling and machine learning techniques to perform automated decision making on a large scale.
  • Developing complex, multi-faceted and highly valuable data products which fuel the growth of Kwalee and our games.
  • Owning and managing data science projects from concept to deployment.
  • Collaborating with key stakeholders across the company to develop new products and avenues of research.

How you will be doing this

  • You’ll be part of an agile, multidisciplinary and creative team and work closely with them to ensure the best results.
  • You'll think creatively and be motivated by challenges and constantly striving for the best.
  • You’ll work with cutting edge technology, if you need software or hardware to get the job done efficiently, you will get it. We even have a robot!

Team

Our talented team is our signature. We have a highly creative atmosphere with more than 200 staff where you’ll have the opportunity to contribute daily to important decisions. You’ll work within an extremely experienced, passionate and diverse team, including David Darling and the creator of the Micro Machines video games.

Skills and Requirements

  • A degree in a numerically focussed degree discipline such as, Maths, Physics, Economics, Chemistry, Engineering, Biological Sciences
  • A record of outstanding contribution to data science projects.
  • Experience using Python for data analysis and visualisation.
  • A good understanding of a deep learning framework such as Tensorflow.
  • Experience manipulating data in SQL and/or NoSQL databases

We offer

  • We want everyone involved in our games to share our success, that’s why we have a generous team profit sharing scheme from day 1 of employment
  • In addition to a competitive salary we also offer private medical cover and life assurance
  • Creative Wednesdays!(Design and make your own games every Wednesday)
  • 20 days of paid holidays plus bank holidays 
  • Hybrid model available depending on the department and the role
  • Relocation support available 
  • Great work-life balance with flexible working hours
  • Quarterly team building days - work hard, play hard!
  • Monthly employee awards
  • Free snacks, fruit and drinks

Our philosophy

We firmly believe in creativity and innovation and that a fundamental requirement for a successful and happy company is having the right mix of individuals. With the right people in the right environment anything and everything is possible.

Kwalee makes games to bring people, their stories, and their interests together. As an employer, we’re dedicated to making sure that everyone can thrive within our team by welcoming and supporting people of all ages, races, colours, beliefs, sexual orientations, genders and circumstances. With the inclusion of diverse voices in our teams, we bring plenty to the table that’s fresh, fun and exciting; it makes for a better environment and helps us to create better games for everyone! This is how we move forward as a company – because these voices are the difference that make all the difference.

Read more
icon
Pune
icon
6 - 10 yrs
icon
Best in industry
Machine Learning (ML)
Data Science
Natural Language Processing (NLP)
Python
SQL
+3 more
  • 5+ years of professional experience in experiment design and applied machine learning predicting outcomes in large-scale, complex datasets.
  • Proficiency in Python, Azure ML, or other statistics/ML tools.
  • Proficiency in Deep Neural Network, Python based frameworks.
  • Proficiency in Azure DataBricks, Hive, Spark.
  • Proficiency in deploying models into production (Azure stack).
  • Moderate coding skills. SQL or similar required. C# or other languages strongly preferred.
  • Outstanding communication and collaboration skills. You can learn from and teach others.
  • Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services.
  • Experience with prediction in adversarial (energy) environments highly desirable.
  • Understanding of the model development ecosystem across platforms, including development, distribution, and best practices, highly desirable.
  • A Masters or Ph.D degree with coursework in Statistics, Data Science, Experimentation Design, and Machine Learning highly desirable


    In-person Interview- 24th Sept, Saturday- Pune Office

Read more
DP
Posted by Vishal Sharma
icon
Remote, Bengaluru (Bangalore)
icon
3 - 7 yrs
icon
₹5L - ₹10L / yr
Data Warehouse (DWH)
Spark
Data engineering
Python
PySpark
+5 more

Basic Qualifications

- Need to have a working knowledge of AWS Redshift.

- Minimum 1 year of designing and implementing a fully operational production-grade large-scale data solution on Snowflake Data Warehouse.

- 3 years of hands-on experience with building productized data ingestion and processing pipelines using Spark, Scala, Python

- 2 years of hands-on experience designing and implementing production-grade data warehousing solutions

- Expertise and excellent understanding of Snowflake Internals and integration of Snowflake with other data processing and reporting technologies

- Excellent presentation and communication skills, both written and verbal

- Ability to problem-solve and architect in an environment with unclear requirements

Read more
icon
Remote, Pune
icon
3 - 8 yrs
icon
₹4L - ₹15L / yr
Big Data
Hadoop
Java
Spark
Hibernate (Java)
+5 more
ob Title/Designation:
Mid / Senior Big Data Engineer
Job Description:
Role: Big Data EngineerNumber of open positions: 5Location: PuneAt Clairvoyant, we're building a thriving big data practice to help enterprises enable and accelerate the adoption of Big data and cloud services. In the big data space, we lead and serve as innovators, troubleshooters, and enablers. Big data practice at Clairvoyant, focuses on solving our customer's business problems by delivering products designed with best in class engineering practices and a commitment to keep the total cost of ownership to a minimum.
Must Have:
  • 4-10 years of experience in software development.
  • At least 2 years of relevant work experience on large scale Data applications.
  • Strong coding experience in Java is mandatory
  • Good aptitude, strong problem solving abilities, and analytical skills, ability to take ownership as appropriate
  • Should be able to do coding, debugging, performance tuning and deploying the apps to Prod.
  • Should have good working experience on
  • o Hadoop ecosystem (HDFS, Hive, Yarn, File formats like Avro/Parquet)
  • o Kafka
  • o J2EE Frameworks (Spring/Hibernate/REST)
  • o Spark Streaming or any other streaming technology.
  • Strong coding experience in Java is mandatory
  • Ability to work on the sprint stories to completion along with Unit test case coverage.
  • Experience working in Agile Methodology
  • Excellent communication and coordination skills
  • Knowledgeable (and preferred hands on) - UNIX environments, different continuous integration tools.
  • Must be able to integrate quickly into the team and work independently towards team goals
Role & Responsibilities:
  • Take the complete responsibility of the sprint stories' execution
  • Be accountable for the delivery of the tasks in the defined timelines with good quality.
  • Follow the processes for project execution and delivery.
  • Follow agile methodology
  • Work with the team lead closely and contribute to the smooth delivery of the project.
  • Understand/define the architecture and discuss the pros-cons of the same with the team
  • Involve in the brainstorming sessions and suggest improvements in the architecture/design.
  • Work with other team leads to get the architecture/design reviewed.
  • Work with the clients and counter-parts (in US) of the project.
  • Keep all the stakeholders updated about the project/task status/risks/issues if there are any.
Education: BE/B.Tech from reputed institute.
Experience: 4 to 9 years
Keywords: java, scala, spark, software development, hadoop, hive
Locations: Pune
Read more
DP
Posted by Shabin Belliappa
icon
Bengaluru (Bangalore)
icon
2 - 5 yrs
icon
₹10L - ₹25L / yr
Data Science
Machine Learning (ML)
Python

YOU'LL BE OUR : Data Scientist                                                            YOU'LL BE BASED AT: IBC Knowledge Park, Bangalore

YOU'LL BE ALIGNED WITH :Engineering Manager

YOU'LL BE A MEMBER OF : Data Intelligence

 

WHAT YOU'LL DO AT ATHER:

  • Work with the vehicle intelligence platform to evolve the algorithms and the platform enhancing ride experience.

  • Provide data driven solutions from simple to fairly complex insights on the data collected from the vehicle

  • Identify measures and metrics that could be used insightfully to make decisions across firmware components and productionize these.

  • Support the data science lead and manager and partner in fairly intensive projects around diagnostics, predictive modeling, BI and Engineering data sciences.

  • Build and automate scripts that could be re-used efficiently.

  • Build interactive reports/dashboards that could be re-used across engineering teams for their discussions/ explorations iteratively

  • Support monitoring, measuring the success of algorithms and features build and lead innovation through objective reasoning and thinking Engage with the data science lead and the engineering team stakeholders on the solution approach and draft a plan of action.

  • Contribute to product/team roadmap by generating and implementing innovative data and analysis based ideas as product features

  • Handhold/Guide team in successful conceptualization and implementation of key product differentiators through effective benchmarking.

 

HERE'S WHAT WE ARE LOOKING FOR :

• Good understanding of C++, Golang programming skills and system architecture understanding

• Experience with IOT, telemetry will be a plus

• Proficient in R markdown/ Python/ Grafana 

• Proficient in SQL and No-SQL

• Proficient in R  / Python programming

• Good understanding of ML techniques/ Sparks ML

 

YOU BRING TO ATHER:

• B.E/B.Tech preferably in Computer Science

•  3 to 5 yrs of work experience as Data Scientist

 

Read more
icon
Pune
icon
6 - 11 yrs
icon
₹3L - ₹18L / yr
Data Analytics
Business Intelligence (BI)
Microsoft Business Intelligence (MSBI)
Google Analytics
MongoDB
+1 more
Job Description :- 5 to 11 years experience- Experienced any of the BI tools like PowerBI, QlikView.- Writing SQLqueries/functions/procedures on big data.- Apt in databases NO SQL : mongoDB/ Cassandra OR MySQL, PostgresSQL, SQLServer- Able to Analyze clean,massage, cleanse and organize raw (big) data. - Manage Security for data on AWS or Azure cloud platform - Create, validate and maintain optimal data pipelines, assemble large, complex data sets - Helps in Structuring for upstream / downstream processing
Read more
DP
Posted by Karthik Kulkarni
icon
Bengaluru (Bangalore)
icon
3 - 7 yrs
icon
₹3L - ₹12L / yr
Python
MS-Excel
R Programming
Good knowledge of SQL , Microsoft Excel One Programming language in SAA/Python or R
Read more
icon
Mumbai
icon
3 - 7 yrs
icon
₹5L - ₹15L / yr
Machine Learning (ML)
Python
Data Science
Big Data
R Programming
+2 more
Data Scientist - We are looking for a candidate to build great recommendation engines and power an intelligent m.Paani user journey Responsibilities : - Data Mining using methods like associations, correlations, inferences, clustering, graph analysis etc. - Scale machine learning algorithm that powers our platform to support our growing customer base and increasing data volume - Design and implement machine learning, information extraction, probabilistic matching algorithms and models - Care about designing the full machine learning pipeline. - Extending company's data with 3rd party sources. - Enhancing data collection procedures. - Processing, cleaning and verifying data collected. - Ad hoc analysis of the data and present clear results. - Creating advanced analytics products that provide actionable insights. The Individual : - We are looking for a candidate with the following skills, experience and attributes: Required : - Someone with 2+ years of work experience in machine learning. - Educational qualification relevant to the role. Degree in Statistics, certificate courses in Big Data, Machine Learning etc. - Knowledge of Machine Learning techniques and algorithms. - Knowledge in languages and toolkits like Python, R, Numpy. - Knowledge of data visualization tools like D3,js, ggplot2. - Knowledge of query languages like SQL, Hive, Pig . - Familiar with Big Data architecture and tools like Hadoop, Spark, Map Reduce. - Familiar with NoSQL databases like MongoDB, Cassandra, HBase. - Good applied statistics skills like distributions, statistical testing, regression etc. Compensation & Logistics : This is a full-time opportunity. Compensation will be in line with startup, and will be based on qualifications and experience. The position is based in Mumbai, India, and the candidate must live in Mumbai or be willing to relocate.
Read more
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Our client company is into Analytics. (RF1)?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort