Data Engineer - Google Cloud Platform

at Datalicious Pty Ltd

DP
Posted by Ramjee Ganti
icon
Bengaluru (Bangalore)
icon
2 - 7 yrs
icon
₹7L - ₹20L / yr
icon
Full time
Skills
Python
Amazon Web Services (AWS)
Google Cloud Storage
Big Data
Data Analytics
Datawarehousing
Software Development
Data Science
DESCRIPTION :- We- re looking for an experienced Data Engineer to be part of our team who has a strong cloud technology experience to help our big data team to take our products to the next level.- This is a hands-on role, you will be required to code and develop the product in addition to your leadership role. You need to have a strong software development background and love to work with cutting edge big data platforms.- You are expected to bring with you extensive hands-on experience with Amazon Web Services (Kinesis streams, EMR, Redshift), Spark and other Big Data processing frameworks and technologies as well as advanced knowledge of RDBS and Data Warehousing solutions.REQUIREMENTS :- Strong background working on large scale Data Warehousing and Data processing solutions.- Strong Python and Spark programming experience.- Strong experience in building big data pipelines.- Very strong SQL skills are an absolute must.- Good knowledge of OO, functional and procedural programming paradigms.- Strong understanding of various design patterns.- Strong understanding of data structures and algorithms.- Strong experience with Linux operating systems.- At least 2+ years of experience working as a software developer or a data-driven environment.- Experience working in an agile environment.Lots of passion, motivation and drive to succeed!Highly desirable :- Understanding of agile principles specifically scrum.- Exposure to Google cloud platform services such as BigQuery, compute engine etc.- Docker, Puppet, Ansible, etc..- Understanding of digital marketing and digital advertising space would be advantageous.BENEFITS :Datalicious is a global data technology company that helps marketers improve customer journeys through the implementation of smart data-driven marketing strategies. Our team of marketing data specialists offer a wide range of skills suitable for any challenge and cover everything from web analytics to data engineering, data science and software development.Experience : Join us at any level and we promise you'll feel up-levelled in no time, thanks to the fast-paced, transparent and aggressive growth of DataliciousExposure : Work with ONLY the best clients in the Australian and SEA markets, every problem you solve would directly impact millions of real people at a large scale across industriesWork Culture : Voted as the Top 10 Tech Companies in Australia. Never a boring day at work, and we walk the talk. The CEO organises nerf-gun bouts in the middle of a hectic day.Money: We'd love to have a long term relationship because long term benefits are exponential. We encourage people to get technical certifications via online courses or digital schools.So if you are looking for the chance to work for an innovative, fast growing business that will give you exposure across a diverse range of the world's best clients, products and industry leading technologies, then Datalicious is the company for you!

About Datalicious Pty Ltd

Datalicious (An Equifax Company) is a global data analytics agency that helps marketers improve customer journeys through the implementation of smart data driven marketing strategies. Our team of marketing data specialists offers a wide range of skills suitable for any challenge and covers everything from web analytics over data engineering and data science to software development.

Founded
2007
Type
Products & Services
Size
20-100 employees
Stage
Raised funding
View full company details
Why apply to jobs via Cutshort
Personalized job matches
Stop wasting time. Get matched with jobs that meet your skills, aspirations and preferences.
Verified hiring teams
See actual hiring teams, find common social connections or connect with them directly. No 3rd party agencies here.
Move faster with AI
We use AI to get you faster responses, recommendations and unmatched user experience.
2101133
Matches delivered
3712187
Network size
15000
Companies hiring

Similar jobs

Data Engineer (Consultant/Senior consultant)

at Thoughtworks

Founded 1993  •  Products & Services  •  5000+ employees  •  Profitable
Apache Kafka
Spark
Hadoop
Apache Hive
Big Data
Scala
Python
Cloud Computing
Apache Oozie
NOSQL Databases
icon
Pune, Bengaluru (Bangalore), Coimbatore, Hyderabad, Gurugram
icon
3 - 10 yrs
icon
₹18L - ₹40L / yr

Data Engineers develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing problems. On other projects, you might be acting as the architect, leading the design of technical solutions, or perhaps overseeing a program inception to build a new product. It could also be a software delivery project where you're equally happy coding and tech-leading the team to implement the solution.



You’ll spend time on the following:

  • You will partner with teammates to create complex data processing pipelines in order to solve our clients’ most ambitious challenges
  • You will collaborate with Data Scientists in order to design scalable implementations of their models
  • You will pair to write clean and iterative code based on TDD
  • Leverage various continuous delivery practices to deploy data pipelines
  • Advise and educate clients on how to use different distributed storage and computing technologies from the plethora of options available
  • Develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
  • Create data models and speak to the tradeoffs of different modeling approaches

Here’s what we’re looking for:

 

  • You have a good understanding of data modelling and experience with data engineering tools and platforms such as Kafka, Spark, and Hadoop
  • You have built large-scale data pipelines and data-centric applications using any of the distributed storage platforms such as HDFS, S3, NoSQL databases (Hbase, Cassandra, etc.) and any of the distributed processing platforms like Hadoop, Spark, Hive, Oozie, and Airflow in a production setting
  • Hands on experience in MapR, Cloudera, Hortonworks and/or cloud (AWS EMR, Azure HDInsights, Qubole etc.) based Hadoop distributions
  • You are comfortable taking data-driven approaches and applying data security strategy to solve business problems 
  • Working with data excites you: you can build and operate data pipelines, and maintain data storage, all within distributed systems
  • Strong communication and client-facing skills with the ability to work in a consulting environment
Job posted by
sabarinath konath

Data Engineer

at Grow Your Staff

Agency job
via Grow Your Staff
Spark
Hadoop
Big Data
Data engineering
PySpark
Amazon Web Services (AWS)
Jenkins
Python
Spring Boot
Kubernetes
Docker
Systems Development Life Cycle (SDLC)
icon
Remote only
icon
5 - 9 yrs
icon
₹7L - ₹20L / yr

Grow Your Staff is looking for a Data Engineer to join an InsurTech company based out of the United States. This is a remote position.


The role will have excellent growth opportunities. You will be working directly with the team based out of the US from 5:00 PM - 3:00 AM IST (Monday - Friday).


About the client

The client is an Insurance- technology company building a marketplace infrastructure to unite insurance companies and workplace employees, retirees, and investors through a trading platform without the manual paperwork and restrictive costs.


We are looking for someone to build and improve our Enterprise Data Platform by tracking and processing our massive client database to build algorithms for our product commercials and business intelligence system, keeping in mind our customers’ data privacy needs and ensuring appropriate controls and audit-ability of our systems.


Responsibilities

  • Collaborate with colleagues on production machine-learning systems and applications.
  • Design, experiment, and evaluate software systems.
  • Write, test, and maintain production-quality code.
  • Represent us at conferences and open-source communities.
  • Demonstrate technical leadership by owning cross-team projects.
  • Build the libraries and frameworks that support fault-tolerant and testable systems.

Requirements

Experience working with:

  • Cassandra, Kafka, Hadoop (must have)
  • AWS
  • Jenkins
  • Java (Spring Boot), Python
  • Kubernetes, Docker
  • Redis, and S3 based storage
  • Agile Methodology
  • Data pipelines (e.g. Dataiku, Alteryx, Spark)
  • Monitoring technologies (Prometheus, Splunk, Humio, GUTS, Grafana, and Jupyter notebooks)
  • Software development life cycle including backend and frontend systems

Qualifications

  • Bachelors, Masters in CS, or relevant industry experience.
  • Experience building simple prototype web apps for data visualization and interaction.
  • Passionate about software engineering.
  • Curious to learn more about applied Machine Learning and Data-driven frameworks.
  • Good team player.
  • Ability to work in a fast-paced environment.

We want someone with 5 to 9 years of work experience. 

Total CTC in the range of 7,00,000-  20,00,000 LPA per annum based on experience.

Job posted by
Erica Cyril

Data Engineer

at Lifesight Technologies Pvt. Ltd.

Founded 2017  •  Product  •  20-100 employees  •  Profitable
Data engineering
Apache Spark
Spark
Big Data
Hadoop
PySpark
Python
Java
Scala
icon
Bengaluru (Bangalore)
icon
2 - 5 yrs
icon
₹10L - ₹25L / yr

Lifesight powers real-time business communications and data solutions that help companies worldwide build better applications and customer experiences by our leading customer intelligence and data platforms that helps brands and enterprises leverage identity resolution and data enrichment to power their customer data strategies like never before. Our industry-leading solution enables the transformation of customer data into actionable insights that help drive business decisions, optimize marketing spend, and improve customer experiences.

 

About The Job

 

Lifesight is growing rapidly and seeking a strong Data Engineer to be a key member of the Enterprise Data & Business Intelligence organization with the focus on data engineering services based in Bengaluru, India. You will be joining as one of the few first data engineers in our Bengaluru office and as the early engineers in the data platform. You will have an opportunity to help define our technical strategy and data engineering team culture in India. 

 

You will design and build data platforms and services, while managing our data infrastructure in cloud environments that fuels strategic business decisions across Lifesight products. 

 

A successful candidate will be a self-starter, who drives excellence, is ready to jump into a variety of big data technologies & frameworks and is able to coordinate and collaborate with other engineers, as well as mentor other engineers in the team.


Why Join us 

  • Be part of building a Zero to One SaaS product
  • Discover amazing career opportunities being initial core team member driving tech strategy of Lifesight.
  • We have a growth plan for everyone
  • Rise to rewarding challenges, enjoy attractive incentives, competitive compensation and more ...
  • Develop your skills - you will enabled with [email protected] program to help you with limitless resources for your learning
  •  

Key Responsibilities

  • Design and implement data management services for data trust, data compliance, data access and metadata management in the form of scalable and configurable while clearly articulating technical rationale behind your design and implementation choices
  • Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
  • Implementing ELT processes
  • Monitoring performance and advising any necessary infrastructure changes
  • Build data platforms and services to handle both real-time and batch processing of large data sets.
  • Take end-to-end ownership of the design and implementation of core product functionality and key technical initiatives.

 

Qualifications

  • 2+ years of data engineering experience is a fast paced company that delivers software
  • A deep understanding of designing and building highly scalable Data warehouses and Data pipelines.
  • Hands on Experience in SQL, Java/Scala programming languages.
  • Deep understanding of Apache Spark, Spark tuning , creating RDDs and building data frames. Create Java/ Scala Spark jobs for data transformation and aggregation.
  • Experience in building distributed environments using any of Kafka, Spark, Hive, Hadoop etc.
  • Good understanding of architecture and functioning of Distributed database systems 
  • Experience working with various file formats like Parquet, Avro etc for large volumes of data
  • Experience with one or more NoSQL databases
  • Strong understanding of engineering best practices and design principles

 

Location

This role will be based in our Bengaluru, India office.

 

WHAT WE CARE ABOUT

 

We like to solve problems, take initiative, pitch in when needed, and are always up for trying new things. What and how you can contribute is what’s important to us. Our consideration is not limited by the kind of education you have or the specific technologies you have experience with. Variety of technical challenges is one of the best things about working @Lifesight as an engineer, but we do not expect you to know every technology we use when you start. What we care about is that you can learn quickly, efficiently manage your goals, contribute to product strategy, and help develop your team as they solve complex problems using the best tools for the job.

 

Our culture runs much deeper than just having fun together (though, we do that well too...) – the people we want on our team are trust-builders, generous givers, scrappy problem solvers, and gritty pursuers of excellence.

 

Does this sound like you? If so, we welcome your application and the chance to meet you.

Job posted by
Anil Singh
PySpark
Python
icon
Chennai, Bengaluru (Bangalore), Mumbai, Hyderabad, Pune
icon
3 - 10 yrs
icon
₹10L - ₹25L / yr
  • Minimum 1 years of relevant experience, in PySpark (mandatory)
  • Hands on experience in development, test, deploy, maintain and improving data integration pipeline in AWS cloud environment is added plus 
  • Ability to play lead role and independently manage 3-5 member of Pyspark development team 
  • EMR ,Python and PYspark mandate.
  • Knowledge and awareness working with AWS Cloud technologies like Apache Spark, , Glue, Kafka, Kinesis, and Lambda in S3, Redshift, RDS
Job posted by
Anupama Lavanya Uppala

Database Performance Engineer

at Freelancer

Founded  •   •  employees  • 
Python
Shell Scripting
MySQL
SQL
Amazon Web Services (AWS)
Amazon VPC
Powershell
IAM
icon
Bengaluru (Bangalore)
icon
4 - 7 yrs
icon
₹20L - ₹35L / yr

   3+ years of experience in deployment, monitoring, tuning, and administration of high concurrency MySQL production databases.

  • Solid understanding of writing optimized SQL queries on MySQL databases
  • Understanding of AWS, VPC, networking, security groups, IAM, and roles.
  • Expertise in scripting in Python or Shell/Powershell
  • Must have experience in large scale data migrations
  • Excellent communication skills.
Job posted by
Nirmala Hk

Machine Learning Engineer

at SmartJoules

Founded 2015  •  Product  •  100-500 employees  •  Profitable
Machine Learning (ML)
Python
Big Data
Apache Spark
Deep Learning
icon
Remote, NCR (Delhi | Gurgaon | Noida)
icon
3 - 5 yrs
icon
₹8L - ₹12L / yr

Responsibilities:

  • Exploring and visualizing data to gain an understanding of it, then identifying differences in data distribution that could affect performance when deploying the model in the real world.
  • Verifying data quality, and/or ensuring it via data cleaning.
  • Able to adapt and work fast in producing the output which upgrades the decision making of stakeholders using ML.
  • To design and develop Machine Learning systems and schemes. 
  • To perform statistical analysis and fine-tune models using test results.
  • To train and retrain ML systems and models as and when necessary. 
  • To deploy ML models in production and maintain the cost of cloud infrastructure.
  • To develop Machine Learning apps according to client and data scientist requirements.
  • To analyze the problem-solving capabilities and use-cases of ML algorithms and rank them by how successful they are in meeting the objective.


Technical Knowledge:


  • Worked with real time problems, solved them using ML and deep learning models deployed in real time and should have some awesome projects under his belt to showcase. 
  • Proficiency in Python and experience with working with Jupyter Framework, Google collab and cloud hosted notebooks such as AWS sagemaker, DataBricks etc.
  • Proficiency in working with libraries Sklearn, Tensorflow, Open CV2, Pyspark,  Pandas, Numpy and related libraries.
  • Expert in visualising and manipulating complex datasets.
  • Proficiency in working with visualisation libraries such as seaborn, plotly, matplotlib etc.
  • Proficiency in Linear Algebra, statistics and probability required for Machine Learning.
  • Proficiency in ML Based algorithms for example, Gradient boosting, stacked Machine learning, classification algorithms and deep learning algorithms. Need to have experience in hypertuning various models and comparing the results of algorithm performance.
  • Big data Technologies such as Hadoop stack and Spark. 
  • Basic use of clouds (VM’s example EC2).
  • Brownie points for Kubernetes and Task Queues.      
  • Strong written and verbal communications.
  • Experience working in an Agile environment.
Job posted by
Saksham Dutta

Senior Data Scientist

at Opscruise

Founded 2018  •  Product  •  20-100 employees  •  Raised funding
Data Science
Python
Machine Learning (ML)
DA
Unsupervised learning
Supervised learning
icon
Remote, Chennai
icon
9 - 25 yrs
icon
₹8L - ₹25L / yr

Responsibilities

  • Research and test novel machine learning approaches for analysing large-scale distributed computing applications.
  • Develop production-ready implementations of proposed solutions across different models AI and ML algorithms, including testing on live customer data to improve accuracy,  efficacy, and robustness
  • Work closely with other functional teams to integrate implemented systems into the SaaS platform
  • Suggest innovative and creative concepts and ideas that would improve the overall platform

Qualifications

The ideal candidate must have the following qualifications:

  • 5 + years experience in practical implementation and deployment of large customer-facing ML based systems.
  • MS or M Tech (preferred) in applied mathematics/statistics;  CS or Engineering disciplines are acceptable but must have with strong quantitative and applied mathematical skills
  • In-depth working, beyond coursework, familiarity with classical and current ML techniques, both supervised and unsupervised learning techniques and algorithms
  • Implementation experiences and deep knowledge of Classification, Time Series Analysis, Pattern Recognition, Reinforcement Learning, Deep Learning, Dynamic Programming and Optimization
  • Experience in working on modeling graph structures related to spatiotemporal systems
  • Programming skills in Python is a must
  • Experience in developing and deploying on cloud (AWS or Google or Azure)
  • Good verbal and written communication skills
  • Familiarity with well-known ML frameworks such as Pandas, Keras, TensorFlow

 

Most importantly, you should be someone who is passionate about building new and innovative products that solve tough real-world problems.

Location

Chennai, India

Job posted by
sharmila M

Senior Consultant Analytics - BI

at Fortunefootprints.com

Founded 2015  •  Services  •  employees  •  Profitable
Business Intelligence (BI)
Data Science
Predictive modelling
icon
NCR (Delhi | Gurgaon | Noida)
icon
3 - 5 yrs
icon
₹12L - ₹20L / yr
As a Senior Consultant Analytics, you shall gather, analyze, identify and lead actionable data insights to solve real complex challenges. Your work will directly influence decisions taken to build the most successful web/mobile gaming platform with games played by millions across the world. Responsibilities Leverage complex data algorithms and numerical methods for forecasting, simulation, and predictive analysis, designed to optimize key decisions and metrics such as retention, engagement, and operations. Work with all teams in the company to understand, plan and drive data-based actionable decisions, championing the Data Science culture. Analyze consumer data to drive insight on how to target effectively, retain customers at low cost, and optimize engagement metrics. Develop a reporting process for all KPIs. Recruit, develop and coach talent. Champion awareness of data indicators within the organization and teach everyone to proactively identify data patterns, growth trends, and areas for improvement. Qualifications B.tech or Bachelor degree in Data science. An MBA or advanced degree in a quantitative discipline is preferred. 3+ years in Data Science and Analytics execution, with specific experience in all stages - gathering, hygiene, and analysis of data resulting in actionable insights for an end-to-end product, marketing and business operations. Knowledge of advanced analytics including but not limited to Regression, GLMs, survival modeling, predictive analytics, forecasting, machine learning, decision trees etc. Past experience in at least one Statistical and analytic tools or language expertise R, Python, SAS etc.. Expertise in analysis Algorithms, Numerical Methods and Data Tactics used to drive operational excellence, user engagement and retention. Expertise in at least one visualization platforms such as Tableau, QlikView, Spotfire and Excel. Tableau skills preferred. Advanced SQL is mandatory skills. Ability to creatively solve business problems through innovative approaches. Ability to work with various teams in a complex environment, ensuring timely delivery of multiple projects. Highly analytical with the ability to collate, analyze and present data, and drive clear insights into decisions that improve KPIs. Ability to effectively communicate and manage relationships with senior management, company divisions and partners.
Job posted by
Rupanjali Sen Gupta

Data Scientist

at Woodcutter Film Technologies Pvt. Ltd.

Founded 2018  •  Products & Services  •  0-20 employees  •  Bootstrapped
Data Science
R Programming
Python
icon
Hyderabad
icon
1 - 5 yrs
icon
₹3L - ₹6L / yr
We're an early stage film-tech startup with a mission to empower filmmakers and independent content creators with data-driven decision-making tools. We're looking for a data person to join the core team. Please get in touch if you would be excited to join us on this super exciting journey of disrupting the film production and distribution business. We are currently collaborating with Rana Daggubatt's Suresh Productions, and work out of their studio in Hyderabad - so exposure and opportunities to work on real issues faced by the media industry will be in plenty.
Job posted by
Athul Krishnan

Data Support/Analyst

at Pepr

Founded 2016  •  Products & Services  •  20-100 employees  •  Bootstrapped
Data Analytics
Statistical Analysis
MS-Excel
Excel VBA
icon
Anywhere
icon
0 - 2 yrs
icon
₹1L - ₹2L / yr
Preparing and analyzing data along with a team of data analysts and business specialists, initially in excel(google Sheet) and as your skills improve on more advanced tools as required. Will work with experts to develop best-in-class solutions and will be responsible for delivering information to support strategic decisions with data-driven insights. Will be expected to understand basic excel formula such as vlookup, if statements, Sum and Count formulae and will be required to upskill advanced excel as well as on multiple tools such as google script custom tools amongst others on your own time to progress to the required standards. Skills Required: - Experience of data audits and defect tracking tools. - Good Knowledge of Microsoft Office/Google Suits - Hands on experience on leading statistical and analytical tools and an affinity for numbers must. - Excellent verbal and written communications skills - Google script is added advantages
Job posted by
Naveen Annapureddy
Did not find a job you were looking for?
icon
Search for relevant jobs from 10000+ companies such as Google, Amazon & Uber actively hiring on Cutshort.
Get to hear about interesting companies hiring right now
iconFollow Cutshort
Want to apply to this role at Datalicious Pty Ltd?
Why apply via Cutshort?
Connect with actual hiring teams and get their fast response. No spam.
Learn more
Get to hear about interesting companies hiring right now
iconFollow Cutshort