Senior Data Engineer

Curl Analytics

Senior Data Engineer

wrackle

Company

Home

Senior Data Engineer at Curl Analytics · Bengaluru (Bangalore) · 5 - 10 years · ₹15L - ₹30L / yr · Posted 30 Mar 2022

Senior Data Engineer

at Curl Analytics

Agency job

via wrackle

5 - 10 yrs

₹15L - ₹30L / yr

Bengaluru (Bangalore)

Skills

ETL

Big Data

Data engineering

Apache Kafka

PySpark

Python

Pipeline management

Spark

Apache Hive

Docker

Kubernetes

MongoDB

SQL server

Oracle

Machine Learning (ML)

BigQuery

What you will do

Bring in industry best practices around creating and maintaining robust data pipelines for complex data projects with/without AI component
- programmatically ingesting data from several static and real-time sources (incl. web scraping)
- rendering results through dynamic interfaces incl. web / mobile / dashboard with the ability to log usage and granular user feedbacks
- performance tuning and optimal implementation of complex Python scripts (using SPARK), SQL (using stored procedures, HIVE), and NoSQL queries in a production environment

Industrialize ML / DL solutions and deploy and manage production services; proactively handle data issues arising on live apps
Perform ETL on large and complex datasets for AI applications - work closely with data scientists on performance optimization of large-scale ML/DL model training
Build data tools to facilitate fast data cleaning and statistical analysis
Ensure data architecture is secure and compliant
Resolve issues escalated from Business and Functional areas on data quality, accuracy, and availability
Work closely with APAC CDO and coordinate with a fully decentralized team across different locations in APAC and global HQ (Paris).

You should be

Expert in structured and unstructured data in traditional and Big data environments – Oracle / SQLserver, MongoDB, Hive / Pig, BigQuery, and Spark
Have excellent knowledge of Python programming both in traditional and distributed models (PySpark)
Expert in shell scripting and writing schedulers
Hands-on experience with Cloud - deploying complex data solutions in hybrid cloud / on-premise environment both for data extraction/storage and computation
Hands-on experience in deploying production apps using large volumes of data with state-of-the-art technologies like Dockers, Kubernetes, and Kafka
Strong knowledge of data security best practices
5+ years experience in a data engineering role
Science / Engineering graduate from a Tier-1 university in the country
And most importantly, you must be a passionate coder who really cares about building apps that can help people do things better, smarter, and faster even when they sleep

What you will do

Bring in industry best practices around creating and maintaining robust data pipelines for complex data projects with/without AI component
- programmatically ingesting data from several static and real-time sources (incl. web scraping)
- rendering results through dynamic interfaces incl. web / mobile / dashboard with the ability to log usage and granular user feedbacks
- performance tuning and optimal implementation of complex Python scripts (using SPARK), SQL (using stored procedures, HIVE), and NoSQL queries in a production environment

Industrialize ML / DL solutions and deploy and manage production services; proactively handle data issues arising on live apps
Perform ETL on large and complex datasets for AI applications - work closely with data scientists on performance optimization of large-scale ML/DL model training
Build data tools to facilitate fast data cleaning and statistical analysis
Ensure data architecture is secure and compliant
Resolve issues escalated from Business and Functional areas on data quality, accuracy, and availability
Work closely with APAC CDO and coordinate with a fully decentralized team across different locations in APAC and global HQ (Paris).

You should be

Expert in structured and unstructured data in traditional and Big data environments – Oracle / SQLserver, MongoDB, Hive / Pig, BigQuery, and Spark
Have excellent knowledge of Python programming both in traditional and distributed models (PySpark)
Expert in shell scripting and writing schedulers
Hands-on experience with Cloud - deploying complex data solutions in hybrid cloud / on-premise environment both for data extraction/storage and computation
Hands-on experience in deploying production apps using large volumes of data with state-of-the-art technologies like Dockers, Kubernetes, and Kafka
Strong knowledge of data security best practices
5+ years experience in a data engineering role
Science / Engineering graduate from a Tier-1 university in the country
And most importantly, you must be a passionate coder who really cares about building apps that can help people do things better, smarter, and faster even when they sleep

Users love Cutshort

Read about what our users have to say about finding their next opportunity on Cutshort.

Shubham Vishwakarma

Full Stack Developer - Averlon

I had an amazing experience. It was a delight getting interviewed via Cutshort. The entire end to end process was amazing. I would like to mention Reshika, she was just amazing wrt guiding me through the process. Thank you team.