Data Engineer

Job Location: Hyderabad/Bangalore/ Chennai/Pune/Nagpur

Notice period: Immediate - 15 days

1.      Python Developer with Snowflake

Job Description :

1. 5.5+ years of Strong Python Development Experience with Snowflake.
2. Strong hands of experience with SQL ability to write complex queries.
3. Strong understanding of how to connect to Snowflake using Python, should be able to handle any type of files
4.  Development of Data Analysis, Data Processing engines using Python
5. Good Experience in Data Transformation using Python. 
6.  Experience in Snowflake data load using Python.
7.  Experience in creating user-defined functions in Snowflake.
8.  Snowsql implementation.
9.  Knowledge of query performance tuning will be added advantage.
10. Good understanding of Datawarehouse (DWH) concepts.
11.  Interpret/analyze business requirements & functional specification
12.  Good to have DBT, FiveTran, and AWS Knowledge.

Title: Platform Engineer Location: Chennai Work Mode: Hybrid (Remote and Chennai Office) Experience: 4+ years Budget: 16 - 18 LPA

Responsibilities:

• Parse data using Python, create dashboards in Tableau.
• Utilize Jenkins for Airflow pipeline creation and CI/CD maintenance.
• Migrate Datastage jobs to Snowflake, optimize performance.
• Work with HDFS, Hive, Kafka, and basic Spark.
• Develop Python scripts for data parsing, quality checks, and visualization.
• Conduct unit testing and web application testing.
• Implement Apache Airflow and handle production migration.
• Apply data warehousing techniques for data cleansing and dimension modeling.

Requirements:

• 4+ years of experience as a Platform Engineer.
• Strong Python skills, knowledge of Tableau.
• Experience with Jenkins, Snowflake, HDFS, Hive, and Kafka.
• Proficient in Unix Shell Scripting and SQL.
• Familiarity with ETL tools like DataStage and DMExpress.
• Understanding of Apache Airflow.
• Strong problem-solving and communication skills.

Note: Only candidates willing to work in Chennai and available for immediate joining will be considered. Budget for this position is 16 - 18 LPA.

Big data Developer

Exp: 3yrs to 7 yrs.
Job Location: Hyderabad
Notice: Immediate / within 30 days

1. Expertise in building AWS Data Engineering pipelines with AWS Glue -> Athena -> Quick sight
2. Experience in developing lambda functions with AWS Lambda
3. Expertise with Spark/PySpark Candidate should be hands on with PySpark code and should be able to do transformations with Spark
4. Should be able to code in Python and Scala.
5. Snowflake experience will be a plus

We can start keeping Hadoop and Hive requirements as good to have or understanding of is enough rather than keeping it as a desirable requirement.

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

Job Overview

                         We are looking for a ML Architect to help us discover the information hidden in vast amounts of data, and help us make smarter decisions to deliver even better products. Your primary focus will be in applying data mining techniques, doing statistical analysis, and building high quality prediction systems integrated with our products. They must have strong experience using variety of data mining and data analysis methods, building and implementing models, using/creating algorithm’s and creating/running simulations.  They must be comfortable working with a wide range of stakeholders and functional teams. The right candidate will have a passion for discovering solutions hidden in large data sets and working with stakeholders to improve business outcomes. Automating to identify the textual data with their properties and structure form various type of document.

Responsibilities

• Selecting features, building and optimizing classifiers using machine learning techniques
• Data mining using state-of-the-art methods
• Enhancing data collection procedures to include information that is relevant for building analytic systems
• Processing, cleansing, and verifying the integrity of data used for analysis
• Creating automated anomaly detection systems and constant tracking of its performance
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Secure and manage when needed GPU cluster resources for events
• Write comprehensive internal feedback reports and find opportunities for improvements
• Manage GPU instances/machines to increase the performance and efficiency of the ML/DL model.

Skills and Qualifications

• Strong Hands-on experience in Python Programming
• Working experience with Computer Vision models - Object Detection Model, Image Classification
• Good experience in feature extraction, feature selection techniques and transfer learning
• Working Experience in building deep learning NLP Models for text classification, image analytics-CNN,RNN,LSTM.
• Working Experience in any of the AWS/GCP cloud platforms, exposure in fetching data from various sources.
• Good experience in exploratory data analysis, data visualisation, and other data preprocessing techniques.
• Knowledge in any one of the DL frameworks like Tensorflow, Pytorch, Keras, Caffe
• Good knowledge in statistics,distribution of data and in supervised and unsupervised machine learning algorithms.
• Exposure to OpenCV Familiarity with GPUs + CUDA
• Experience with NVIDIA software for cluster management and provisioning such as nvsm, dcgm and DeepOps.
• We are looking for a candidate with 14+ years of experience, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with AWS cloud services: EC2, RDS, AWS-Sagemaker(Added advantage)
• Experience with object-oriented/object function scripting languages in any: Python, Java, C++, Scala, etc.