Senior Technical Consultant

Job Title: AWS-Azure Data Engineer with Snowflake

Location: Bangalore, India

Experience: 4+ years

Budget: 15 to 20 LPA

Notice Period: Immediate joiners or less than 15 days

Job Description:

We are seeking an experienced AWS-Azure Data Engineer with expertise in Snowflake to join our team in Bangalore. As a Data Engineer, you will be responsible for designing, implementing, and maintaining data infrastructure and systems using AWS, Azure, and Snowflake. Your primary focus will be on developing scalable and efficient data pipelines, optimizing data storage and processing, and ensuring the availability and reliability of data for analysis and reporting.

Responsibilities:

1. Design, develop, and maintain data pipelines on AWS and Azure to ingest, process, and transform data from various sources.
2. Optimize data storage and processing using cloud-native services and technologies such as AWS S3, AWS Glue, Azure Data Lake Storage, Azure Data Factory, etc.
3. Implement and manage data warehouse solutions using Snowflake, including schema design, query optimization, and performance tuning.
4. Collaborate with cross-functional teams to understand data requirements and translate them into scalable and efficient technical solutions.
5. Ensure data quality and integrity by implementing data validation, cleansing, and transformation processes.
6. Develop and maintain ETL processes for data integration and migration between different data sources and platforms.
7. Implement and enforce data governance and security practices, including access control, encryption, and compliance with regulations.
8. Collaborate with data scientists and analysts to support their data needs and enable advanced analytics and machine learning initiatives.
9. Monitor and troubleshoot data pipelines and systems to identify and resolve performance issues or data inconsistencies.
10. Stay updated with the latest advancements in cloud technologies, data engineering best practices, and emerging trends in the industry.

Requirements:

1. Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
2. Minimum of 4 years of experience as a Data Engineer, with a focus on AWS, Azure, and Snowflake.
3. Strong proficiency in data modelling, ETL development, and data integration.
4. Expertise in cloud platforms such as AWS and Azure, including hands-on experience with data storage and processing services.
5. In-depth knowledge of Snowflake, including schema design, SQL optimization, and performance tuning.
6. Experience with scripting languages such as Python or Java for data manipulation and automation tasks.
7. Familiarity with data governance principles and security best practices.
8. Strong problem-solving skills and ability to work independently in a fast-paced environment.
9. Excellent communication and interpersonal skills to collaborate effectively with cross-functional teams and stakeholders.
10. Immediate joiner or notice period less than 15 days preferred.

If you possess the required skills and are passionate about leveraging AWS, Azure, and Snowflake to build scalable data solutions, we invite you to apply. Please submit your resume and a cover letter highlighting your relevant experience and achievements in the AWS, Azure, and Snowflake domains.

Designation: Principal Data Engineer

Experience: Experienced

Position Type: Full Time Position

Location: Hyderabad

Office Timings: 9AM to 6PM

Compensation: As Per Industry standards

About Monarch:

At Monarch, we’re leading the digital transformation of farming. Monarch Tractor augments both muscle and mind with fully loaded hardware, software, and service machinery that will spur future generations of farming technologies. With our farmer-first mentality, we are building a smart tractor that will enhance (not replace) the existing farm ecosystem, alleviate labor availability, and cost issues, and provide an avenue for competitive organic and beyond farming by providing mechanical solutions to replace harmful chemical solutions. Despite all the cutting-edge technology we will incorporate, our tractor will still plow, till, and haul better than any other tractor in its class. We have all the necessary ingredients to develop, build and scale the Monarch Tractor and digitally transform farming around the world.

Description:

Monarch Tractor likes to invite an experience Python data engineer to lead our internal data engineering team in India. This is a unique opportunity to work on computer vision AI data pipelines for electric tractors. You will be dealing with data from a farm environment like videos, images, tractor logs, GPS coordinates and map polygons. You will be responsible for collecting data for research and development. For example, this includes setting up ETL data pipelines to extract data from tractors, loading these data into the cloud and recording AI training results.

This role includes, but not limited to, the following tasks:

● Lead data engineering team

● Own and contribute to more than 50% of the data engineering code base

● Scope out new project requirements

● Costing data pipeline solutions

● Create data engineering tooling

● Design custom data structures for efficient processing of data

Data engineering skills we are looking for:

● Able to work with large amounts of text log data, image data, and video data

● Fluently use AWS cloud solutions like S3, Lambda, and EC2

● Able to work with data from Robot Operating System

Required Experience:

● 3 to 5 years of experience using Python

● 3 to 5 years of experience using PostgreSQL

● 3 to 5 years of experience using AWS EC2, S3, Lambda

● 3 to 5 years of experience using Ubuntu OS or WSL

Good to have experience:

● Ray

● Robot Operating System

What you will get:

At Monarch Tractor, you’ll play a key role on a capable, dedicated, high-performing team of rock stars. Our compensation package includes a competitive salary, excellent health, dental and vision benefits, and company equity commensurate with the role you’ll play in our success. 

Role : Senior Customer Scientist 

Experience : 6-8 Years 

Location : Chennai (Hybrid) 
 
 

Who are we? 
 
 

A young, fast-growing AI and big data company, with an ambitious vision to simplify the world’s choices. Our clients are top-tier enterprises in the banking, e-commerce and travel spaces. They use our core AI-based choice engine http://maya.ai/">maya.ai, to deliver personal digital experiences centered around taste. The http://maya.ai/">maya.ai platform now touches over 125M customers globally. You’ll find Crayon Boxes in Chennai and Singapore. But you’ll find Crayons in every corner of the world. Especially where our client projects are – UAE, India, SE Asia and pretty soon the US. 
 
 

Life in the Crayon Box is a little chaotic, largely dynamic and keeps us on our toes! Crayons are a diverse and passionate bunch. Challenges excite us. Our mission drives us. And good food, caffeine (for the most part) and youthful energy fuel us. Over the last year alone, Crayon has seen a growth rate of 3x, and we believe this is just the start. 
 

 
We’re looking for young and young-at-heart professionals with a relentless drive to help Crayon double its growth. Leaders, doers, innovators, dreamers, implementers and eccentric visionaries, we have a place for you all. 
 

Can you say “Yes, I have!” to the below? 
 
 

1. Experience with exploratory analysis, statistical analysis, and model development 
    

1. Knowledge of advanced analytics techniques, including Predictive Modelling (Logistic regression), segmentation, forecasting, data mining, and optimizations 
    

1. Knowledge of software packages such as SAS, R, Rapidminer for analytical modelling and data management. 
    

1. Strong experience in SQL/ Python/R working efficiently at scale with large data sets 
    

1. Experience in using Business Intelligence tools such as PowerBI, Tableau, Metabase for business applications 
    

Can you say “Yes, I will!” to the below? 
 
 

1. Drive clarity and solve ambiguous, challenging business problems using data-driven approaches. Propose and own data analysis (including modelling, coding, analytics) to drive business insight and facilitate decisions. 
    

1. Develop creative solutions and build prototypes to business problems using algorithms based on machine learning, statistics, and optimisation, and work with engineering to deploy those algorithms and create impact in production. 
    

1. Perform time-series analyses, hypothesis testing, and causal analyses to statistically assess the relative impact and extract trends 
    

1. Coordinate individual teams to fulfil client requirements and manage deliverable 
    

1. Communicate and present complex concepts to business audiences 
    

1. Travel to client locations when necessary  

Crayon is an equal opportunity employer. Employment is based on a person's merit and qualifications and professional competences. Crayon does not discriminate against any employee or applicant because of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, marital status, pregnancy or related.  
 
 

More about Crayon: https://www.crayondata.com/">https://www.crayondata.com/  
 

More about http://maya.ai/">maya.ai: https://maya.ai/">https://maya.ai/  

• Should act as a technical resource for the Data Science team and be involved in creating and implementing current and future Analytics projects like data lake design, data warehouse design, etc.
• Analysis and design of ETL solutions to store/fetch data from multiple systems like Google Analytics, CleverTap, CRM systems etc.
• Developing and maintaining data pipelines for real time analytics as well as batch analytics use cases.
• Collaborate with data scientists and actively work in the feature engineering and data preparation phase of model building
• Collaborate with product development and dev ops teams in implementing the data collection and aggregation solutions
• Ensure quality and consistency of the data in Data warehouse and follow best data governance practices
• Analyse large amounts of information to discover trends and patterns
• Mine and analyse data from company databases to drive optimization and improvement of product development, marketing techniques and business strategies.\

• Bachelor’s or Masters in a highly numerate discipline such as Engineering, Science and Economics
• 2-6 years of proven experience working as a Data Engineer preferably in ecommerce/web based or consumer technologies company
• Hands on experience of working with different big data tools like Hadoop, Spark , Flink, Kafka and so on
• Good understanding of AWS ecosystem for big data analytics
• Hands on experience in creating data pipelines either using tools or by independently writing scripts
• Hands on experience in scripting languages like Python, Scala, Unix Shell scripting and so on
• Strong problem solving skills with an emphasis on product development.
• Experience using business intelligence tools e.g. Tableau, Power BI would be an added advantage (not mandatory)

To be considered as a candidate for a Senior Data Engineer position, a person must have a proven track record of architecting data solutions on current and advanced technical platforms. They must have leadership abilities to lead a team providing data centric solutions with best practices and modern technologies in mind. They look to build collaborative relationships across all levels of the business and the IT organization. They possess analytic and problem-solving skills and have the ability to research and provide appropriate guidance for synthesizing complex information and extract business value. Have the intellectual curiosity and ability to deliver solutions with creativity and quality. Effectively work with business and customers to obtain business value for the requested work. Able to communicate technical results to both technical and non-technical users using effective story telling techniques and visualizations. Demonstrated ability to perform high quality work with innovation both independently and collaboratively.

ETL Developer – Talend

Job Duties:

• ETL Developer is responsible for Design and Development of ETL Jobs which follow standards,

best practices and are maintainable, modular and reusable.

• Proficiency with Talend or Pentaho Data Integration / Kettle.
• ETL Developer will analyze and review complex object and data models and the metadata

repository in order to structure the processes and data for better management and efficient

access.

• Working on multiple projects, and delegating work to Junior Analysts to deliver projects on time.
• Training and mentoring Junior Analysts and building their proficiency in the ETL process.
• Preparing mapping document to extract, transform, and load data ensuring compatibility with

all tables and requirement specifications.

• Experience in ETL system design and development with Talend / Pentaho PDI is essential.
• Create quality rules in Talend.
• Tune Talend / Pentaho jobs for performance optimization.
• Write relational(sql) and multidimensional(mdx) database queries.
• Functional Knowledge of Talend Administration Center/ Pentaho data integrator, Job Servers &

Load balancing setup, and all its administrative functions.

• Develop, maintain, and enhance unit test suites to verify the accuracy of ETL processes,

dimensional data, OLAP cubes and various forms of BI content including reports, dashboards,

and analytical models.

• Exposure in Map Reduce components of Talend / Pentaho PDI.
• Comprehensive understanding and working knowledge in Data Warehouse loading, tuning, and

maintenance.

• Working knowledge of relational database theory and dimensional database models.
• Creating and deploying Talend / Pentaho custom components is an add-on advantage.
• Nice to have java knowledge.

Skills and Qualification:

• BE, B.Tech / MS Degree in Computer Science, Engineering or a related subject.
• Having an experience of 3+ years.
• Proficiency with Talend or Pentaho Data Integration / Kettle.
• Ability to work independently.
• Ability to handle a team.
• Good written and oral communication skills.