Data Engineering Lead

Required Qualifications: 

∙Bachelor’s degree in computer science, Information Technology, or related field, or equivalent experience. 

∙5+ years of experience in a DevOps role, preferably for a SaaS or software company. 

∙Expertise in cloud computing platforms (e.g., AWS, Azure, GCP). 

∙Proficiency in scripting languages (e.g., Python, Bash, Ruby). 

∙Extensive experience with CI/CD tools (e.g., Jenkins, GitLab CI, Travis CI). 

∙Extensive experience with NGINX and similar web servers. 

∙Strong knowledge of containerization and orchestration technologies (e.g., Docker, Kubernetes). 

∙Familiarity with infrastructure-as-code tools (e.g. Terraform, CloudFormation). 

∙Ability to work on-call as needed and respond to emergencies in a timely manner. 

∙Experience with high transactional e-commerce platforms.

Preferred Qualifications: 

∙Certifications in cloud computing or DevOps are a plus (e.g., AWS Certified DevOps Engineer, 

Azure DevOps Engineer Expert). 

∙Experience in a high availability, 24x7x365 environment. 

∙Strong collaboration, communication, and interpersonal skills. 

∙Ability to work independently and as part of a team.

Intuitive cloud (http://www.intuitive.cloud">www.intuitive.cloud) is one of the fastest growing top-tier Cloud Solutions and SDx Engineering solution and service company supporting 80+ Global Enterprise Customer across Americas, Europe and Middle East.

Intuitive is a recognized professional and manage service partner for core superpowers in cloud(public/ Hybrid), security, GRC, DevSecOps, SRE, Application modernization/ containers/ K8 -as-a- service and cloud application delivery.

• 9+ years’ experience as data engineer.
• Must have 4+ Years in implementing data engineering solutions with Databricks.
• This is hands on role building data pipelines using Databricks. Hands-on technical experience with Apache Spark.
• Must have deep expertise in one of the programming languages for data processes (Python, Scala). Experience with Python, PySpark, Hadoop, Hive and/or Spark to write data pipelines and data processing layers
• Must have worked with relational databases like Snowflake. Good SQL experience for writing complex SQL transformation.
• Performance Tuning of Spark SQL running on S3/Data Lake/Delta Lake/ storage and Strong Knowledge on Databricks and Cluster Configurations.
• Hands on architectural experience
• Nice to have Databricks administration including security and infrastructure features of Databricks.

Specific Responsibilities

• Minimum of 2 years Experience in Google Big Query and Google Cloud Platform.
• Design and develop the ETL framework using BigQuery
• Expertise in Big Query concepts like Nested Queries, Clustering, Partitioning, etc.
• Working Experience of Clickstream database, Google Analytics/ Adobe Analytics.
• Should be able to automate the data load from Big Query using APIs or scripting language.
• Good experience in Advanced SQL concepts.
• Good experience with Adobe launch Web, Mobile & e-commerce tag implementation.
• Identify complex fuzzy problems, break them down in smaller parts, and implement creative, data-driven solutions
• Responsible for defining, analyzing, and communicating key metrics and business trends to the management teams
• Identify opportunities to improve conversion & user experience through data. Influence product & feature roadmaps.
• Must have a passion for data quality and be constantly looking to improve the system. Drive data-driven decision making through the stakeholders & drive Change Management
• Understand requirements to translate business problems & technical problems into analytics problems.
• Effective storyboarding and presentation of the solution to the client and leadership.
• Client engagement & management
• Ability to interface effectively with multiple levels of management and functional disciplines.
• Assist in developing/coaching individuals technically as well as on soft skills during the project and as part of Client Project’s training program.

• 2 to 3 years of working experience in Google Big Query & Google Cloud Platform
• Relevant experience in Consumer Tech/CPG/Retail industries
• Bachelor’s in engineering, Computer Science, Math, Statistics or related discipline
• Strong problem solving and web analytical skills. Acute attention to detail.
• Experience in analyzing large, complex, multi-dimensional data sets.
• Experience in one or more roles in an online eCommerce or online support environment.

• Expertise in Google Big Query & Google Cloud Platform
• Experience in Advanced SQL, Scripting language (Python/R)
• Hands-on experience in BI tools (Tableau, Power BI)
• Working Experience & understanding of Adobe Analytics or Google Analytics
• Experience in creating and debugging website & app tracking (Omnibus, Dataslayer, GA debugger, etc.)
• Excellent analytical thinking, analysis, and problem-solving skills.
• Knowledge of other GCP services is a plus

Datametica is looking for talented Big Query engineers

Total Experience - 2+ yrs.

Notice Period – 0 - 30 days

Work Location – Pune, Hyderabad

Job Description:

• Sound understanding of Google Cloud Platform Should have worked on Big Query, Workflow, or Composer
• Experience in migrating to GCP and integration projects on large-scale environments ETL technical design, development, and support
• Good SQL skills and Unix Scripting Programming experience with Python, Java, or Spark would be desirable.
• Experience in SOA and services-based data solutions would be advantageous

About the Company: 

www.datametica.com

Datametica is amongst one of the world's leading Cloud and Big Data analytics companies.

Datametica was founded in 2013 and has grown at an accelerated pace within a short span of 8 years. We are providing a broad and capable set of services that encompass a vision of success, driven by innovation and value addition that helps organizations in making strategic decisions influencing business growth.

Datametica is the global leader in migrating Legacy Data Warehouses to the Cloud. Datametica moves Data Warehouses to Cloud faster, at a lower cost, and with few errors, even running in parallel with full data validation for months.

Datametica's specialized team of Data Scientists has implemented award-winning analytical models for use cases involving both unstructured and structured data.

Datametica has earned the highest level of partnership with Google, AWS, and Microsoft, which enables Datametica to deliver successful projects for clients across industry verticals at a global level, with teams deployed in the USA, EU, and APAC.

Recognition:

We are gratified to be recognized as a Top 10 Big Data Global Company by CIO story.

If it excites you, please apply.

• The incumbent should have hands on experience in data engineering and GCP data technologies.

• Should Work with client teams to design and implement modern, scalable data solutions using a range of new and emerging technologies from the Google Cloud Platform.

• Should Work with Agile and DevOps techniques and implementation approaches in the delivery.

• Showcase your GCP Data engineering experience when communicating with clients on their requirements, turning these into technical data solutions.

• Build and deliver Data solutions using GCP products and offerings.
• Have hands on Experience on Python 
Experience on SQL or MySQL. Experience on Looker is an added advantage.

ears of Exp: 3-6+ Years 
Skills: Scala, Python, Hive, Airflow, Spark

Languages: Java, Python, Shell Scripting

GCP: BigTable, DataProc,  BigQuery, GCS, Pubsub

OR
AWS: Athena, Glue, EMR, S3, Redshift

MongoDB, MySQL, Kafka

Platforms: Cloudera / Hortonworks
AdTech domain experience is a plus.
Job Type - Full Time 

• 5+ years of experience in a Data Engineer role
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases such as Cassandra.
• Experience with AWS cloud services: EC2, EMR, Athena
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Advanced SQL knowledge and experience working with relational databases, query authoring (SQL) as well as familiarity with unstructured datasets.
• Deep problem-solving skills to perform root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

Problem Formulation: Identifies possible options to address the business problems and must possess good understanding of dimension modelling

Must have worked on at least one end to end project using any Cloud Datawarehouse (Azure Synapses, AWS Redshift, Google Big query)

Good to have an understand of POWER BI and integration with any Cloud services like Azure or GCP

Experience of working with SQL Server, SSIS(Preferred)

Applied Business Acumen: Supports the development of business cases and recommendations. Owns delivery of project activity and tasks assigned by others. Supports process updates and changes. Solves business issues.

Data Transformation/Integration/Optimization:

The ETL developer is responsible for designing and creating the Data warehouse and all related data extraction, transformation and load of data function in the company

The developer should provide the oversight and planning of data models, database structural design and deployment and work closely with the data architect and Business analyst

Duties include working in a cross functional software development teams (Business analyst, Testers, Developers) following agile ceremonies and development practices.

The developer plays a key role in contributing to the design, evaluation, selection, implementation and support of databases solution.

Development and Testing: Develops codes for the required solution by determining the appropriate approach and leveraging business, technical, and data requirements.

Creates test cases to review and validate the proposed solution design. Work on POCs and deploy the software to production servers.

Good to Have (Preferred Skills):

• Minimum 4-8 Years of experience in Data warehouse design and development for large scale application
• Minimum 3 years of experience with star schema, dimensional modelling and extract transform load (ETL) Design and development
• Expertise working with various databases (SQL Server, Oracle)
• Experience developing Packages, Procedures, Views and triggers
• Nice to have Big data technologies
• The individual must have good written and oral communication skills.
• Nice to have SSIS

Education and Experience

• Minimum 4-8 years of software development experience
• Bachelor's and/or Master’s degree in computer science

Please revert back with below details.

Total Experience:
Relevant Experience:

Current CTC:
Expected CTC:

Any offers: Y/N

Notice Period:

Qualification:

DOB:
Present Company Name:

Designation:

Domain

Reason for job change:

Current Location:

CANDIDATE WILL BE DEPLOYED IN A FINANCIAL CAPTIVE ORGANIZATION @ PUNE (KHARADI)

Below are the job Details :-

Experience 10 to 18 years

Mandatory skills –

• data migration,
• data flow

The ideal candidate for this role will have the below experience and qualifications:  

• Experience of building a range of Services in a Cloud Service provider (ideally GCP)  
• Hands-on design and development of Google Cloud Platform (GCP), across a wide range of GCP services including hands on experience of GCP storage & database technologies. 
• Hands-on experience in architecting, designing or implementing solutions on GCP, K8s, and other Google technologies. Security and Compliance, e.g. IAM and cloud compliance/auditing/monitoring tools 
• Desired Skills within the GCP stack - Cloud Run, GKE, Serverless, Cloud Functions, Vision API, DLP, Data Flow, Data Fusion 
• Prior experience of migrating on-prem applications to cloud environments. Knowledge and hands on experience on Stackdriver, pub-sub, VPC, Subnets, route tables, Load balancers, firewalls both for on premise and the GCP.  
• Integrate, configure, deploy and manage centrally provided common cloud services (e.g. IAM, networking, logging, Operating systems, Containers.)  
• Manage SDN in GCP Knowledge and experience of DevOps technologies around Continuous Integration & Delivery in GCP using Jenkins.  
• Hands on experience of Terraform, Kubernetes, Docker, Stackdriver, Terraform  
• Programming experience in one or more of the following languages: Python, Ruby, Java, JavaScript, Go, Groovy, Scala  
• Knowledge or experience in DevOps tooling such as Jenkins, Git, Ansible, Splunk, Jira or Confluence, AppD, Docker, Kubernetes  
• Act as a consultant and subject matter expert for internal teams to resolve technical deployment obstacles, improve product's vision. Ensure compliance with centrally defined Security 
• Financial experience is preferred 
• Ability to learn new technologies and rapidly prototype newer concepts 
• Top-down thinker, excellent communicator, and great problem solver

Exp:- 10  to 18 years

Location:- Pune

Candidate must have experience in below.

• GCP Data Platform
• Data Processing:- Data Flow, Data Prep, Data Fusion
• Data Storage:- Big Query, Cloud Sql,
• Pub Sub, GCS Bucket