Data Architect

Mandatory Skills: Azure Data Lake Storage, Azure SQL databases, Azure Synapse, Data Bricks (Pyspark/Spark), Python, SQL, Azure Data Factory.

Good to have: Power BI, Azure IAAS services, Azure Devops, Microsoft Fabric

Ø Very strong understanding on ETL and ELT

Ø Very strong understanding on Lakehouse architecture.

Ø Very strong knowledge in Pyspark and Spark architecture.

Ø Good knowledge in Azure data lake architecture and access controls

Ø Good knowledge in Microsoft Fabric architecture

Ø Good knowledge in Azure SQL databases

Ø Good knowledge in T-SQL

Ø Good knowledge in CI /CD process using Azure devops

Ø Power BI

 Hiring for Azure Data Engineers.

Location: Bangalore

Employment type: Full-time, permanent

website: www.amazech.com

Qualifications: 

B.E./B.Tech/M.E./M.Tech in Computer Science, Information Technology, Electrical or Electronic with good academic background.

Experience and Required Skill Sets:

•       Minimum 5 years of hands-on experience with Azure Data Lake, Azure Data Factory, SQL Data Warehouse, Azure Blob, Azure Storage Explorer

•       Experience in Data warehouse/analytical systems using Azure Synapse.

Proficient in creating Azure Data Factory pipelines for ETL processing; copy activity, custom Azure development, Synapse, etc.

•       Knowledge of Azure Data Catalog, Event Grid, Service Bus, SQL, and Purview.

•       Good technical knowledge in Microsoft SQL Server BI Suite (ETL, Reporting, Analytics, Dashboards) using SSIS, SSAS, SSRS, Power BI

•       Design and develop batch and real-time streaming of data loads to data warehouse systems

 Other Requirements:

A Bachelor's or Master's degree (Engineering or computer-related degree preferred)

Strong understanding of Software Development Life Cycles including Agile/Scrum

Responsibilities: 

•       Ability to create complex, enterprise-transforming applications that meet and exceed client expectations. 

•       Responsible for the bottom line. Strong project management abilities. Ability to encourage the team to stick to timelines.

ADF Developer with top Conglomerates for Kochi location_ Air India

conducting F2F Interviews on 22nd April 2023

Experience - 2-12 years.

Location - Kochi only (work from the office only)

Notice period - 1 month only.

If you are interested, please share the following information at your earliest

Job Overview

We are looking for a Data Engineer to join our data team to solve data-driven critical

business problems. The hire will be responsible for expanding and optimizing the existing

end-to-end architecture including the data pipeline architecture. The Data Engineer will

collaborate with software developers, database architects, data analysts, data scientists and platform team on data initiatives and will ensure optimal data delivery architecture is

consistent throughout ongoing projects. The right candidate should have hands on in

developing a hybrid set of data-pipelines depending on the business requirements.

Responsibilities

• Develop, construct, test and maintain existing and new data-driven architectures.
• Align architecture with business requirements and provide solutions which fits best
• to solve the business problems.
• Build the infrastructure required for optimal extraction, transformation, and loading
• of data from a wide variety of data sources using SQL and Azure ‘big data’
• technologies.
• Data acquisition from multiple sources across the organization.
• Use programming language and tools efficiently to collate the data.
• Identify ways to improve data reliability, efficiency and quality
• Use data to discover tasks that can be automated.
• Deliver updates to stakeholders based on analytics.
• Set up practices on data reporting and continuous monitoring
  
  

Required Technical Skills

• Graduate in Computer Science or in similar quantitative area
• 1+ years of relevant work experience as a Data Engineer or in a similar role.
• Advanced SQL knowledge, Data-Modelling and experience working with relational
• databases, query authoring (SQL) as well as working familiarity with a variety of
• databases.
• Experience in developing and optimizing ETL pipelines, big data pipelines, and datadriven
• architectures.
• Must have strong big-data core knowledge & experience in programming using Spark - Python/Scala
• Experience with orchestrating tool like Airflow or similar
• Experience with Azure Data Factory is good to have
• Build processes supporting data transformation, data structures, metadata,
• dependency and workload management.
• Experience supporting and working with cross-functional teams in a dynamic
• environment.
• Good understanding of Git workflow, Test-case driven development and using CICD
• is good to have
• Good to have some understanding of Delta tables It would be advantage if the candidate also have below mentioned experience using
• the following software/tools:
• Experience with big data tools: Hadoop, Spark, Hive, etc.
• Experience with relational SQL and NoSQL databases
• Experience with cloud data services
• Experience with object-oriented/object function scripting languages: Python, Scala, etc.

• Establish and maintain a trusted advisor relationship within the company’s IT, Commercial Digital Solutions, Functions, and Businesses you interact with
• Establish and maintain close working relationships with teams responsible for delivering solutions to the company’s businesses and functions
• Perform key management and thought leadership in the areas of advanced data techniques, including data modeling, data access, data integration, data visualization, big data solutions, text mining, data discovery, statistical methods, and database design
• Work with business partners to define ways to leverage data to develop platforms and solutions to drive business growth
• Engage collaboratively with project teams to support project objectives through the application of sound data architectural principles; support the project with knowledge of existing data assets and provide guidance on reusable data structures
• Share knowledge of external and internal data capabilities and trends, provide leadership, and facilitate the evaluation of vendors and products
• Utilize advanced data analysis, including statistical analysis and data mining techniques
• Collaborate with others to set an enterprise data vision with solid recommendations, and work to gain business and IT consensus

• Overall 15+ years of IT Environment Experience.
• Solid background as a data architect/cloud architect with a minimum of 5 years as a core architect
• Architecting experience in the cloud data warehouse solutions (Snowflake preferred, big query/redshift/synapse analytics-nice to have)
• Strong architecture and design skills using Azure Services( Like ADF/Data Flows/Evengrids/IOTHUB/EvenHub/ADLS Gen2 /Serverless Azure Functions/Logic Apps/Azure Analysis Services Cube Design patterns/Azure SQL Db)
• Working Knowledge on Lambda/Kappa frameworks within the data Lake designs& architecture solutions
• Deep understanding of the DevOps/DataOps patterns
• Architecting the semantic models
• Data modeling experience with Data vault principles
• Cloud-native Batch & Realtime ELT/ETL patterns
• Familiarity with Lucid Chart/Visio
• Logging & monitoring pattern designs in the data lake /data warehouse context
• Meta data-driven design patterns * solutioning expertise
• Data Catalog integration experience in the data lake designs
• 3+ years of experience partnering with business managers to develop technical strategies and architectures to support their objectives
• 2+ years of hands-on experience with analytics deployment in the cloud (prefer Azure but AWS knowledge is acceptable)
• 5+ years of delivering analytics in modern data architecture (Hadoop, Massively Parallel Processing Database Platforms, and Semantic Modeling)
• Demonstrable knowledge of ETL and ELT patterns and when to use either one; experience selecting among different tools that could be leveraged to accomplish this (Talend, Informatica, Azure Data Factory, SSIS, SAP Data Services)
• Demonstrable knowledge of and experience with different scripting languages (python, javascript, PIG, or object-oriented programming like Java or . NET)

• Bachelor’s degree in Computer Science, MIS, related field, or equivalent experience
• Experience working with solutions delivery teams using Agile/Scrum or similar methodologies
• 2+ years of experience designing solutions leveraging Microsoft Cortana Intelligence Suite of Products [Azure SQL, Azure SQL DW, Cosmos DB, HDInsight, DataBricks]
• Experience with enterprise systems, like CRM, ERP, Field Services Management, Supply Chain solutions, HR systems
• Ability to work independently, establishing strategic objectives, project plans, and milestones
• Exceptional written, verbal & presentation skills