Azure Data Engineer-ADF

Mandatory Skills: Azure Data Lake Storage, Azure SQL databases, Azure Synapse, Data Bricks (Pyspark/Spark), Python, SQL, Azure Data Factory.

Good to have: Power BI, Azure IAAS services, Azure Devops, Microsoft Fabric

Ø Very strong understanding on ETL and ELT

Ø Very strong understanding on Lakehouse architecture.

Ø Very strong knowledge in Pyspark and Spark architecture.

Ø Good knowledge in Azure data lake architecture and access controls

Ø Good knowledge in Microsoft Fabric architecture

Ø Good knowledge in Azure SQL databases

Ø Good knowledge in T-SQL

Ø Good knowledge in CI /CD process using Azure devops

Ø Power BI

ADF Developer with top Conglomerates for Kochi location_ Air India

conducting F2F Interviews on 22nd April 2023

Experience - 2-12 years.

Location - Kochi only (work from the office only)

Notice period - 1 month only.

If you are interested, please share the following information at your earliest

Company - Tekclan Software Solutions

Position – SQL Developer

Experience – Minimum 4+ years of experience in MS SQL server, SQL Programming, ETL development.

Location - Chennai

We are seeking a highly skilled SQL Developer with expertise in MS SQL Server, SSRS, SQL programming, writing stored procedures, and proficiency in ETL using SSIS. The ideal candidate will have a strong understanding of database concepts, query optimization, and data modeling.

Responsibilities:

1. Develop, optimize, and maintain SQL queries, stored procedures, and functions for efficient data retrieval and manipulation.

2. Design and implement ETL processes using SSIS for data extraction, transformation, and loading from various sources.

3. Collaborate with cross-functional teams to gather business requirements and translate them into technical specifications.

4. Create and maintain data models, ensuring data integrity, normalization, and performance.

5. Generate insightful reports and dashboards using SSRS to facilitate data-driven decision making.

6. Troubleshoot and resolve database performance issues, bottlenecks, and data inconsistencies.

7. Conduct thorough testing and debugging of SQL code to ensure accuracy and reliability.

8. Stay up-to-date with emerging trends and advancements in SQL technologies and provide recommendations for improvement.

9. Should be an independent and individual contributor.

Requirements:

1. Minimum of 4+ years of experience in MS SQL server, SQL Programming, ETL development.

2. Proven experience as a SQL Developer with a strong focus on MS SQL Server.

3. Proficiency in SQL programming, including writing complex queries, stored procedures, and functions.

4. In-depth knowledge of ETL processes and hands-on experience with SSIS.

5. Strong expertise in creating reports and dashboards using SSRS.

6. Familiarity with database design principles, query optimization, and data modeling.

7. Experience with performance tuning and troubleshooting SQL-related issues.

8. Excellent problem-solving skills and attention to detail.

9. Strong communication and collaboration abilities.

10. Ability to work independently and handle multiple tasks simultaneously.

Preferred Skills:

1. Certification in MS SQL Server or related technologies.

2. Knowledge of other database systems such as Oracle or MySQL.

3. Familiarity with data warehousing concepts and tools.

4. Experience with version control systems.

4 - 8 overall experience.

• 1-2 years’ experience in Azure Data Factory - schedule Jobs in Flows and ADF Pipelines, Performance Tuning, Error logging etc..
• 1+ years of experience with Power BI - designing and developing reports, dashboards, metrics and visualizations in Powe BI.
• (Required) Participate in video conferencing calls - daily stand-up meetings and all day working with team members on cloud migration planning, development, and support.
• Proficiency in relational database concepts & design using star, Azure Datawarehouse, and data vault.
• Requires 2-3 years of experience with SQL scripting (merge, joins, and stored procedures) and best practices.
• Knowledge on deploying and run SSIS packages in Azure.
• Knowledge of Azure Data Bricks.
• Ability to write and execute complex SQL queries and stored procedures.

We are looking for a Senior Data Engineer to join the Customer Innovation team, who will be responsible for acquiring, transforming, and integrating customer data onto our Data Activation Platform from customers’ clinical, claims, and other data sources. You will work closely with customers to build data and analytics solutions to support their business needs, and be the engine that powers the partnership that we build with them by delivering high-fidelity data assets.

In this role, you will work closely with our Product Managers, Data Scientists, and Software Engineers to build the solution architecture that will support customer objectives. You'll work with some of the brightest minds in the industry, work with one of the richest healthcare data sets in the world, use cutting-edge technology, and see your efforts affect products and people on a regular basis. The ideal candidate is someone that

• Has healthcare experience and is passionate about helping heal people,
• Loves working with data,
• Has an obsessive focus on data quality,
• Is comfortable with ambiguity and making decisions based on available data and reasonable assumptions,
• Has strong data interrogation and analysis skills,
• Defaults to written communication and delivers clean documentation, and,
• Enjoys working with customers and problem solving for them.

A day in the life at Innovaccer:

• Define the end-to-end solution architecture for projects by mapping customers’ business and technical requirements against the suite of Innovaccer products and Solutions.
• Measure and communicate impact to our customers.
• Enabling customers on how to activate data themselves using SQL, BI tools, or APIs to solve questions they have at speed.

What You Need:

• 4+ years of experience in a Data Engineering role, a Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field.
• 4+ years of experience working with relational databases like Snowflake, Redshift, or Postgres.
• Intermediate to advanced level SQL programming skills.
• Data Analytics and Visualization (using tools like PowerBI)
• The ability to engage with both the business and technical teams of a client - to document and explain technical problems or concepts in a clear and concise way.
• Ability to work in a fast-paced and agile environment.
• Easily adapt and learn new things whether it’s a new library, framework, process, or visual design concept.

What we offer:

• Industry certifications: We want you to be a subject matter expert in what you do. So, whether it’s our product or our domain, we’ll help you dive in and get certified.
• Quarterly rewards and recognition programs: We foster learning and encourage people to take risks. We recognize and reward your hard work.
• Health benefits: We cover health insurance for you and your loved ones.
• Sabbatical policy: We encourage people to take time off and rejuvenate, learn new skills, and pursue their interests so they can generate new ideas with Innovaccer.
• Pet-friendly office and open floor plan: No boring cubicles.

About Company:

Working with a multitude of clients populating the FTSE and Fortune 500s, Audit Partnership is a people focused organization with a strong belief in our employees. We hire the best people to provide the best services to our clients.

APL offers profit recovery services to organizations of all sizes across a number of sectors. APL was borne out of a desire to offer an alternative from the stagnant service provision on offer in the profit recovery industry.

Every year we cover million of pounds for our clients and also work closely with them, sharing our audit findings to minimize future losses. Our dedicated and highly experienced audit teams utilize progressive & dynamic financial service solutions & industry leading technology to achieve maximum success.

We provide dynamic work environments focused on delivering data-driven solutions at a rapidly increased pace over traditional development. Be a part of our passionate and motivated team who are excited to use the latest in software technologies within financial services.

Headquartered in the UK, we have expanded from a small team in 2002 to a market leading organization serving clients across the globe while keeping our clients at the heart of all decisions we make.

Job description:

We are looking for a high-potential, enthusiastic SQL Data Engineer with a strong desire to build a career in data analysis, database design and application solutions. Reporting directly to our UK based Technology team, you will provide support to our global operation in the delivery of data analysis, conversion, and application development to our core audit functions.

Duties will include assisting with data loads, using T-SQL to analyse data, front-end code changes, data housekeeping, data administration, and supporting the Data Services team as a whole.  Your contribution will grow in line with your experience and skills, becoming increasingly involved in the core service functions and client delivery.  A self-starter with a deep commitment to the highest standards of quality and customer service. We are offering a fantastic career opportunity, working for a leading international financial services organisation, serving the world’s largest organisations.

What we are looking for:

• 1-2 years of previous experience in a similar role
• Data analysis and conversion skills using Microsoft SQL Server is essential
• An understanding of relational database design and build
• Schema design, normalising data, indexing, query performance analysis
• Ability to analyse complex data to identify patterns and detect anomalies
• Assisting with ETL design and implementation projects
• Knowledge or experience in one or more of the key technologies below would be preferable:
  • Microsoft SQL Server (SQL Server Management Studio, Stored Procedure writing etc)
  • T-SQL
  • Programming languages (C#, VB, Python etc)
  • Use of Python to manipulate and import data
  •  
  • Experience of ETL/automation advantageous but not essential (SSIS/Prefect/Azure)
• A self-starter who can drive projects with minimal guidance
• Meeting stakeholders to agree system requirements
• Someone who is enthusiastic and eager to learn
• Very good command of English and excellent communication skills

Perks & Benefits:

• A fantastic work life balance
• Competitive compensation and benefits
• Exposure of working with Fortune 500 organization
• Expert guidance and nurture from global leaders
• Opportunities for career and personal advancement with our continued global growth strategy
• Industry leading training programs
• A working environment that is exciting, fun and engaging

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Author data services using a variety of programming languages
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centres and Azure regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Work in an Agile environment with Scrum teams.
• Ensure data quality and help in achieving data governance.

• 2+ years of experience in a Data Engineer role
• Undergraduate degree required (Graduate degree preferred) in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
• Experience using the following software/tools:
• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases
• Experience with data pipeline and workflow management tools
• Experience with Azure cloud services: ADLS, ADF, ADLA, AAS
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
• Understanding of ELT and ETL patterns and when to use each. Understanding of data models and transforming data into the models
• Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets
• Strong analytic skills related to working with unstructured datasets
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores
• Experience supporting and working with cross-functional teams in a dynamic environment

• Expertise in designing and implementing enterprise scale database (OLTP) and Data warehouse solutions.
• Hands on experience in implementing Azure SQL Database, Azure SQL Date warehouse (Azure Synapse Analytics) and big data processing using Azure Databricks and Azure HD Insight.
• Expert in writing T-SQL programming for complex stored procedures, functions, views and query optimization.
• Should be aware of Database development for both on-premise and SAAS Applications using SQL Server and PostgreSQL.
• Experience in ETL and ELT implementations using Azure Data Factory V2 and SSIS.
• Experience and expertise in building machine learning models using Logistic and linear regression, Decision tree  and Random forest Algorithms.
• PolyBase queries for exporting and importing data into Azure Data Lake.
• Building data models both tabular and multidimensional using SQL Server data tools.
• Writing data preparation, cleaning and processing steps using Python, SCALA, and R.
• Programming experience using python libraries NumPy, Pandas and Matplotlib.
• Implementing NOSQL databases and writing queries using cypher.
• Designing end user visualizations using Power BI, QlikView and Tableau.
• Experience working with all versions of SQL Server 2005/2008/2008R2/2012/2014/2016/2017/2019
• Experience using the expression languages MDX and DAX.
• Experience in migrating on-premise SQL server database to Microsoft Azure.
• Hands on experience in using Azure blob storage, Azure Data Lake Storage Gen1 and Azure Data Lake Storage Gen2.
• Performance tuning complex SQL queries, hands on experience using SQL Extended events.
• Data modeling using Power BI for Adhoc reporting.
• Raw data load automation using T-SQL and SSIS
• Expert in migrating existing on-premise database to SQL Azure.
• Experience in using U-SQL for Azure Data Lake Analytics.
• Hands on experience in generating SSRS reports using MDX.
• Experience in designing predictive models using Python and SQL Server.
• Developing machine learning models using Azure Databricks and SQL Server