7+ Data integration Jobs in Chennai | Data integration Job openings in Chennai

AWS Glue Developer 

Work Experience: 6 to 8 Years

Work Location:  Noida, Bangalore, Chennai & Hyderabad

Must Have Skills: AWS Glue, DMS, SQL, Python, PySpark, Data integrations and Data Ops, 

Job Reference ID:BT/F21/IND

Job Description:

Design, build and configure applications to meet business process and application requirements.

Responsibilities:

7 years of work experience with ETL, Data Modelling, and Data Architecture Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark Extensive experience to build data platforms on AWS using core AWS services Step function, EMR, Lambda, Glue and Athena, Redshift, Postgres, RDS etc and design/develop data engineering solutions. Orchestrate using Airflow.

Technical Experience:

Hands-on experience on developing Data platform and its components Data Lake, cloud Datawarehouse, APIs, Batch and streaming data pipeline Experience with building data pipelines and applications to stream and process large datasets at low latencies.

➢ Enhancements, new development, defect resolution and production support of Big data ETL development using AWS native services.

➢ Create data pipeline architecture by designing and implementing data ingestion solutions.

➢ Integrate data sets using AWS services such as Glue, Lambda functions/ Airflow.

➢ Design and optimize data models on AWS Cloud using AWS data stores such as Redshift, RDS, S3, Athena.

➢ Author ETL processes using Python, Pyspark.

➢ Build Redshift Spectrum direct transformations and data modelling using data in S3.

➢ ETL process monitoring using CloudWatch events.

➢ You will be working in collaboration with other teams. Good communication must.

➢ Must have experience in using AWS services API, AWS CLI and SDK

Professional Attributes:

➢ Experience operating very large data warehouses or data lakes Expert-level skills in writing and optimizing SQL Extensive, real-world experience designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloud technology.

➢ Must have 6+ years of big data ETL experience using Python, S3, Lambda, Dynamo DB, Athena, Glue in AWS environment.

➢ Expertise in S3, RDS, Redshift, Kinesis, EC2 clusters highly desired.

Qualification:

➢ Degree in Computer Science, Computer Engineering or equivalent.

Salary: Commensurate with experience and demonstrated competence

In this role, you will: 

As part of a team focused on the preserving the customer experience across the organization, this Analytic Consultant will be responsible for: 

• Understand business objectives and provide credible challenge to analysis requirements. 
• Verify sound analysis practices and data decisions were leveraged throughout planning and data sourcing phases. 
• Conduct in-depth research within complex data environments to identify data integrity issues and propose solutions to improve analysis accuracy. 
• Applying critical evaluation to challenge assumptions, formulate a defendable hypothesis, and ensuring high quality analysis results. 
• Ensure adherence to data management/ data governance regulations and policies. 
• Performing and testing highly complex data analytics for customer remediation. 
• Designing analysis projects flow and documentation that is structured for consistency, easy to understand, and to be offered to multiple levels of reviewers, partners, and regulatory agents demonstrating research and analysis completed. 
• Investigate and ensure data integrity from multiple sources. 
• Ensure data recommended and used is the best “source of truth”. 
• Applies knowledge of business, customers, and products to synthesize data to 'form a story' and align information to contrast/compare to industry perspective. Data involved typically very large, structured or unstructured, and from multiple sources. 
• Must have a strong attention to detail and be able to meet high quality standards consistently. 
• Other duties as assigned by manager. 
• Willing to assist on high priority work outside of regular business hours or weekend as needed. 

Essential Qualifications:  

• Around 5+ years in similar analytics roles 
• Bachelors, M.A./M.Sc. College Degree or Higher in applied mathematics, statistics, engineering, physics, accounting, finance, economics, econometrics, computer sciences, or business/social and behavioral sciences with a quantitative emphasis. 
• Preferred programming knowledge SQL/SAS  
• Knowledge of PVSI, Non-Lending, Student Loans, Small Business and Personal Lines and Loans is a plus. 
• Strong experience with data integration, database structures and data warehouses. 
• Persuasive written and verbal communication skills. 

Desired Qualifications: 

• Certifications in Data Science, or BI Reporting tools. 
• Ability to prioritize work, meet deadlines, achieve goals and work under pressure in a dynamic and complex environment – Soft Skills. 
• Detail oriented, results driven, and has the ability to navigate in a quickly changing and high demand environment while balancing multiple priorities. 
• Ability to research and report on a variety of issues using problem solving skills. 
• Ability to act with integrity and a high level of professionalism with all levels of team members and management. 
• Ability to make timely and independent judgment decisions while working in a fast-paced and results-driven environment. 
• Ability to learn the business aspects quickly, multitask and prioritize between projects. 
• Exhibits appropriate sense of urgency in managing responsibilities. 
• Ability to accurately process high volumes of work within established deadlines. 
• Available to flex schedule periodically based on business need. 
• Demonstrate strong negotiation, communication & presentation skills. 
• Demonstrates a high degree of reliability, integrity and trustworthiness. 
• Takes ownership of assignments and helps drive assignments of the team. 
• Dedicated, enthusiastic, driven and performance-oriented; possesses a strong work ethic and good team player. 
• Be proactive and get engaged in organizational initiatives.

Designation – Deputy Manager - TS

Job Description

1. Total of  8/9 years of development experience Data Engineering . B1/BII role
2. Minimum of 4/5 years in AWS Data Integrations and should be very good on Data modelling skills.
3. Should be very proficient in end to end AWS Data solution design, that not only includes strong data ingestion, integrations (both Data @ rest and Data in Motion) skills but also complete DevOps knowledge.
4. Should have experience in delivering at least 4 Data Warehouse or Data Lake Solutions on AWS.
5. Should be very strong experience on Glue, Lambda, Data Pipeline, Step functions, RDS, CloudFormation etc.
6. Strong Python skill .
7. Should be an expert in Cloud design principles, Performance tuning and cost modelling. AWS certifications will have an added advantage
8. Should be a team player with Excellent communication and should be able to manage his work independently with minimal or no supervision.
9. Life Science & Healthcare domain background will be a plus

Qualifications

BE/Btect/ME/MTech

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

• 4-7 years of Industry experience in IT or consulting organizations
• 3+ years of experience defining and delivering Informatica Cloud Data Integration & Application Integration enterprise applications in lead developer role
• Must have working knowledge on integrating with Salesforce, Oracle DB, JIRA Cloud  
• Must have working scripting knowledge (windows or Nodejs) 

Soft Skills 

• Superb interpersonal skills, both written and verbal, in order to effectively develop materials that are appropriate for variety of audience in business & technical teams
• Strong presentation skills, successfully present and defend point of view to Business & IT audiences
• Excellent analysis skills and ability to rapidly learn and take advantage of new concepts, business models, and technologies

• Should have good hands-on experience in Informatica MDM Customer 360, Data Integration(ETL) using PowerCenter, Data Quality.
• Must have strong skills in Data Analysis, Data Mapping for ETL processes, and Data Modeling.
• Experience with the SIF framework including real-time integration
• Should have experience in building C360 Insights using Informatica
• Should have good experience in creating performant design using Mapplets, Mappings, Workflows for Data Quality(cleansing), ETL.
• Should have experience in building different data warehouse architecture like Enterprise,
• Federated, and Multi-Tier architecture.
• Should have experience in configuring Informatica Data Director in reference to the Data
• Governance of users, IT Managers, and Data Stewards.
• Should have good knowledge in developing complex PL/SQL queries.
• Should have working experience on UNIX and shell scripting to run the Informatica workflows and to control the ETL flow.
• Should know about Informatica Server installation and knowledge on the Administration console.
• Working experience with Developer with Administration is added knowledge.
• Working experience in Amazon Web Services (AWS) is an added advantage. Particularly on AWS S3, Data pipeline, Lambda, Kinesis, DynamoDB, and EMR.
• Should be responsible for the creation of automated BI solutions, including requirements, design,development, testing, and deployment