Azure Data Architect

AWS Glue Developer 

Work Experience: 6 to 8 Years

Work Location:  Noida, Bangalore, Chennai & Hyderabad

Must Have Skills: AWS Glue, DMS, SQL, Python, PySpark, Data integrations and Data Ops, 

Job Reference ID:BT/F21/IND

Job Description:

Design, build and configure applications to meet business process and application requirements.

Responsibilities:

7 years of work experience with ETL, Data Modelling, and Data Architecture Proficient in ETL optimization, designing, coding, and tuning big data processes using Pyspark Extensive experience to build data platforms on AWS using core AWS services Step function, EMR, Lambda, Glue and Athena, Redshift, Postgres, RDS etc and design/develop data engineering solutions. Orchestrate using Airflow.

Technical Experience:

Hands-on experience on developing Data platform and its components Data Lake, cloud Datawarehouse, APIs, Batch and streaming data pipeline Experience with building data pipelines and applications to stream and process large datasets at low latencies.

➢ Enhancements, new development, defect resolution and production support of Big data ETL development using AWS native services.

➢ Create data pipeline architecture by designing and implementing data ingestion solutions.

➢ Integrate data sets using AWS services such as Glue, Lambda functions/ Airflow.

➢ Design and optimize data models on AWS Cloud using AWS data stores such as Redshift, RDS, S3, Athena.

➢ Author ETL processes using Python, Pyspark.

➢ Build Redshift Spectrum direct transformations and data modelling using data in S3.

➢ ETL process monitoring using CloudWatch events.

➢ You will be working in collaboration with other teams. Good communication must.

➢ Must have experience in using AWS services API, AWS CLI and SDK

Professional Attributes:

➢ Experience operating very large data warehouses or data lakes Expert-level skills in writing and optimizing SQL Extensive, real-world experience designing technology components for enterprise solutions and defining solution architectures and reference architectures with a focus on cloud technology.

➢ Must have 6+ years of big data ETL experience using Python, S3, Lambda, Dynamo DB, Athena, Glue in AWS environment.

➢ Expertise in S3, RDS, Redshift, Kinesis, EC2 clusters highly desired.

Qualification:

➢ Degree in Computer Science, Computer Engineering or equivalent.

Salary: Commensurate with experience and demonstrated competence

Are you a motivated, organized person seeking a demanding and rewarding opportunity in a fast-paced environment? Would you enjoy being part of a dedicated team that works together to create a relevant, memorable difference in the lives of our customers and employees? If you're looking for change, and you're ready to make changes … we're looking for you.

This role is part of our global Team and will be responsible for driving our digitalization roadmap. You will be responsible for analyzing reporting requirements and defining solutions that meet or exceed those requirements. You will need to understand and apply systems analysis concepts and principles to effectively translate and validate business systems solutions. Further, you will apply IT and internal team methodologies and procedures to ensure solutions are defined in a consistent, standard and repeatable method.

Responsibilities

What are you accountable for achieving as Senior Oracle Fusion Reporting Specialist?

• Design, Development and Support of Oracle Reporting applications and Dashboards.
• Interact with internal stakeholders and translate business needs to technical specifications
• Preparing BIP reports (Data Model and Report Templates)
• Effectively deliver projects and ongoing support for Oracle HCM BI solutions.
• Develop data models and reports.

Requirements

What will you need as a successful Senior Oracle Fusion Reporting Specialist / Developer?

• Bachelor's Degree in Computer Science or more than 2 years of experience in business intelligence projects
• Experience in programming with Oracle tools and in writing SQL Server/Oracle/SQL Stored Procedures and functions.
• Experience in a BI environment.
• Broad understanding of Oracle HCM Cloud Applications and database structure of the HCM application module.
• Exposure to Oracle BI, Automation, JIRA and ETL will be added advantage.

• We are looking for : Data engineer
• Sprak
• Scala
• Hadoop

• Play a critical role as a member of the leadership team in shaping and supporting our overall company vision, day-to-day operations, and culture.
• Set the technical vision and build the technical product roadmap from launch to scale; including defining long-term goals and strategies
• Define best practices around coding methodologies, software development, and quality assurance
• Define innovative technical requirements and systems while balancing time, feasibility, cost and customer experience
• Build and support production products
• Ensure our internal processes and services comply with privacy and security regulations
• Establish a high performing, inclusive engineering culture focused on innovation, execution, growth and development
• Set a high bar for our overall engineering practices in support of our mission and goals
• Develop goals, roadmaps and delivery dates to help us scale quickly and sustainably
• Collaborate closely with Product, Business, Marketing and Data Science
• Experience with financial and transactional systems
• Experience engineering for large volumes of data at scale
• Experience with financial audit and compliance is a plus
• Experience building a successful consumer facing web and mobile apps at scale

• Participate in planning, implementation of solutions, and transformation programs from legacy system to a cloud-based system
• Work with the team on Analysis, High level and low-level design for solutions using ETL or ELT based solutions and DB services in RDS
• Work closely with the architect and engineers to design systems that effectively reflect business needs, security requirements, and service level requirements
• Own deliverables related to design and implementation
• Own Sprint tasks and drive the team towards the goal while understanding the change and release process defined by the organization.
• Excellent communication skills, particularly those relating to complex findings and presenting them to ensure audience appeal at various levels of the organization
• Ability to integrate research and best practices into problem avoidance and continuous improvement
• Must be able to perform as an effective member in a team-oriented environment, maintain a positive attitude, and achieve desired results while working with minimal supervision

Basic Qualifications:

• Minimum of 5+ years of technical work experience in the implementation of complex, large scale, enterprise-wide projects including analysis, design, core development, and delivery
• Minimum of 3+ years of experience with expertise in Informatica ETL, Informatica Power Center, and Informatica Data Quality
• Experience with Informatica MDM tool is good to have
• Should be able to understand the scope of the work and ask for clarifications
• Should have advanced SQL skills. Including complex PL/SQL coding skills
• Knowledge of Agile is plus
• Well-versed with SOAP, Webservice, and REST API.
• Hand on development using Java would be a plus.

• Hands-on experience in Development
• 4-6 years of Hands on experience with Python scripts
• 2-3 years of Hands on experience in PySpark coding. Worked in spark cluster computing technology.
• 3-4 years of Hands on end to end data pipeline experience working on AWS environments
• 3-4 years of Hands on experience working on AWS services – Glue, Lambda, Step Functions, EC2, RDS, SES, SNS, DMS, CloudWatch etc.
• 2-3 years of Hands on experience working on AWS redshift
• 6+ years of Hands on experience with writing Unix Shell scripts
• Good communication skills

• Insurance P&C and Specialty domain experience a plus
• Experience in a cloud-based architecture preferred, such as Databricks, Azure Data Lake, Azure Data Factory, etc.
• Strong understanding of ETL fundamentals and solutions. Should be proficient in writing advanced / complex SQL, expertise in performance tuning and optimization of SQL queries required.
• Strong experience in Python/PySpark and Spark SQL
• Experience in troubleshooting data issues, analyzing end to end data pipelines, and working with various teams in resolving issues and solving complex problems.
• Strong experience developing Spark applications using PySpark and SQL for data extraction, transformation, and aggregation from multiple formats for analyzing & transforming the data to uncover insights and actionable intelligence for internal and external use