Min 5 to 8 years of IT experience, including experience in developing and implementing big data & Azure cloud solution Minimum 3 years of experience in cloud technology (AWS) Strong hands on knowledge on Spark (with Python as language) End to end implementation experience in data analytics solutions (data ingestion, processing, provisioning and Orchestration Strong experience in AWS ecosystem such as glue, Lambda, RDS, Redshift, IAM, S3, Shield Strong SQL and Shell script knowledge Hands on experience developing enterprise solutions using designing and building frameworks, enterprise patterns, database design and development End to end Cloud solution on AWS (glue, Lambda, RDS, Redshift, IAM ,S3, Shield) Batch solution and distributed computing using ETLELT (Spark SQL Spark Data frame ADF) Implementation of data encryption at rest and in transit DWBI (MSBI, Oracle, Sql Server), Data modelling, performance tuning, memory optimization DB partitioning Frameworks, reusable components, accelerators, CICD automation Mentor and lead a data engineering teams to design, develop, test and deploy high performance data analytics solutions Key Skills: AWS Native scripting components (Glue, RDS, Redshift) and Spark
Responsible for planning, connecting, designing, scheduling, and deploying data warehouse systems. Develops, monitors, and maintains ETL processes, reporting applications, and data warehouse design. Role and Responsibility · Plan, create, coordinate, and deploy data warehouses. · Design end user interface. · Create best practices for data loading and extraction. · Develop data architecture, data modeling, and ETFL mapping solutions within structured data warehouse environment. · Develop reporting applications and data warehouse consistency. · Facilitate requirements gathering using expert listening skills and develop unique simple solutions to meet the immediate and long-term needs of business customers. · Supervise design throughout implementation process. · Design and build cubes while performing custom scripts. · Develop and implement ETL routines according to the DWH design and architecture. · Support the development and validation required through the lifecycle of the DWH and Business Intelligence systems, maintain user connectivity, and provide adequate security for data warehouse. · Monitor the DWH and BI systems performance and integrity provide corrective and preventative maintenance as required. · Manage multiple projects at once. DESIRABLE SKILL SET · Experience with technologies such as MySQL, MongoDB, SQL Server 2008, as well as with newer ones like SSIS and stored procedures · Exceptional experience developing codes, testing for quality assurance, administering RDBMS, and monitoring of database · High proficiency in dimensional modeling techniques and their applications · Strong analytical, consultative, and communication skills; as well as the ability to make good judgment and work with both technical and business personnel · Several years working experience with Tableau, MicroStrategy, Information Builders, and other reporting and analytical tools · Working knowledge of SAS and R code used in data processing and modeling tasks · Strong experience with Hadoop, Impala, Pig, Hive, YARN, and other “big data” technologies such as AWS Redshift or Google Big Data
Job Responsibilities : As a Data Warehouse Engineer in our team, you should have a proven ability to deliver high-quality work on time and with minimal supervision. Develops or modifies procedures to solve complex database design problems, including performance, scalability, security and integration issues for various clients (on-site and off-site). Design, develop, test, and support the data warehouse solution. Adapt best practices and industry standards, ensuring top quality deliverable''s and playing an integral role in cross-functional system integration. Design and implement formal data warehouse testing strategies and plans including unit testing, functional testing, integration testing, performance testing, and validation testing. Evaluate all existing hardware's and software's according to required standards and ability to configure the hardware clusters as per the scale of data. Data integration using enterprise development tool-sets (e.g. ETL, MDM, Quality, CDC, Data Masking, Quality). Maintain and develop all logical and physical data models for enterprise data warehouse (EDW). Contributes to the long-term vision of the enterprise data warehouse (EDW) by delivering Agile solutions. Interact with end users/clients and translate business language into technical requirements. Acts independently to expose and resolve problems. Participate in data warehouse health monitoring and performance optimizations as well as quality documentation.Job Requirements : 2+ years experience working in software development & data warehouse development for enterprise analytics. 2+ years of working with Python with major experience in Red-shift as a must and exposure to other warehousing tools. Deep expertise in data warehousing, dimensional modeling and the ability to bring best practices with regard to data management, ETL, API integrations, and data governance. Experience working with data retrieval and manipulation tools for various data sources like Relational (MySQL, PostgreSQL, Oracle), Cloud-based storage.Experience with analytic and reporting tools (Tableau, Power BI, SSRS, SSAS). Experience in AWS cloud stack (S3, Glue, Red-shift, Lake Formation). Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement. Strong verbal and written communication skills with other developers and business clients. Knowledge of Logistics and/or Transportation Domain is a plus. Ability to handle/ingest very huge data sets (both real-time data and batched data) in an efficient manner.
Job Responsibilities : - Developing new data pipelines and ETL jobs for processing millions of records and it should be scalable with growth. Pipelines should be optimised to handle both real time data, batch update data and historical data. Establish scalable, efficient, automated processes for complex, large scale data analysis. Write high quality code to gather and manage large data sets (both real time and batch data) from multiple sources, perform ETL and store it in a data warehouse.Manipulate and analyse complex, high-volume, high-dimensional data from varying sources using a variety of tools and data analysis techniques. Participate in data pipelines health monitoring and performance optimisations as well as quality documentation. Interact with end users/clients and translate business language into technical requirements. Acts independently to expose and resolve problems.Job Requirements :- 2+ years experience working in software development & data pipeline development for enterprise analytics. 2+ years of working with Python with exposure to various warehousing tools In-depth working with any of commercial tools like AWS Glue, Ta-lend, Informatica, Data-stage, etc. Experience with various relational databases like MySQL, MSSql, Oracle etc. is a must. Experience with analytics and reporting tools (Tableau, Power BI, SSRS, SSAS).Experience in various DevOps practices helping the client to deploy and scale the systems as per requirement. Strong verbal and written communication skills with other developers and business client. Knowledge of Logistics and/or Transportation Domain is a plus. Hands-on with traditional databases and ERP systems like Sybase and People-soft.