Senior Developer - Spark

Owns the end to end implementation of the assigned data processing components/product features  i.e. design, development, deployment, and testing of the data processing components and associated flows conforming to best coding practices  

Creation and optimization of data engineering pipelines for analytics projects.  

Support data and cloud transformation initiatives  

Contribute to our cloud strategy based on prior experience  

Independently work with all stakeholders across the organization to deliver enhanced functionalities 

Create and maintain automated ETL processes with a special focus on data flow, error recovery, and exception handling and reporting  

Gather and understand data requirements, work in the team to achieve high-quality data ingestion and build systems that can process the data, transform the data  

Be able to comprehend the application of database index and transactions  

Involve in the design and development of a Big Data predictive analytics SaaS-based customer data platform using object-oriented analysis, design and programming skills, and design patterns  

Implement ETL workflows for data matching, data cleansing, data integration, and management  

Maintain existing data pipelines, and develop new data pipeline using big data technologies  

Responsible for leading the effort of continuously improving reliability, scalability, and stability of microservices and platform