We are looking to add a Lead BigData Engineer to our team, Would love to connect and tell you more about what were building, and to learn more about what interests you.
Company Name : PayU - Kindly visit our company site for more details .
Location : Bangalore, Mumbai & Gurgaon
Role and Background Information:
As a Lead/Manager - Data Engineering, you will execute the strategy and roadmap for data engineering at PayU, you will define, evolve and mature PayU's existing data platform and processing framework that handles terabytes of data from our various properties.
You will work with a team of data engineers, analytics and data scientist teams to identify the needs and contribute towards building the next generation data ecosystem.
You will engage with analysts and leaders to research and develop new data engineering capabilities.
Responsible to ingest data from files, streams and databases. Process the data with Python and Pyspark and its storage into time-series database.
Develop programs in Python as part of data extraction, data cleaning, transformation and processing. Develop and maintain scalable data pipelines.
Rest API’s development.
What you’d need to bring to the table:
Overall 5-10 years of experience as Data Engineer.
Advanced working SQL knowledge to create complex queries.
Experience in working with Time-series Database, relational databases as well as working familiarity with a variety of databases (structured & Unstructured).
Hands On experience on visualization tools like Grafana & Power BI.
Experience in designing and implementing scalable architecture.
Good experience in doing object-oriented programming in python.
Very strong in Object-Oriented Analysis and Design (OOAD).
Strong knowledge on REST APIs.
Experience working on Azure Cloud services IaaS PaaS.
Hands on Experience in working on Microsoft Azure Services like ADLS/Blob Storage solutions, Event Hubs, Service Bus, scale sets, Load Balancers, Azure Functions, Databricks. - Hands on Experience in working on Kafka.
Knowledge on continuous integration/continuous deployment.
Experience of data migration and deployment from On-Prem to Cloud environment and vice-versa.
Bachelor of Engineering (IIT/NIT/Bits is Preferred)