Data Engineer at PayU · Remote, Bengaluru (Bangalore) · 2 - 5 years · ₹5L - ₹20L / yr (ESOP available) · Profitable · Remote friendly · Posted 4 Aug 2022
PythonRole: Data Engineer
Company: PayU
Location: Bangalore/ Mumbai
Experience : 2-5 yrs
About Company:
PayU is the payments and fintech business of Prosus, a global consumer internet group and one of the largest technology investors in the world. Operating and investing globally in markets with long-term growth potential, Prosus builds leading consumer internet companies that empower people and enrich communities.
The leading online payment service provider in 36 countries, PayU is dedicated to creating a fast, simple and efficient payment process for merchants and buyers. Focused on empowering people through financial services and creating a world without financial borders where everyone can prosper, PayU is one of the biggest investors in the fintech space globally, with investments totalling $700 million- to date. PayU also specializes in credit products and services for emerging markets across the globe. We are dedicated to removing risks to merchants, allowing consumers to use credit in ways that suit them and enabling a greater number of global citizens to access credit services.
Our local operations in Asia, Central and Eastern Europe, Latin America, the Middle East, Africa and South East Asia enable us to combine the expertise of high growth companies with our own unique local knowledge and technology to ensure that our customers have access to the best financial services.
India is the biggest market for PayU globally and the company has already invested $400 million in this region in last 4 years. PayU in its next phase of growth is developing a full regional fintech ecosystem providing multiple digital financial services in one integrated experience. We are going to do this through 3 mechanisms: build, co-build/partner; select strategic investments.
PayU supports over 350,000+ merchants and millions of consumers making payments online with over 250 payment methods and 1,800+ payment specialists. The markets in which PayU operates represent a potential consumer base of nearly 2.3 billion people and a huge growth potential for merchants.
Job responsibilities:
- Design infrastructure for data, especially for but not limited to consumption in machine learning applications
- Define database architecture needed to combine and link data, and ensure integrity across different sources
- Ensure performance of data systems for machine learning to customer-facing web and mobile applications using cutting-edge open source frameworks, to highly available RESTful services, to back-end Java based systems
- Work with large, fast, complex data sets to solve difficult, non-routine analysis problems, applying advanced data handling techniques if needed
- Build data pipelines, includes implementing, testing, and maintaining infrastructural components related to the data engineering stack.
- Work closely with Data Engineers, ML Engineers and SREs to gather data engineering requirements to prototype, develop, validate and deploy data science and machine learning solutions
Requirements to be successful in this role:
- Strong knowledge and experience in Python, Pandas, Data wrangling, ETL processes, statistics, data visualisation, Data Modelling and Informatica.
- Strong experience with scalable compute solutions such as in Kafka, Snowflake
- Strong experience with workflow management libraries and tools such as Airflow, AWS Step Functions etc.
- Strong experience with data engineering practices (i.e. data ingestion pipelines and ETL)
- A good understanding of machine learning methods, algorithms, pipelines, testing practices and frameworks
- Preferred) MEng/MSc/PhD degree in computer science, engineering, mathematics, physics, or equivalent (preference: DS/ AI)
- Experience with designing and implementing tools that support sharing of data, code, practices across organizations at scale
Shubham Vishwakarma
Full Stack Developer - Averlon
About PayU
About
Company video
Company social profiles
Similar jobs (10)
PythonJob Summary
Role Overview
We are looking for an experienced Data Engineer with strong expertise in Python, ETL, Advanced SQL, CI/CD, DevOps, and Data Analytics. The ideal candidate should have hands-on experience designing and developing scalable data pipelines, transforming large datasets, and supporting data-driven applications.
Experience with Google Cloud Platform (GCP) will be an added advantage.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and SQL.
- Develop complex and optimized SQL queries, stored procedures, and data transformations.
- Build and maintain reliable data integration workflows across multiple data sources.
- Perform data cleansing, validation, transformation, and quality checks.
- Analyze data and provide insights to support business and technical requirements.
- Implement and maintain CI/CD pipelines for data engineering applications.
- Work with DevOps practices and tools to automate deployments, monitoring, and infrastructure processes.
- Troubleshoot data pipeline failures, performance issues, and production incidents.
- Optimize data processing workflows for performance, scalability, and reliability.
- Collaborate with Data Analysts, Data Scientists, Developers, and other stakeholders.
- Follow best practices for version control, testing, documentation, and deployment.
- Contribute to cloud-based data engineering initiatives, preferably on GCP.
Required Skills
- 5–7 years of hands-on experience in Data Engineering.
- Strong programming skills in Python.
- Strong expertise in Advanced SQL and database concepts.
- Hands-on experience with ETL/ELT processes and data pipelines.
- Good understanding of Data Warehousing and Data Modeling concepts.
- Experience with CI/CD practices and tools.
- Strong understanding of DevOps principles, automation, and deployment processes.
- Strong data analytics and problem-solving skills.
- Experience working with large datasets and performance optimization.
- Good understanding of Git/version control and software development best practices.
Good to Have
- Hands-on experience with Google Cloud Platform (GCP).
- Exposure to GCP data services such as BigQuery, Cloud Storage, Dataflow, Composer, or Pub/Sub.
- Experience with containerization/orchestration technologies such as Docker/Kubernetes.
- Experience with workflow orchestration tools such as Airflow.
- Knowledge of cloud-based data architecture and distributed data processing.
Preferred Candidate Profile
- Strong analytical and problem-solving abilities.
- Good communication and stakeholder management skills.
- Ability to work independently as well as in a collaborative team environment.
- Strong ownership of data pipelines and production systems.
- Candidates who can join at short notice are preferred.
Mandatory Skills
Data Engineer, Python , ETL, GCP, Advanced SQL, Strong Data Analytics skills, CICD, Devops
PythonAbout Solutionara
Solutionara is an AI-led IT Consulting and Technology Services organization delivering enterprise transformation through Cloud, Integration, Data Engineering, Digital Experience, AI, and Enterprise Applications.
Headquartered in the United States with a Global Delivery Center (GDC) in Noida, India, Solutionara partners with enterprise organizations across retail, manufacturing, distribution, consumer goods, and other industries to design, implement, and support scalable technology solutions.
As a rapidly growing consulting organization, we foster a culture of ownership, innovation, collaboration, and continuous improvement, empowering our teams to contribute beyond their immediate responsibilities while delivering exceptional value to our global clients.
Every team member is expected to think like a consultant and business problem solver—understanding client objectives, challenging assumptions when appropriate, leveraging AI responsibly, and continuously identifying opportunities to improve quality, productivity, and customer outcomes.
AI is embedded into how we work, but it is an enabler—not a replacement for expertise, critical thinking, or accountability.
Role Overview
Solutionara is seeking a Senior Data Engineer with strong data engineering and database expertise to design, build, optimize, and support enterprise-scale data solutions.
This is a hands-on engineering position requiring strong SQL, Python, ETL/ELT, database, cloud, and troubleshooting capabilities. The engineer will work closely with technical leads and client stakeholders while taking ownership of major engineering components.
Location: Noida, India – Onsite | Employment Type: Full-Time | Experience: 6–9 Years
Key Responsibilities
Data Engineering
- Design, develop, test, deploy, and support scalable ETL/ELT pipelines.
- Build ingestion and transformation solutions across databases, APIs, files, and enterprise applications.
- Develop high-quality SQL and Python-based data processing solutions.
- Implement batch and incremental data-processing patterns.
- Develop reusable engineering components and frameworks.
- Support complex data migration and transformation activities.
- Perform source-to-target mapping and technical data analysis.
- Implement pipeline logging, monitoring, exception handling, and recovery mechanisms.
Database & Platform Engineering
- Design and optimize relational database schemas and data structures.
- Analyze and improve SQL/query performance.
- Support indexing, partitioning, storage optimization, and database performance.
- Implement appropriate database security and access controls.
- Support database monitoring, availability, backup/recovery, and operational requirements in collaboration with infrastructure/platform teams.
- Troubleshoot database and pipeline performance issues.
- Contribute to cloud data-platform configuration and optimization.
Data Quality
- Implement automated validation and data-quality controls.
- Develop reconciliation processes between source and target systems.
- Identify duplicate, incomplete, inconsistent, and invalid data.
- Implement cleansing, standardization, validation, and exception-management logic.
- Work with technical and business stakeholders to understand and implement business data rules.
Technical Delivery
- Convert technical designs into reliable production solutions.
- Participate in technical design and code reviews.
- Mentor junior/mid-level engineers.
- Support estimation and technical planning.
- Troubleshoot complex production and integration issues.
- Collaborate with technical leads, application teams, DevOps, QA, BI, and client teams.
- Document technical designs, mappings, configurations, operational procedures, and engineering decisions.
Consulting & AI-Led Working
- Understand the business purpose behind assigned engineering requirements.
- Raise risks, assumptions, dependencies, and potential improvements proactively.
- Communicate effectively with client and internal stakeholders.
- Use AI-assisted tools to improve coding, testing, analysis, documentation, and troubleshooting productivity.
- Independently verify AI-generated code and recommendations before implementation.
- Identify opportunities for engineering automation and process improvement.
Required Qualifications
- 6–9 years of experience in data engineering, database engineering, data integration, or related areas.
- Advanced SQL skills.
- Strong hands-on Python experience.
- Strong ETL/ELT development experience.
- Strong relational database knowledge, such as SQL Server, PostgreSQL, Oracle, or equivalent.
- Experience with data modeling and database design.
- Experience with AWS, Azure, or another major cloud platform.
- Experience with modern data technologies such as Databricks, Snowflake, Redshift, Synapse, Glue, ADF, Airflow, dbt, or equivalent.
- Experience with performance tuning and optimization.
- Experience with data validation and reconciliation.
- Strong troubleshooting and analytical skills.
- Ability to take ownership of engineering deliverables with limited supervision.
Preferred Qualifications
- Experience working with enterprise ERP, CRM, PIM, PLM, MES, or related systems.
- Experience with manufacturing, retail, distribution, or consumer-goods data.
- Experience with CI/CD and infrastructure/platform engineering practices.
- Experience with API and event-driven integrations.
- Previous consulting or technology-services experience.
What Success Looks Like
- Pipelines and databases perform reliably at enterprise scale.
- Engineering solutions are maintainable, reusable, and well documented.
- Data issues are detected and addressed systematically.
- Junior engineers receive effective technical guidance.
- Problems are owned through resolution rather than simply escalated.
- Engineering decisions reflect both technical requirements and client business objectives.
PythonJob Summary
We are looking for a skilled and experienced Data Engineer to join our growing data team. The ideal candidate will have strong expertise in Python, PySpark, Data Modeling, and Power BI, with hands-on experience in designing, developing, and optimizing scalable data solutions. The role requires working closely with business stakeholders, data architects, and analytics teams to build robust data pipelines and semantic models that enable data-driven decision-making.
Technical Skills
- Strong hands-on experience in Python and PySpark development.
- Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
- Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
- Experience with Power BI Data Modeling and Semantic Layer development.
- Proficiency in DAX (Data Analysis Expressions).
- Experience designing and managing Semantic Models in Power BI.
- Strong SQL skills and experience working with large datasets.
- Knowledge of data warehousing concepts and best practices.
Preferred Skills
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Exposure to modern data platforms like Databricks.
- Understanding of data governance and data quality frameworks.
PythonJob Title : Data / AI / GenAI – Engineering & Analytics
Experience : 4+ Years
Location : Noida / Gurugram / Ahmedabad
Employment Type : Full-time | On-site at client location
Working Days : 5 days per week
Working Hours : Standard business hours
Position Overview : We are seeking professionals to deliver modern data engineering, business intelligence, cloud analytics and AI-powered applications, according to their specialization.
Mandatory Skills :
Python, SQL, ETL/ELT, Data Engineering, Power BI, Azure Data Factory, Databricks, PySpark, Data Modeling, GenAI/LLMs, RAG, Vector Databases, AI Agents (as per role specialization).
Key Roles / Specializations :
- Power BI Developers : Dashboards, DAX, Power Query, reporting and data modeling
- Azure Data Engineers : Azure Data Factory, Data Lake, Azure SQL and cloud pipelines
- Databricks Engineers : Apache Spark, PySpark, Delta Lake and scalable processing
- Data Engineers : ETL/ELT, SQL, Python, data warehousing and quality
- GenAI / RAG Engineers : LLMs, prompts, embeddings, vector databases and retrieval pipelines
- Agentic AI Engineers : AI agents, tool calling, orchestration and multi-agent workflows
Key Responsibilities :
- Design, develop and maintain data pipelines, analytics solutions or AI applications.
- Build and optimize data integration, transformation and processing workflows.
- Develop dashboards and analytical models for business decision-making.
- Use cloud data platforms and modern engineering frameworks.
- Develop and evaluate LLM applications, RAG solutions or AI agents as applicable.
- Ensure data quality, security, performance and reliability.
- Collaborate with engineering teams and stakeholders on testing, deployment and documentation.
Required Skills & Qualifications :
- Bachelor’s degree in Computer Science, IT, Data Science, Engineering or related discipline.
- 4–5 years of relevant experience in data, analytics, AI or software engineering.
- Hands-on expertise in at least one listed specialization; not all technologies are required.
- Relevant Python, SQL, DAX or other technical skills depending on the position.
- Understanding of cloud platforms, APIs, integration and software practices.
Preferred Qualifications :
- Azure, Databricks or Microsoft Fabric experience.
- For GenAI roles : Azure OpenAI, OpenAI APIs, LangChain, LlamaIndex or LangGraph.
- For relevant roles : vector databases, CI/CD, Docker, MLOps or LLMOps.
Note : Candidates are expected to have expertise in their relevant specialization, not in every listed technology or role.
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
PythonData Engineer Short Hiring Post
🚨 Hiring: Data Engineer
🔹 Experience: 5–9 Years
🔹 Location: Bangalore / Hyderabad
🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling
🔹 Process: L1 Virtual → L2 F2F Karat Test
🔹 F2F: Bangalore / Hyderabad Location
🔹 Positions: Immediate requirement
⚠️ Note: Candidates must be available for F2F Karat immediately after L1.
#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners
SHARE YOUR RESUME : { pavithra.e @vysystems.com },
{ 8 9 2 5 8 0 2 4 5 7 }
Job Description – Data / Metadata Engineer
Position
Data / Metadata Engineer
Location
Pune
Experience
- 6–9 Years: Up to 28 LPA
- 10–17 Years: Up to 32 LPA
Notice Period
Immediate to 15 Days Joiners Preferred
Role Overview
We are looking for an experienced Data / Metadata Engineer with strong expertise in Data Engineering, Metadata Management, Data Modeling, Cloud technologies, Python/PySpark, and ETL/Data Pipelines.
The ideal candidate should have hands-on experience designing and developing data solutions, working with metadata and data models, and building scalable data pipelines on cloud platforms.
Mandatory Skills
- Strong experience in Data Engineering
- Hands-on experience in Metadata Management / Metadata Engineering
- Strong understanding of Data Modeling
- Experience with DDL and ER/Data Modeling
- Experience with Cloud technologies – Any Cloud Platform
- Strong programming experience in Python or PySpark
- Experience with ETL processes and Data Pipelines
- Good understanding of data architecture and data integration concepts
Key Responsibilities
- Design, develop, and maintain scalable data engineering solutions.
- Develop and manage data pipelines and ETL workflows.
- Work with metadata and data cataloging concepts.
- Create and maintain data models, DDLs, ER diagrams, and database structures.
- Develop data processing solutions using Python or PySpark.
- Work with cloud-based data platforms and services.
- Collaborate with Data Architects, Data Engineers, Developers, and Business teams.
- Ensure data quality, consistency, availability, and reliability across data pipelines.
- Troubleshoot and optimize existing data pipelines and data processes.
- Follow best practices for data engineering, modeling, and cloud-based implementations.
Candidate Profile
- Strong experience in Data Engineering + Metadata + Data Modeling.
- Good hands-on experience with Python/PySpark and ETL/Data Pipelines.
- Experience working with any major cloud platform.
- Strong understanding of databases, data structures, and data modeling concepts.
- Candidates who can join immediately or within 15 days are preferred.
Job Details
Role: Data / Metadata Engineer
Location: Pune
Experience: 6–17 Years
Budget: 28 LPA (6–9 Yrs) / 32 LPA (10–17 Yrs)
Notice Period: Immediate to 15 Days
Python1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
9+yrs
Bangalore/Hyderabad
immediate to 15days.
Python1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
Job Title: Senior Data Architect
Experience: 7+ Years
Location: India (Ahmedabad / Hybrid/Remote)
Job Summary
We are looking for an experienced Senior Data Architect to support the modernization and enhancement of Enterprise Analytic Data Warehouse (ADW). The candidate will be responsible for evaluating existing data architecture, designing scalable data models, assessing data integration across clinical, operational and financial domains, and establishing data governance and data quality standards.
Key Responsibilities
- Assess and review the existing Enterprise Analytic Data Warehouse architecture, database structures and data integration processes.
- Design and improve enterprise data models covering clinical, operational, financial and revenue-cycle information.
- Define data architecture standards, database structures, data relationships and integration strategies.
- Evaluate data flows between source systems and the data warehouse.
- Design and review source-to-target mappings and data lineage.
- Identify architectural gaps affecting data completeness, consistency, integrity and reporting accuracy.
- Establish standardized data definitions, data governance practices and metadata management.
- Ensure consistent patient, facility, payer and other critical identifiers across different datasets.
- Support integration of billing, claims, payment, denial and accounts receivable (AR) data.
- Work with SQL and Python-based data processing solutions to investigate and validate data structures.
- Recommend improvements to warehouse scalability, performance, reliability and maintainability.
- Develop a prioritized architecture and modernization roadmap for future implementation.
- Coordinate with data engineers, analytics engineers and business stakeholders.
Required Technical Skills
- Strong experience in Data Architecture and Enterprise Data Warehousing.
- Advanced SQL and relational database concepts.
- Experience with data modelling (conceptual, logical and physical).
- Knowledge of ETL/ELT pipelines and data integration.
- Experience with data profiling, data lineage and metadata management.
- Understanding of data governance, data quality and data validation frameworks.
- Experience working with large and complex datasets.
- Python knowledge for data analysis and validation.
- Experience with cloud-based data platforms or modern data warehouse technologies.
Preferred Skills
- Experience with Databricks, Azure, AWS or similar cloud data platforms.
- Experience with healthcare data warehouses.
- Knowledge of clinical, financial, billing and claims datasets.
- Understanding of HIPAA, PHI and healthcare data security principles.
- Experience with healthcare analytics, reimbursement and revenue-cycle data.
Amazon Web Services (AWS)PythonStrong Data Engineer (AWS) Profile
2
Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.
3
Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.
4
Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.
5
Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.
6
Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.
7
Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.
8
Mandatory (Education) – B.Tech / B.E./ M.Tech
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon