data engineer at Codnatives · Hyderabad, Bengaluru (Bangalore) · 5 - 10 years · ₹4L - ₹16L / yr · Profitable · Posted 17 Aug 2026

Skills Referential (Required knowledge, skills and abilities)
Technical Skills:
Python
Pyspark
SQL
ETL Aws, Azure, gcp

Similar jobs (10)
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
1st virtual , 2nd round F2F -
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
Bang/hyderabad
immediate to 15days.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
role: data engineer
Python pyspark, SQL, data engineer
5+yrs
Bangalore/Hyderabad
immediate to 15days.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
1st virtual , 2nd round F2F
Python pyspark, SQL, data engineer
5+yrs
9+yrs
Bangalore/Hyderabad
immediate to 15days.
GCP Data engineer
Exp 9+yrs
Bangalore/Hyderabad
Lead exp+ Python + pyspark + sql or plsql
immediate to 15days joiners
Call or WhatsApp: 6 3 8 5 0 7 6 1 4 2
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
🚀 Hiring: Senior GCP Data Engineer
We are looking for experienced Senior Data Engineers with strong hands-on expertise in Google Cloud Platform (GCP), Python, PySpark, and SQL. Candidates with team lead experience will be preferred.
📌 Job Details:
- Role: Senior GCP Data Engineer
- Experience: 8+ Years
- Budget: Up to ₹30 LPA
- Location: Bangalore / Hyderabad
- Relocation: Candidates from Chennai and Pune willing to relocate are welcome.
- Notice Period: Immediate to 15 days
- Interview Process: L1 – Virtual | L2 / DM – Face-to-Face (F2F)
💻 Required Technical Skills:
- Strong hands-on experience with GCP data engineering.
- Excellent proficiency in Python, PySpark, and SQL.
- Experience in designing, developing, and optimizing data pipelines.
- Good understanding of ETL/ELT processes and large-scale data processing.
- Experience working with cloud-based data platforms and data warehousing.
- Strong analytical, problem-solving, and communication skills.
⭐ Preferred Skills:
- Previous experience as a Technical Lead or leading a data engineering team.
- Knowledge of GCP services such as BigQuery, Dataproc, Dataflow, and Cloud Storage.
Applications on this job are not being reviewed actively. You may apply but please follow up with the company via other channels too.
About Us:
The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.
Job Summary:
We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.
As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.
Key Responsibilities:
- Design, develop, test, and maintain optimal data pipeline and ETL architectures.
- Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
- Prepare and optimize data for predictive and prescriptive modeling.
- Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
- Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
- Utilize big data tools and frameworks to optimize data acquisition and preparation.
- Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
- Develop and curate data models for analytics, dashboards, and reports.
- Conduct code reviews, maintain production-level code, and implement testing approaches.
- Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
- Drive innovation and implement efficient new approaches to data engineering tasks.
Must-Have Skills:
- Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
- 5+ years of experience working with enterprise data platforms, including building and managing data lakes.
- 3–5 years of experience designing and implementing data warehouse solutions.
- Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
- Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
- Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
- Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
- Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
- Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
- Strong problem-solving, communication, and collaboration skills.
Good-to-Have Skills:
- Experience in integrating ERP data into data lakes.
- Experience with traditional ETL tools (e.g., Talend, Pentaho).
Competencies:
- Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
- Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
- Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
- Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
- Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.
Why Join Us?
- Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
- Work on impactful projects that make a difference across industries.
- Opportunities for professional growth and continuous learning.
- Competitive salary and benefits package.
Application Details
Ready to make an impact? Apply today and become part of the QX Impact team!
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python and PySpark.
- Work extensively with Databricks and Delta Lake, including Delta tables, MERGE operations, schema evolution, Time Travel, and data optimization.
- Develop, schedule, and monitor Apache Airflow DAGs for workflow orchestration and dependency management.
- Work with Kubernetes for containerized application deployment, orchestration, monitoring, and troubleshooting.
- Develop reusable and optimized Python code following software engineering best practices.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Implement data quality checks, error handling, logging, and monitoring mechanisms.
- Use Git for version control, branching, pull requests, code reviews, and collaborative development.
- Troubleshoot pipeline failures, investigate data issues, and implement reliable solutions.
- Collaborate with cross-functional teams to understand business requirements and deliver efficient data solutions.
Required Technical Skills
- Python: Strong programming skills, including data manipulation, exception handling, modular programming, and debugging.
- PySpark: Hands-on experience with Spark DataFrames, Spark SQL, joins, transformations, partitioning, and performance tuning.
- Databricks: Strong understanding of Databricks architecture, notebooks, workflows, compute management, and job execution.
- Delta Lake: Experience with Delta tables, ACID transactions, MERGE/UPSERT, schema enforcement, schema evolution, VACUUM, and Time Travel.
- Apache Airflow: Experience developing DAGs, managing task dependencies, configuring retries, scheduling workflows, and troubleshooting failures.
- Kubernetes: Practical knowledge of pods, deployments, services, ConfigMaps, Secrets, resource management, and troubleshooting containerized workloads.
- Git: Proficiency in branching strategies, pull requests, merging, and resolving conflicts.
- SQL: Strong SQL skills, including complex joins, CTEs, window functions, and query optimization.
Good to Have
- Experience with Docker and containerization best practices.
- Experience with Databricks Unity Catalog and data governance.
- Knowledge of CI/CD pipelines and automated deployments.
- Experience deploying and managing Airflow workloads on Kubernetes.
- Familiarity with Kubernetes monitoring, autoscaling, and resource optimization.
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Understanding of data lakehouse architecture and data modeling.
- Experience with batch and incremental data processing.
- Familiarity with data quality frameworks and automated testing.
Preferred Candidate Profile
- Strong analytical and problem-solving skills.
- Ability to independently design, develop, deploy, and troubleshoot data pipelines.
- Good understanding of distributed data processing and container orchestration.
- Experience working in an Agile development environment.
- Strong communication and collaboration skills.
🚨 HIRING | Data Engineer – Amazon EMR 🔥
We are looking for experienced Data Engineers with strong hands-on expertise in Amazon EMR, Python, and Scala.
📌 JOB DETAILS:
💼 Experience: 6+ Years
💰 CTC: Up to 30 LPA
📍 Location: Bangalore
🔑 MANDATORY SKILLS:
• Amazon EMR
• Python
• Scala
• Data Engineering
📋 KEY REQUIREMENTS:
• 6+ years of relevant experience in Data Engineering.
• Strong hands-on experience with Amazon EMR.
• Strong programming skills in Python and Scala.
• Experience with large-scale data processing and data engineering solutions.
• Strong problem-solving and communication skills.
📩 Interested candidates can share their updated resumes:
📧 pavithra.e @vysystems. com
📞 8 9 2 5 8 0 2 4 5 7
🔄 Know someone who would be a good fit? Tag them or share this post!
#Hiring #DataEngineer #DataEngineering #AmazonEMR #Python #Scala #AWSJobs #BangaloreJobs #ITJobs #TechHiring #HiringNow
Strong Data Engineer (AWS) Profile
2
Mandatory (Experience 1) – Must have minimum 6+ years of experience in Data Engineering, with strong experience designing, developing, and maintaining scalable data pipelines and ETL processes.
3
Mandatory (Experience 2) – Must have 3+ years of hands-on experience with AWS data engineering services, particularly S3, AWS Glue, EMR, Lambda, and Redshift.
4
Mandatory (Experience 3) – Must have strong programming experience in Python, Java, or Scala, with hands-on development of data processing and pipeline solutions.
5
Mandatory (Experience 4) – Must have strong expertise in SQL, data warehousing, and database technologies, including experience with both SQL and NoSQL databases.
6
Mandatory (Experience 5) – Must have hands-on experience working with Big Data technologies and distributed data processing, including designing and optimizing large-scale data pipelines.
7
Mandatory (Experience 6) – Must have experience with data pipeline troubleshooting, monitoring, data quality, governance, and security best practices.
8
Mandatory (Education) – B.Tech / B.E./ M.Tech










