Data Engineer at RedSeer Consulting · Bengaluru (Bangalore) · 0 - 2 years · ₹10L - ₹15L / yr · Posted 11 Jul 2022
BRIEF DESCRIPTION:
At-least 1 year of Python, Spark, SQL, data engineering experience
Primary Skillset: PySpark, Scala/Python/Spark, Azure Synapse, S3, RedShift/Snowflake
Relevant Experience: Legacy ETL job Migration to AWS Glue / Python & Spark combination
ROLE SCOPE:
Reverse engineer the existing/legacy ETL jobs
Create the workflow diagrams and review the logic diagrams with Tech Leads
Write equivalent logic in Python & Spark
Unit test the Glue jobs and certify the data loads before passing to system testing
Follow the best practices, enable appropriate audit & control mechanism
Analytically skillful, identify the root causes quickly and efficiently debug issues
Take ownership of the deliverables and support the deployments
REQUIREMENTS:
Create data pipelines for data integration into Cloud stacks eg. Azure Synapse
Code data processing jobs in Azure Synapse Analytics, Python, and Spark
Experience in dealing with structured, semi-structured, and unstructured data in batch and real-time environments.
Should be able to process .json, .parquet and .avro files
PREFERRED BACKGROUND:
Tier1/2 candidates from IIT/NIT/IIITs
However, relevant experience, learning attitude takes precedence

Similar jobs (10)
Job Description – AWS Data Engineer
Experience: 7+ Years
Location: Kochi, Kerala
Role: AWS Data Engineer
Job Summary
We are looking for an experienced AWS Data Engineer with strong expertise in Python, PySpark, SQL, and AWS cloud services. The ideal candidate should have hands-on experience in developing, implementing, and maintaining scalable data pipelines and ETL processes, along with processing and transforming large datasets.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using AWS services.
- Develop data processing and transformation solutions using Python and PySpark.
- Write complex SQL queries for data extraction, transformation, and analysis.
- Build and maintain ETL/ELT workflows for large-scale data processing.
- Work with AWS services to implement data lake and data warehouse solutions.
- Optimize data pipelines for performance, scalability, and reliability.
- Troubleshoot data processing issues and ensure data quality.
- Collaborate with cross-functional teams to understand data requirements and deliver effective solutions.
Required Skills
- 7+ years of experience in Data Engineering.
- Strong hands-on experience in Python and PySpark.
- Excellent SQL skills, including complex queries, joins, CTEs, and window functions.
- Strong experience working with AWS cloud services.
- Hands-on experience with AWS Glue, Amazon S3, EMR, and Redshift.
- Good understanding of ETL/ELT processes and data pipeline development.
- Experience in data lake and data warehousing concepts.
Preferred Skills
- Experience with AWS Lambda and Step Functions.
- Knowledge of Apache Spark and distributed data processing.
- Experience with data lakehouse architecture.
- Knowledge of data optimization and performance tuning.
Location
Kochi, Kerala- F2F mandatory
Job Summary
We are looking for a skilled and experienced Data Engineer to join our growing data team. The ideal candidate will have strong expertise in Python, PySpark, Data Modeling, and Power BI, with hands-on experience in designing, developing, and optimizing scalable data solutions. The role requires working closely with business stakeholders, data architects, and analytics teams to build robust data pipelines and semantic models that enable data-driven decision-making.
Technical Skills
- Strong hands-on experience in Python and PySpark development.
- Expertise in building and optimizing Data Engineering solutions and ETL pipelines.
- Strong understanding of Data Modeling concepts (Star Schema, Snowflake Schema, Dimensional Modeling).
- Experience with Power BI Data Modeling and Semantic Layer development.
- Proficiency in DAX (Data Analysis Expressions).
- Experience designing and managing Semantic Models in Power BI.
- Strong SQL skills and experience working with large datasets.
- Knowledge of data warehousing concepts and best practices.
Preferred Skills
- Experience with cloud platforms such as Azure, AWS, or GCP.
- Exposure to modern data platforms like Databricks.
- Understanding of data governance and data quality frameworks.
Job Summary
We are looking for an experienced Data Engineer to build and optimize scalable data pipelines and AWS-based data platforms.
Key Responsibilities
- Develop and maintain data pipelines using Python and PySpark.
- Design and implement data lakehouse solutions on AWS.
- Build and manage data workflows using AWS Glue, EMR, ECS, and Step Functions.
- Work with DynamoDB and other data storage technologies.
- Monitor data pipelines and ensure data quality, performance, and reliability.
- Collaborate with stakeholders and engineering teams to deliver data solutions.
Mandatory Skills
- 5+ years of overall professional experience.
- 3+ years of experience in data platforms and big data technologies.
- Strong proficiency in Python and PySpark.
- Hands-on experience with AWS EMR, ECS, Glue, Step Functions, and DynamoDB.
- Experience designing and implementing AWS data lakehouse architectures.
Good to Have
- Apache Spark and Apache Iceberg.
- AWS Lambda, EventBridge, Kinesis, and Redshift.
- Experience with real-time data pipelines and event-driven architectures.

Job Title : AWS Data Engineer / Data Lake Engineer
Experience : 2 to 4 Years
Location : Gurugram Sector 43
Work Mode : 5 Days WFO
Employment Type : Full-Time
About the Role :
We are looking for a skilled AWS Data Engineer with 2 to 4 years of experience in designing, developing, and maintaining data pipelines and cloud-based data platforms.
The candidate will work on an enterprise AWS Data Lake platform that collects data from multiple business systems, processes large datasets, and makes them available for analytics, reporting, and application development.
The ideal candidate should have strong hands-on experience with AWS data services, Python, PySpark, SQL, and Apache Airflow, along with an understanding of data warehousing and API-based data serving.
Mandatory skills :
AWS Data Lake, S3, EMR, Glue, Athena, Redshift, Lambda, RDS, DMS, IAM, Python, PySpark, Apache Airflow, Advanced SQL, ETL / ELT Pipelines, Data Modeling, PostgreSQL, Production Support.
Key Responsibilities :
- Develop, maintain, and optimize data ingestion and ETL / ELT pipelines on AWS.
- Work with AWS services such as S3, Glue, EMR, Athena, Redshift, Lambda, RDS, and DMS.
- Build and manage data processing workflows using Python, PySpark, and Apache Airflow.
- Implement data lake architecture with staging, curated, and consumption layers.
- Write complex SQL queries and perform data transformation, validation, and modeling.
- Work with PostgreSQL and RDS for structured data storage and consumption.
- Support API-based data access using Hasura GraphQL or similar technologies.
- Monitor production data pipelines, troubleshoot failures, and perform root cause analysis.
- Optimize Spark jobs, SQL queries, and data pipelines for performance and cost efficiency.
- Follow AWS security practices, IAM permissions, and data governance standards.
- Collaborate with application, analytics, and business teams to understand data requirements.
Mandatory Skills :
- 2 to 4 years of experience in Data Engineering, AWS Data Engineering, or Data Platform Development.
- Strong hands-on experience with AWS services : S3, Glue, EMR, Athena, Redshift, Lambda, RDS, DMS, and IAM.
- Good experience in Python and PySpark.
- Practical experience with Apache Airflow for workflow orchestration.
- Strong SQL skills and understanding of database concepts.
- Experience in building and supporting ETL / ELT pipelines.
- Understanding of Data Lake architecture, data warehousing, and data modeling.
- Working knowledge of PostgreSQL or similar relational databases.
- Experience handling production pipeline issues and troubleshooting.
Good to Have :
- Experience with Hasura GraphQL or any GraphQL-based API / data-serving platform.
- Knowledge of DBT (Data Build Tool).
- Understanding of Lakehouse architecture and table formats such as Apache Iceberg or Delta Lake.
- Experience with Kafka or other streaming data technologies.
- Knowledge of Amazon Redshift performance optimization, distribution keys, and sort keys.
- Experience with AWS Aurora and API Gateway.
- BFSI, Banking, or Insurance domain experience.
Ideal Candidate Profile :
- Strong hands-on technical and problem-solving skills.
- Experience working on real-time or batch-based production data pipelines.
- Good understanding of data processing, transformation, and storage concepts.
- Ability to troubleshoot technical issues and work independently.
- Good communication and collaboration skills.
Note : Candidates with strong practical experience in AWS Data Lake, PySpark, Airflow, Python, and SQL will be preferred. Real-world project experience is more important than theoretical knowledge or certifications.
Job Description – Data Engineer
Role: Data Engineer
Experience: 9+ Years
Location: Bangalore / Hyderabad
Notice Period: Immediate to 15 Days
Interview Process: 1st Round – Virtual | 2nd Round – F2F
Mandatory Skills
- Python
- PySpark
- SQL
- Azure Data Engineering
Job Description
We are looking for an experienced Azure Data Engineer with 9+ years of experience and strong hands-on expertise in Python, PySpark, SQL, and Azure Data Engineering.
Key Responsibilities
- Develop and maintain scalable data engineering solutions using Azure.
- Build and optimize data processing pipelines using PySpark and Python.
- Write complex SQL queries for data extraction and transformation.
- Work with Azure data services and cloud-based data platforms.
- Perform data processing, transformation, and integration.
- Troubleshoot data pipeline and production issues.
- Collaborate with technical and business teams to deliver data solutions.
Preferred: Immediate to 15 Days joiners.

🚀 Job Title : Data Engineer – AWS, PySpark & Airflow
Experience : 5+ Years
Industry : Health Insurance / BFSI
Location : Sector 43, Gurugram, Haryana
Work Mode : Work From Office (5 Days a Week)
Employment Type : Full-Time
Role Overview :
We are looking for an experienced Data Engineer with strong hands-on expertise in AWS Data Engineering, Data Lake Architecture, Python, SQL, PySpark, and Apache Airflow.
The candidate will be responsible for designing, developing, and maintaining scalable, reliable, and high-performance data pipelines using AWS cloud technologies. The role involves building end-to-end ETL / ELT workflows, implementing data transformation and processing solutions, optimizing pipeline performance, and ensuring the reliability of production data systems.
Mandatory Skills :
AWS Data Engineering | Data Lake Architecture | Amazon S3 | AWS Glue | Amazon EMR | Athena | Redshift | Python | SQL | PySpark | Apache Spark | Apache Airflow | ETL/ELT | PostgreSQL | Data Modelling | Pipeline Optimization | Production Support | Root Cause Analysis
Key Responsibilities :
- Design, develop, and maintain end-to-end data pipelines for data ingestion, staging, transformation, curation, and consumption.
- Build and manage AWS-based data lake solutions using Amazon S3, AWS Glue, Amazon EMR, Athena, and Redshift.
- Develop, maintain, and optimize ETL / ELT pipelines using Python, SQL, Apache Spark, and PySpark.
- Implement and manage workflow orchestration using Apache Airflow, including scheduling, dependency management, retries, monitoring, and failure handling.
- Integrate data from multiple structured and semi-structured sources.
- Design and maintain data models, dimensional models, and analytical data marts.
- Work with relational databases, including PostgreSQL and Amazon RDS.
- Optimize Spark jobs, SQL queries, data transformations, and cloud resource utilization.
- Implement data quality checks, validation rules, and error-handling mechanisms.
- Monitor production data pipelines, troubleshoot failures, and perform root cause analysis (RCA).
- Ensure pipeline reliability, scalability, performance, and compliance with service-level agreements (SLAs).
- Follow data security, access management, governance, and sensitive-data protection best practices.
- Collaborate with business, analytics, and application teams to deliver effective data solutions.
- Maintain technical documentation and follow established data engineering standards.
Required Technical Skills :
- AWS Data Engineering : Strong hands-on experience with AWS-based data engineering and data lake architecture.
- AWS Services : Amazon S3, AWS Glue, Glue Data Catalog, Amazon EMR, Athena, Redshift, AWS Lambda, and IAM.
- Programming : Strong proficiency in Python and SQL.
- Big Data Processing : Hands-on experience with Apache Spark and PySpark.
- Workflow Orchestration : Experience developing and managing workflows using Apache Airflow.
- ETL/ELT Development : Experience building scalable data ingestion, transformation, and processing pipelines.
- Database Technologies : Good understanding of relational databases and PostgreSQL concepts.
- Data Modelling : Experience with data modelling, data transformation, and analytical data structures.
- Production Support : Ability to monitor pipelines, troubleshoot failures, perform RCA, and resolve production issues.
- Performance Optimization : Experience optimizing SQL queries, Spark jobs, and data processing workflows.
Good-to-Have Skills :
- Experience with AWS Database Migration Service (DMS) and Amazon RDS.
- Familiarity with dbt for data transformation and modelling.
- Knowledge of Delta Lake, Apache Iceberg, or Apache Hudi.
- Exposure to Apache Kafka and real-time data streaming.
- Familiarity with GraphQL, Hasura, or similar data-serving technologies.
- Understanding of data governance and data quality frameworks.
- Experience in the BFSI, healthcare, or health insurance domain.
Candidate Requirements :
- 5+ years of relevant experience in Data Engineering.
- Proven hands-on experience building and maintaining production-grade data pipelines on AWS.
- Strong understanding of data lake architecture and large-scale data processing.
- Practical experience with Apache Airflow and Spark / PySpark-based transformations.
- Strong programming, analytical, debugging, and problem-solving skills.
- Experience handling production incidents, pipeline failures, and performance bottlenecks.
- Ability to take ownership of deliverables and work effectively with cross-functional teams.
- Good communication, collaboration, and technical documentation skills.
- Willingness to work from the Gurugram office five days a week.
Ideal Candidate :
The ideal candidate is a hands-on Data Engineer who can independently build, optimize, and maintain AWS-based data pipelines, work with large datasets using PySpark, orchestrate workflows through Airflow, and resolve production issues in a business-critical environment.
Location: Bangalore Experience: 5 to 7 years Employment type: Full-time, permanent Work Hours: General Shift (10.00 AM to 7.00 PM) website: www.amazech.com Qualifications: Minimum B.E./B.Tech, or higher in Computer Science, Information Technology, Data Engineering, or a related field, with a good academic background. Key Responsibilities: • Design, develop, and maintain end-to-end data pipelines. • Build and optimize ETL/ELT processes for large-scale data processing. • Implement Azure-based data solutions ensuring scalability and performance. • Collaborate with stakeholders to understand business data requirements. • Perform performance tuning and optimization across data platforms. • Support production environments, conduct root cause analysis, and resolve data-related issues. • Maintain comprehensive technical documentation and structured knowledge transfer documents. • Work effectively within Agile/Scrum frameworks and contribute to sprint planning and delivery. • Ensure secure and compliant data handling using Azure best practices. Required Skills & Experience • End-to-end ETL/ELT pipeline development, integration, and performance optimization • Azure Synapse, Azure Logic Apps, Azure SQL, and Azure Databricks, with a strong focus on performance tuning • Microsoft Azure services, including Storage Accounts, Key Vault, and Cognitive Services • Advanced proficiency in Python development and modern productivity tools such as GitHub Copilot and Cursor • Strong documentation discipline, including technical design documentation, and structured knowledge transfer documents • Experience operating within Agile/Scrum frameworks, including production support, root cause analysis, and effective stakeholder collaboration
Senior Data Engineer – PySpark & Oracle
Experience: 7+ Years
Location: Bangalore
Notice Period: Immediate to 10 Days
Key Skills:
- Strong expertise in Data Modeling, Data Design & Modernization
- Primary skills: PySpark, Oracle SQL/PLSQL
- Secondary skills: Python, ETL & Data Pipelines
- Experience with Kafka and Hadoop
- Exposure to AWS / Azure / GCP
- Good knowledge of Git and JIRA
Roles & Responsibilities:
- Design, develop, and modernize scalable data models and data architecture.
- Develop and optimize data processing solutions using PySpark and Oracle SQL/PLSQL.
- Build and maintain robust ETL workflows and data pipelines.
- Work with Kafka, Hadoop, and cloud platforms for data processing and integration.
- Perform data transformation, optimization, and performance tuning.
- Collaborate with technical teams on data design, development, testing, and deployment.

Job Title : AWS Data Engineer
Experience : 4+ Years
Location : Bengaluru (HSR – Hybrid, 3 Days WFO)
Notice Period : Immediate Joiner
💡 Role Overview :
We are looking for a skilled AWS Data Engineer to design, build, and scale modern data platforms. The role involves working with AWS-native services, Python, Spark, and DBT to deliver secure, scalable, and high-performance data solutions in an Agile environment.
🔥 Mandatory Skills :
Python, SQL, Spark, AWS (S3, Glue, EMR, Redshift, Athena, Lambda), DBT, ETL/ELT pipeline development, Airflow/Step Functions, Data Lake (Parquet/ORC/Iceberg), Terraform & CI/CD, Data Governance & Security
🚀 Key Responsibilities :
- Design, build, and optimize ETL/ELT pipelines using Python, DBT, and AWS services
- Develop and manage scalable data lakes on S3 using formats like Parquet, ORC, and Iceberg
- Build end-to-end data solutions using Glue, EMR, Lambda, Redshift, and Athena
- Implement data governance, security, and metadata management using Glue Data Catalog, Lake Formation, IAM, and KMS
- Orchestrate workflows using Airflow, Step Functions, or AWS-native tools
- Ensure reliability and automation via CloudWatch, CloudTrail, CodePipeline, and Terraform
- Collaborate with data analysts and data scientists to deliver actionable insights
- Work in an Agile environment to deliver high-quality data solutions
✅ Mandatory Skills :
- Strong Python (including AWS SDKs), SQL, Spark
- Hands-on experience with AWS data stack (S3, Glue, EMR, Redshift, Athena, Lambda)
- Experience with DBT and ETL/ELT pipeline development
- Workflow orchestration using Airflow / Step Functions
- Knowledge of data lake formats (Parquet, ORC, Iceberg)
- Exposure to DevOps practices (Terraform, CI/CD)
- Strong understanding of data governance and security best practices
- Minimum 4–7 years in Data Engineering (3+ years on AWS)
➕ Good to Have :
- Understanding of Data Mesh architecture
- Experience with platforms like Data.World
- Exposure to Hadoop / HDFS ecosystems
🤝 What We’re Looking For :
- Strong problem-solving and analytical skills
- Ability to work in a collaborative, cross-functional environment
- Good communication and stakeholder management skills
- Self-driven and adaptable to fast-paced environments
📝 Interview Process :
- Online Assessment
- Technical Interview
- Fitment Round
- Client Round
Company: Wissen Technology
Position: Databricks Engineer
Experience: 6-10 Years
Location: Bengaluru/Mumbai
Employment Type: Full-time
About Wissen Technology
Wissen Technology is a global technology services company focused on delivering innovative software engineering, data, and technology solutions to leading enterprises. The company works with clients across industries to build scalable, high-performance technology platforms and solve complex business and technology challenges.
With a strong focus on engineering excellence, innovation, and collaboration, Wissen Technology brings together skilled technology professionals across areas such as software engineering, data engineering, cloud, analytics, and digital transformation.
At Wissen Technology, employees have the opportunity to work on challenging technology projects, collaborate with experienced engineering teams, and contribute to solutions that create measurable business impact.
Key Responsibilities
- Design, develop, and maintain scalable data processing applications using Python, PySpark, and Spark.
- Develop and optimize data pipelines and workflows on Databricks.
- Collaborate with data engineers, data scientists, business stakeholders, and other technical teams to understand requirements and deliver high-quality solutions.
- Ensure data integrity, quality, performance, and reliability across data processing pipelines.
- Write clean, maintainable, scalable, and efficient code following established coding standards and best practices.
- Perform data analysis and implement appropriate data validation and quality checks.
- Monitor, troubleshoot, and optimize performance issues across data workflows and pipelines.
- Work with relational databases and develop efficient SQL queries for data extraction and transformation.
- Participate in code reviews, testing, deployment, and continuous improvement of data engineering solutions.
- Use Git/version control and follow established software development and deployment practices.
Required Skills & Qualifications
- Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related field.
- Proven experience as a Databricks Developer, Data Engineer, or similar role.
- Strong hands-on expertise in Apache Spark and PySpark.
- Strong programming skills in Python; experience with Scala is an advantage.
- Strong proficiency in SQL and hands-on experience with relational databases.
- Practical experience developing and optimizing data pipelines and data processing applications.
- Familiarity with Git and version control systems.
- Strong understanding of data engineering concepts, data transformation, and data validation.
- Excellent analytical and problem-solving skills.
- Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.









