PythonJob Description
Job Title: Data Engineer
Location: Hyderabad, India
Job Type: Full Time
Experience: 5 – 8 Years
Working Model: On-Site (No remote or work-from-home options available)
Work Schedule: Mountain Time Zone (3:00 PM to 11:00 PM IST)
Role Overview
The Data Engineer will be responsible for designing and implementing scalable backend systems, leveraging Python and PySpark to build high-performance solutions. The role requires a proactive and detail-orientated individual who can solve complex data engineering challenges while collaborating with cross-functional teams to deliver quality results.
Key Responsibilities
- Develop and maintain backend systems using Python and PySpark.
- Optimise and enhance system performance for large-scale data processing.
- Collaborate with cross-functional teams to define requirements and deliver solutions.
- Debug, troubleshoot, and resolve system issues and bottlenecks.
- Follow coding best practices to ensure code quality and maintainability.
- Utilise tools like Palantir Foundry for data management workflows (good to have).
Qualifications
- Strong proficiency in Python backend development.
- Hands-on experience with PySpark for data engineering.
- Excellent problem-solving skills and attention to detail.
- Good communication skills for effective team collaboration.
- Experience with Palantir Foundry or similar platforms is a plus.
Preferred Skills
- Experience with large-scale data processing and pipeline development.
- Familiarity with agile methodologies and development tools.
- Ability to optimise and streamline backend processes effectively.
Shubham Vishwakarma
Full Stack Developer - Averlon
About Indigrators solutions
About
Company social profiles
Similar jobs
Amazon Web Services (AWS)Job Title: Data Engineer (Python, AWS, ETL)
Experience: 6+ years
Location: PAN India (Remote / Work From Home)
Employment Type: Full-time
Preferred domain: Real Estate
Key Responsibilities:
Develop and optimize ETL workflows using Python, Pandas, and PySpark.
Design and implement SQL queries for data extraction, transformation, and optimization.
Work with JSON and REST APIs for data integration and automation.
Manage and optimize Amazon S3 storage, including partitioning and lifecycle policies.
Utilize AWS Athena for SQL-based querying, performance tuning, and cost optimization.
Develop and maintain AWS Lambda functions for serverless processing.
Manage databases using Amazon RDS and Amazon DynamoDB, ensuring performance and scalability.
Orchestrate workflows with AWS Step Functions for efficient automation.
Implement Infrastructure as Code (IaC) using AWS CloudFormation for resource provisioning.
Set up AWS Data Pipelines for CI/CD deployment of data workflows.
Required Skills:
Programming & Scripting: Python (ETL, Automation, REST API Integration).
Databases: SQL (Athena / RDS), Query Optimization, Schema Design.
Big Data & Processing: Pandas, PySpark (Data Transformation, Aggregation).
Cloud & Storage: AWS (S3, Athena, RDS, DynamoDB, Step Functions, CloudFormation, Lambda, Data Pipelines).
Good to Have Skills:
Experience with Azure services such as Table Storage, AI Search, Cognitive Services, Functions, Service Bus, and Storage.
Qualifications:
Bachelor’s degree in Data Science, Statistics, Computer Science, or a related field.
6+ years of experience in data engineering, ETL, and cloud-based data processing.
Vue.js
AngularJS (1.x)
React.js
Angular (2+)
Javascript· The Objective:
You will play a crucial role in designing, implementing, and maintaining our data infrastructure, run tests and update the systems
· Job function and requirements
o Expert in Python, Pandas and Numpy with knowledge of Python web Framework such as Django and Flask.
o Able to integrate multiple data sources and databases into one system.
o Basic understanding of frontend technologies like HTML, CSS, JavaScript.
o Able to build data pipelines.
o Strong unit test and debugging skills.
o Understanding of fundamental design principles behind a scalable application
o Good understanding of RDBMS databases among Mysql or Postgresql.
o Able to analyze and transform raw data.
· About us
Mitibase helps companies find warm prospects every month that are most relevant, and then helps their team to act on those with automation. We do so by automatically tracking key accounts and contacts for job changes and relationships triggers and surfaces them as warm leads in your sales pipeline.
PythonAmazon Web Services (AWS)
ScalaWe have urgent requirement of Data Engineer/Sr Data Engineer for reputed MNC company.
Exp: 4-9yrs
Location: Pune/Bangalore/Hyderabad
Skills: We need candidate either Python AWS or Pyspark AWS or Spark Scala
ScalaEXP-Developer-4 to 12 years
Must have low-level design and development skills. Should able to design a solution for given use cases.
- Agile delivery- Person must able to show design and code on a daily basis
- Must be an experienced PySpark developer and Scala coding. Primary skill is PySpark
- Must have experience in designing job orchestration, sequence, metadata design, Audit trail, dynamic parameter passing and error/exception handling
- Good experience with unit, integration and UAT support
- Able to design and code reusable components and functions
- Should able to review design, code & provide review comments with justification
- Zeal to learn new tool/technologies and adoption
- Good to have experience with Devops and CICD
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon