3+ Apache Spark Jobs in Indore | Apache Spark Job openings in Indore

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines.
• Build and optimize data architectures, data lakes, and warehousing solutions.
• Integrate data from multiple APIs, databases, and third-party systems.
• Ensure data quality, consistency, security, and reliability across systems.
• Develop automated workflows for data ingestion, transformation, and validation.
• Work with structured and unstructured datasets at scale.
• Optimize SQL queries and database performance.
• Collaborate with backend, analytics, and AI teams for data-driven solutions.
• Monitor pipelines and troubleshoot production issues.
• Implement logging, monitoring, and alerting mechanisms for data systems.
• Maintain proper technical documentation and workflow diagrams.

Required Skills & Qualifications

• 2+ years of experience as a Data Engineer or similar role.
• Strong proficiency in SQL and database design.
• Experience with relational and NoSQL databases such as:
• MySQL
• PostgreSQL
• MongoDB
• BigQuery / Redshift / Snowflake
• Hands-on experience with ETL tools and data pipeline development.
• Strong programming skills in at least one language:
• Python
• Node.js
• Java
• Experience with cloud platforms:
• AWS / GCP / Azure
• Familiarity with data orchestration tools:
• Airflow
• Prefect
• Dagster
• Understanding of APIs, webhooks, and real-time data processing.
• Experience with Git and CI/CD workflows.
• Knowledge of Docker and containerized deployments.
• Good understanding of data security and governance practices.

Preferred Qualifications

• Experience with Apache Spark, Kafka, or distributed processing systems.
• Exposure to AI/ML data pipelines.
• Knowledge of analytics and BI tools such as:
• Power BI
• Tableau
• Looker
• Experience working in startup or fast-paced product environments.
• Familiarity with microservices architecture.

What We Offer

• Opportunity to work on scalable and impactful data systems.
• Collaborative and growth-focused work environment.
• Exposure to AI, analytics, and cloud-native technologies.
• Flexible work culture and learning opportunities.
• Competitive salary and performance-based growth.

Position: Team Lead – Data Engineering

Client: NucleusTeq

We are looking for an experienced Team Lead / Architect – Data Engineer to design, architect, and lead scalable cloud-based data platforms.

Key Responsibilities:

·     Design and architect scalable, high-performance data solutions.

·     Lead and mentor a team of data engineers.

·     Build and optimize large-scale data pipelines.

·     Drive architectural decisions and technical strategy.

·     Ensure data quality, governance, and security best practices.

·     Collaborate with cross-functional teams for end-to-end delivery.

Required Skills:

·     Strong hands-on experience in Java & Python.

·     Strong expertise in SQL and Big Data technologies.

·     Hands-on experience with Google Cloud Platform (GCP).

·     Experience with Apache Airflow for workflow orchestration.

·     Strong understanding of distributed systems and data architecture.

·     Experience in CI/CD pipelines (mandatory).

·     Experience in data warehousing and ETL processes.

·     Excellent leadership and stakeholder management skills.

Good to Have:

·     Experience with Dataflow (Apache Beam).

·     Experience designing enterprise-level data platforms.

·     Exposure to DevOps best practices.

• Does analytics to extract insights from raw historical data of the organization. 
• Generates usable training dataset for any/all MV projects with the help of Annotators, if needed.
• Analyses user trends, and identifies their biggest bottlenecks in Hammoq Workflow.
• Tests the short/long term impact of productized MV models on those trends.
• Skills - Numpy, Pandas, SPARK, APACHE SPARK, PYSPARK, ETL mandatory.