Data Engineer at Pluginlive · Chennai, Mumbai · 4 - 6 years · ₹10L - ₹20L / yr · Bootstrapped · Posted 10 Sep 2025
PythonRole Overview:
We are seeking a talented and experienced Data Architect with strong data visualization capabilities to join our dynamic team in Mumbai. As a Data Architect, you will be responsible for designing, building, and managing our data infrastructure, ensuring its reliability, scalability, and performance. You will also play a crucial role in transforming complex data into insightful visualizations that drive business decisions. This role requires a deep understanding of data modeling, database technologies (particularly Oracle Cloud), data warehousing principles, and proficiency in data manipulation and visualization tools, including Python and SQL.
Responsibilities:
- Design and implement robust and scalable data architectures, including data warehouses, data lakes, and operational data stores, primarily leveraging Oracle Cloud services.
- Develop and maintain data models (conceptual, logical, and physical) that align with business requirements and ensure data integrity and consistency.
- Define data governance policies and procedures to ensure data quality, security, and compliance.
- Collaborate with data engineers to build and optimize ETL/ELT pipelines for efficient data ingestion, transformation, and loading.
- Develop and execute data migration strategies to Oracle Cloud.
- Utilize strong SQL skills to query, manipulate, and analyze large datasets from various sources.
- Leverage Python and relevant libraries (e.g., Pandas, NumPy) for data cleaning, transformation, and analysis.
- Design and develop interactive and insightful data visualizations using tools like [Specify Visualization Tools - e.g., Tableau, Power BI, Matplotlib, Seaborn, Plotly] to communicate data-driven insights to both technical and non-technical stakeholders.
- Work closely with business analysts and stakeholders to understand their data needs and translate them into effective data models and visualizations.
- Ensure the performance and reliability of data visualization dashboards and reports.
- Stay up-to-date with the latest trends and technologies in data architecture, cloud computing (especially Oracle Cloud), and data visualization.
- Troubleshoot data-related issues and provide timely resolutions.
- Document data architectures, data flows, and data visualization solutions.
- Participate in the evaluation and selection of new data technologies and tools.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Data Science, Information Systems, or a related field.
- Proven experience (typically 5+ years) as a Data Architect, Data Modeler, or similar role.
- Deep understanding of data warehousing concepts, dimensional modeling (e.g., star schema, snowflake schema), and ETL/ELT processes.
- Extensive experience working with relational databases, particularly Oracle, and proficiency in SQL.
- Hands-on experience with Oracle Cloud data services (e.g., Autonomous Data Warehouse, Object Storage, Data Integration).
- Strong programming skills in Python and experience with data manipulation and analysis libraries (e.g., Pandas, NumPy).
- Demonstrated ability to create compelling and effective data visualizations using industry-standard tools (e.g., Tableau, Power BI, Matplotlib, Seaborn, Plotly).
- Excellent analytical and problem-solving skills with the ability to interpret complex data and translate it into actionable insights.
- Strong communication and presentation skills, with the ability to effectively communicate technical concepts to non-technical audiences.
- Experience with data governance and data quality principles.
- Familiarity with agile development methodologies.
- Ability to work independently and collaboratively within a team environment.
Application Link- https://forms.gle/km7n2WipJhC2Lj2r5
Shubham Vishwakarma
Full Stack Developer - Averlon
About Pluginlive
About
Connect with the team
Company social profiles
Similar jobs (10)
Amazon Web Services (AWS)Job description: Data Architect – Databricks / AWS
Job Summary
We are looking for an experienced Data Architect to define and drive the target data architecture for the Horizon MVP and its future evolution. The role will be responsible for designing a scalable, secure, governed cloud data platform covering ingestion, storage, processing, analytics, APIs, and downstream data consumption.
The architect will work closely with Data Engineering, Backend, DevOps, QA, and business stakeholders to establish architecture standards and ensure the platform is ready for advanced analytics, AI/ML, vector storage, and future LLM-based capabilities.
- Job Title: Data Architect
- Experience: 8+ Years
- Relevant Architecture Experience: 3+ Years in Data Architecture
- Location: Chennai / Pune
- Work Mode: Hybrid – 3 Days WFO
- Budget: Up to 24 LPA
- Payroll: Haparz
- Notice Period: Immediate Preferred
Key Responsibilities
- Define the target data architecture for the Horizon MVP and establish an architecture roadmap for future scalability.
- Design end-to-end architecture covering data ingestion, storage, processing, serving, reporting, APIs, and downstream applications.
- Establish canonical data models and schemas for travel signals, corridors, sources, evidence, scores, and generated insights.
- Define data normalization strategies for structured, semi-structured, and unstructured data from multiple external sources.
- Design and govern the Databricks platform architecture, including Unity Catalog, data schemas, access controls, and governance standards.
- Establish data-retention, lineage, data-quality, security, privacy, and compliance controls.
- Define secure integration patterns between Databricks, AWS PRODIGY, SharePoint, external APIs, and downstream applications.
- Design scalable data processing for 30-day signal windows, convergence/divergence scoring, corridor ranking, and spike detection.
- Define architecture patterns that support future vector storage, embeddings, LLM integration, and multi-year analytics.
- Design reliable batch and API-driven ingestion frameworks for structured and unstructured data.
- Review technical designs, identify architectural risks, and provide technical direction to engineering teams.
- Guide backend, data engineering, DevOps, and QA teams in implementing architecture standards.
- Ensure architecture decisions align with enterprise security, RBAC, PII handling, privacy, and operational requirements.
- Communicate architecture decisions, trade-offs, and technical recommendations effectively to technical and business stakeholders.
What We’re Looking For
- 8+ years of experience in data engineering, data platforms, or data architecture, with at least 3+ years in a Data Architect capacity.
- Strong hands-on experience designing cloud-based data platforms, lakehouses, or analytical platforms.
- Advanced knowledge of Databricks, Apache Spark/PySpark, Delta Lake, and Unity Catalog.
- Strong understanding of AWS data services, IAM, networking, and secure cloud integration patterns.
- Strong expertise in data modelling, metadata management, data lineage, data quality, retention, and governance.
- Experience architecting batch and API-based ingestion pipelines for structured, semi-structured, and unstructured data.
- Understanding of AI/ML workloads, feature pipelines, vector databases, embeddings, and LLM integration patterns.
- Experience designing APIs and downstream data-serving architectures.
- Strong knowledge of PII protection, RBAC, data privacy, and enterprise security controls.
- Excellent architectural communication and stakeholder-management skills.
PythonAmazon Web Services (AWS)About Us:
The QX Impact was launched with a mission to make A.I accessible and affordable and deliver AI Products/Solutions at scale for the enterprises by bringing the power of Data, AI, and Engineering to drive digital transformation. We believe without insights; businesses will continue to face challenges to better understand their customers and even lose them. Secondly, without insights businesses won't’ be able to deliver differentiated products/services; and finally, without insights, businesses can’t achieve a new level of “Operational Excellence” is crucial to remain competitive, meeting rising customer expectations, expanding markets, and digitalization.
Job Summary:
We are looking for a Senior Data Engineer who is creative, collaborative, and adaptable to join our agile team of data scientists, engineers, and UX developers. The role focuses on building and maintaining robust data pipelines to support advanced analytics, data science, and BI solutions.
As a Senior Data Engineer, you will work with internal and external data, collaborate with data scientists, and contribute to the design, development, and deployment of innovative solutions.
Key Responsibilities:
- Design, develop, test, and maintain optimal data pipeline and ETL architectures.
- Map out data systems and define/design required integrations, ETL, BI, and AI systems/processes.
- Prepare and optimize data for predictive and prescriptive modeling.
- Collaborate with teams to integrate ERP data into the enterprise data lake, ensuring seamless flow and quality.
- Enhance cloud data infrastructure on AWS or Azure for scalability and performance.
- Utilize big data tools and frameworks to optimize data acquisition and preparation.
- Build architectures to move data to/from data lakes and data warehouses for advanced analytics.
- Develop and curate data models for analytics, dashboards, and reports.
- Conduct code reviews, maintain production-level code, and implement testing approaches.
- Monitor, troubleshoot, and resolve data ingestion workflows to maintain reliability and uptime.
- Drive innovation and implement efficient new approaches to data engineering tasks.
Must-Have Skills:
- Bachelor’s degree in Computer Science, Mathematics, Engineering, or a related field.
- 5+ years of experience working with enterprise data platforms, including building and managing data lakes.
- 3–5 years of experience designing and implementing data warehouse solutions.
- Expertise in SQL, including developing stored procedures (SP) and applying advanced data design concepts.
- Proficiency in Spark (Python/Scala) and Spark Streaming for real-time data pipelines.
- Experience with AWS or Azure services (e.g., AWS Glue, Azure Data Factory, Redshift, Snowflake).
- Familiarity with big data tools such as Apache Kafka, Apache Spark, or Flink.
- Hands-on experience with orchestration tools (e.g., Apache Airflow, Prefect).
- Knowledge of CI/CD processes, version control (e.g., Git, Jenkins), and deployment automation.
- Strong problem-solving, communication, and collaboration skills.
Good-to-Have Skills:
- Experience in integrating ERP data into data lakes.
- Experience with traditional ETL tools (e.g., Talend, Pentaho).
Competencies:
- Tech Savvy - Anticipating and adopting innovations in business-building digital and technology applications.
- Self-Development - Actively seeking new ways to grow and be challenged using both formal and informal development channels.
- Action Oriented - Taking on new opportunities and tough challenges with a sense of urgency, high energy, and enthusiasm.
- Customer Focus - Building strong customer relationships and delivering customer-centric solutions.
- Optimize Work Processes - Knowing the most effective and efficient processes to get things done, with a focus on continuous improvement.
Why Join Us?
- Be part of a collaborative and agile team driving cutting-edge AI and data engineering solutions.
- Work on impactful projects that make a difference across industries.
- Opportunities for professional growth and continuous learning.
- Competitive salary and benefits package.
Application Details
Ready to make an impact? Apply today and become part of the QX Impact team!
PostgreSQL10+ years of experience in Data Architecture, Data Engineering, or Data Platforms
• Strong expertise in IBM DB2 / On-Prem Relational Databases
• Strong expertise in PostgreSQL
• Hands-on experience with the Azure Data Ecosystem, including:
▪ Azure Data Factory (ADF)
▪ Azure Data Lake Storage Gen2 (ADLS Gen2)
▪ Azure Databricks / Synapse Analytics
▪ Azure Event Hub / Service Bus
▪ Azure Functions
Solution Architect – AZURE Data Engineering
Job Overview
We are looking for an experienced Solution Architect – Data Engineering with strong expertise in designing data solutions and hands-on experience with Azure, Synapse, PySpark, Data Warehousing, and Data Lakes. The ideal candidate should have strong architectural and data engineering knowledge.
Key Responsibilities
- Design and implement scalable data architecture and solutions.
- Develop and manage Data Warehouse and Data Lake architectures.
- Design data platforms using Medallion Architecture.
- Lead Data Engineering and ETL activities.
- Work with Azure Synapse Analytics for data processing and analytics.
- Develop data solutions using PySpark / Apache Spark.
- Define and implement data validation and data quality processes.
- Collaborate with business, data, and technology teams to deliver effective data solutions.
Required Skills
- Strong experience in Solution Architecture / Data Architecture.
- Strong knowledge of Data Warehouse and Data Lake architecture.
- Good understanding of Medallion Architecture.
- Strong experience in Data Engineering and ETL.
- Hands-on experience with Microsoft Azure and Azure Synapse Analytics.
- Strong knowledge of PySpark / Apache Spark.
- Experience with Data Validation and Data Quality.
- Good communication and stakeholder management skills.
Experience
8+ Years
The recruiter has not been active on this job recently. You may apply but please expect a delayed response.
Python- Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data ingestion and transformation.
- Develop and optimize data lake and data warehouse architectures (e.g., Snowflake, BigQuery, Redshift).
- Work with cloud platforms GCP, Azure to manage data infrastructure.
- GCP as mandatory skills
- Collaborate with analytics and product teams to understand data needs and deliver solutions.
- Ensure data quality, reliability, security, and compliance across all data systems.
- Mentor junior data engineers and contribute to best practices and code reviews.
- Monitor and troubleshoot data pipeline performance and resolve data-related issues.
- Automate data validation, monitoring, and alerting processes.
- 8+ years of experience in data engineering or software engineering with a data focus.
- Proficient in SQL and at least one programming language (e.g., Python, Scala, Java).
- Experience with modern data warehousing tools (e.g., Snowflake, Redshift, BigQuery).
- Strong understanding of data modeling, data lakes, and ETL/ELT design.
- Hands-on experience with orchestration tools like Airflow, dbt, or similar.
- Solid experience with cloud data platforms (AWS/GCP/Azure).
- Familiarity with CI/CD pipelines, containerization (Docker/Kubernetes), and version control (Git).
- Experience working in a DevOps or DataOps environment.
- Knowledge of data governance, lineage, and cataloging tools (e.g., Collibra, Alation).
- Familiarity with streaming technologies (Kafka, Spark Streaming, Flink).
- Experience supporting machine learning workflows and data science initiatives.
Example:
We are looking for an experienced Data Architect to design, develop, and manage the organization's enterprise data architecture. The candidate will be responsible for building scalable data platforms, ensuring data quality and governance, and supporting business analytics through modern data solutions.
Experience Required
Mention the minimum years of experience.
Example:
- Minimum 12 years of experience in Data Architecture, Data Engineering, or related fields.
- 5+ years of experience in the Banking/Financial Services domain is preferred.
Educational Qualification
Mention the required degree.
Example:
- BE/BTech in Computer Science, Information Technology, Software Engineering, Electronics & Communication Engineering, or equivalent.
- OR MCA/MTech/MSc in Computer Science, IT, or related disciplines.
- MBA is preferred.
Technical Skills
List the skills the candidate must have.
Example:
- AWS, Azure, or GCP
- Data Warehousing (DWH)
- ETL/ELT
- Database Management
- Data Modeling
- Data Analytics
- Data Lakes
- Data Governance
Key Responsibilities
Convert the points you received into simple action statements.
Example:
- Design and maintain enterprise data architecture.
- Develop data warehouses and data lakes.
- Define data standards and governance policies.
- Ensure data quality and security.
- Design ETL/ELT processes.
- Integrate data from multiple systems.
- Plan and execute data migration projects.
- Review existing data architecture and recommend improvements.
- Provide technical guidance to project teams.
- Evaluate new data technologies and tools.
Preferred Skills
These are not mandatory but are an advantage.
Example:
- Banking domain experience
- Strong analytical and problem-solving skills
- Good communication skills
- Leadership and stakeholder management
- Experience mentoring technical teams
Data Engineer Short Hiring Post
🚨 Hiring: Data Engineer
🔹 Experience: 5–9 Years
🔹 Location: Bangalore / Hyderabad
🔹 Skills: PySpark, Python, SQL, ETL, CI/CD, Data Modeling
🔹 Process: L1 Virtual → L2 F2F Karat Test
🔹 F2F: Bangalore / Hyderabad Location
🔹 Positions: Immediate requirement
⚠️ Note: Candidates must be available for F2F Karat immediately after L1.
#Hiring #DataEngineer #PySpark #Python #SQL #BangaloreJobs #HyderabadJobs #Mphasis #ImmediateJoiners
PythonAmazon Web Services (AWS)About the Role
You'll be at the forefront of designing and implementing robust data platform solutions that power advanced analytics, AI, and machine learning. Working with modern cloud technologies, you'll build scalable data foundations that enable clients to make smarter, data-driven decisions.
Key Responsibilities
- Build scalable data pipelines using Snowflake, AWS, GCP, and Databricks.
- Design and optimize data models for AI and machine learning workloads.
- Develop reliable data foundations for MLOps, governance, and data lineage.
- Integrate data from multiple sources into modern data platforms.
- Leverage Snowpark ML and Snowflake's native AI capabilities.
- Ensure data platforms are secure, scalable, and high-performing.
What We're Looking For
- 5+ years of hands-on experience with Snowflake.
- Strong proficiency in SQL and Python.
- Experience with AWS, Azure, or GCP.
- Knowledge of cloud storage services such as S3, ADLS, or GCS.
- Strong understanding of Dimensional Modeling and Data Vault.
- Experience with Scala or Java is a plus.
Tech Stack
- Data Warehouse: Snowflake
- Programming: SQL, Python, Scala (Good to Have), Java (Good to Have)
- Cloud: AWS, Azure, GCP
- Storage: S3, ADLS, GCS
- AI/ML: Snowpark ML, MLOps
Perks & Benefits
- Public Speaking & Communication Program
- Mentoring Program with Senior Support Leads
- 360° Progress Reviews
- Weekly Learning Sessions & Guilds
- Paid Certifications
- Hackathons & Innovation Days
- Recognition & Rewards Programs
- Team Socials & Annual Offsites
- Employee Assistance Program (24/7 Wellbeing Support)
The Data People Shaping Tomorrow
Our client helps organizations unlock the power of data through modern cloud, analytics, and AI solutions. We believe in creating an environment where talented technologists can learn, innovate, and make a real impact while building cutting-edge data platforms for global clients. If you're passionate about data engineering and want to work with the latest technologies in AI, cloud, and analytics, we'd love to hear from you.
What you'll need
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field.
- 5+ years of professional data engineering experience.
- Experience designing and building cloud-native data solutions.
- Strong expertise with Google Cloud Platform, including BigQuery. Experience developing transformation frameworks using dbt.
- Strong SQL and Python programming skills.
- Experience with PostgreSQL or other relational databases.
- Experience orchestrating workflows using Apache Airflow or Cloud Composer.
- Experience implementing Infrastructure as Code using Terraform. Experience building CI/CD pipelines using GitHub Actions, GitLab CI, Jenkins, or similar platforms.
- Experience developing scalable batch and streaming data pipelines. Strong problem-solving skills with the ability to balance scalability, reliability, and cloud cost optimization.
Preferred Qualifications
- Experience with Pub/Sub, Datastream, Dataflow, Cloud Storage, Cloud Functions, or Cloud Run.
- Experience building multi-tenant SaaS platforms.
- Experience implementing metadata-driven governance, lineage, and data quality frameworks.
- Experience supporting AI, machine learning, or customer-facing analytics platforms.
- Experience with Kubernetes and Docker.
PythonJob Summary
We are seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure. The ideal candidate should have strong expertise in SQL, Python, Linux, and modern data engineering practices to support data integration, transformation, and analytics.
Key Responsibilities
- Design, develop, and maintain ETL/ELT data pipelines.
- Write efficient and optimized SQL queries for data extraction, transformation, and reporting.
- Develop automation scripts using Python for data processing and workflow optimization.
- Work with Linux environments for deployment, monitoring, and troubleshooting.
- Ensure data quality, integrity, and reliability across data platforms.
- Collaborate with data analysts, software engineers, and business stakeholders to deliver data solutions.
- Monitor, troubleshoot, and optimize data pipelines for performance and scalability.
- Implement best practices for data security, governance, and documentation.
Required Skills
- Strong experience in Data Engineering concepts and ETL/ELT processes.
- Proficiency in SQL, including query optimization and database design.
- Strong programming skills in Python.
- Hands-on experience with Linux commands, shell scripting, and system administration basics.
- Experience with relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle.
- Familiarity with Git/version control.
- Strong analytical and problem-solving skills.
Preferred Skills
- Experience with cloud platforms (AWS, Azure, or GCP).
- Knowledge of Apache Spark, Airflow, Kafka, or similar data engineering tools.
- Experience with data warehousing solutions and big data technologies.
- Understanding of CI/CD pipelines and containerization (Docker/Kubernetes).
Qualifications
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
- Relevant certifications in cloud or data engineering are an added advantage.
Follow CutshortShubham Vishwakarma
Full Stack Developer - Averlon