Data Engineer at Scremer · Pune, Mumbai · 6 - 11 years · ₹15L - ₹15L / yr · Bootstrapped · Posted 28 May 2024

Primary Skills
DynamoDB, Java, Kafka, Spark, Amazon Redshift, AWS Lake Formation, AWS Glue, Python
Skills:
Good work experience showing growth as a Data Engineer.
Hands On programming experience
Implementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS Lake Formation.
Excellent knowledge in: Python, Scala/Java, Spark, AWS (Lambda, Step Functions, Dynamodb, EMR), Terraform, UI (Angular), Git, Mavena
Experience of performance optimization in Batch and Real time processing applications
Expertise in Data Governance and Data Security Implementation
Good hands-on design and programming skills building reusable tools and products Experience developing in AWS or similar cloud platforms. Preferred:, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, Quick Sight or similar.
Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark).
Knowledge and hands-on experience with server less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus.
Expertise in practices like Agile, Peer reviews, Continuous Integration
Roles and responsibilities:
Determining project requirements and developing work schedules for the team.
Delegating tasks and achieving daily, weekly, and monthly goals.
Responsible for designing, building, testing, and deploying the software releases.
Salary: 25LPA-40LPA

About Scremer
About
Similar jobs (5)
Join our product development team at Planview as a Senior Software Engineer I and become a pivotal force on the Viz Core Team. This role offers the unique opportunity to shape and lead the development of data-processing pipelines and APIs that are at the heart of our software solutions. These solutions are designed to streamline and enhance the efficiency of software delivery, resonating deeply with software engineers who strive to build better and faster.
At Planview Viz, which is powered by the innovative “Flow Framework,” you will tackle complex data challenges and develop scalable solutions within an AWS cloud environment. Your efforts will be crucial in revolutionizing how businesses harness and interpret vast amounts of workflow data, transforming it into actionable insights that propel organizational efficiency and effectiveness.
Responsibilities (What you'll do)
- Create and refine powerful data-processing architectures that integrate seamlessly with a diverse array of external tools, enhancing the way software is delivered across industries.
- Drive operational excellence by critically analyzing problems, defining requirements, and devising robust solutions that push the boundaries of technology.
- Lead and inspire a team of talented engineers, promoting a culture of ownership, meticulous attention to quality, and proactive problem-solving.
- Stay at the cutting edge of technology by updating and expanding your team’s knowledge of cloud architectures, data processing, and advanced analytics, ensuring that you and your team remain leaders in technological innovation.
- Make a direct impact on the efficiency and effectiveness of software delivery worldwide through innovation and leadership.
Qualifications (What you'll bring)
Who We’re Looking For
The ideal candidate is a seasoned professional in cloud software development with a strong foundation in data processing and scalable software systems. You thrive in collaborative environments and are passionate about advancing cloud technology and data architecture to new heights. You are a leader who enjoys mentoring, guiding, and inspiring others.
Preferred Qualifications
Skills, Knowledge, and Expertise
- A degree in Computer Science, Engineering, or a related field.
- 6+ years of experience with a modern programming language, with a focus on back-end systems and cloud-based technologies.
- Strong capability in architecting and designing robust, scalable software systems.
- Proficiency in AWS or another popular cloud platform.
Additional Qualifications
- Experience with big-data technologies such as Spark and Apache Kafka.
- Experience with Java (version 17), Scala, or other JVM programming languages.
- Experience with tools such as Amazon Redshift and MongoDB.
- Experience with continuous integration and continuous deployment tools such as GitHub, Jenkins, Travis CI, or similar platforms.
- Proficiency in test-driven development.
- Experience with containers and orchestration tools such as Kubernetes.
- Experience working on remote or distributed teams and projects.
Job Title : Tech Lead – Data Lake Platform
Number of Positions : 2
Experience : 7+ Years
Role Type : Technical Lead / Data Platform Lead
Domain : Data Engineering / Data Platform / AWS
About the Role :
We are looking for an experienced Tech Lead – Data Lake Platform to lead the design, development, and operations of an enterprise-scale AWS-based Data Lake Platform.
The platform will ingest data from multiple business systems, process it through structured data layers, and serve data for analytics, reporting, APIs, and operational applications.
As a Tech Lead, you will be responsible for setting the technical direction, leading data engineering teams, driving platform reliability and performance, and owning the platform's delivery, governance, and production support end to end.
Core Tech Stack :
AWS | S3 | EMR | Glue | Athena | Redshift | DMS | Lambda | RDS | IAM | Airflow | Spark/PySpark | SQL | Data Modeling | Hasura | GraphQL | DBT | PostgreSQL/Aurora | Kafka
Key Responsibilities :
- Own the overall Data Lake Platform architecture, covering data ingestion, staging, curated layers, consumption, analytics, and API/data serving.
- Lead the design and development of production-grade data pipelines using AWS Glue, EMR/Spark, Airflow, DBT, Athena, and Redshift.
- Design scalable batch and analytics pipelines with a focus on reliability, performance, data quality, and maintainability.
- Own the API/data-serving architecture from consumption data → RDS/PostgreSQL → Hasura GraphQL → Lambda/API Gateway.
- Drive improvements in platform stability, including orchestration failures, cluster sizing, pipeline SLAs, query performance, and production reliability.
- Design and implement appropriate AWS security, access control, IAM, PII handling, and data governance practices.
- Lead technical discussions, architecture decisions, code reviews, and engineering best practices.
- Mentor and guide data engineers while ensuring high-quality and scalable engineering delivery.
- Own production support, incident management, troubleshooting, and root-cause analysis (RCA) for critical data platform issues.
- Develop and maintain runbooks, operational procedures, monitoring, and incident response practices.
- Collaborate with business, product, application, and analytics teams to onboard new datasets and support reporting, API, and data consumption requirements.
- Ensure the platform meets defined availability, performance, security, data quality, and compliance requirements.
Must-Have Skills :
- 7+ years of experience in Data Engineering, Data Platform Engineering, or related areas, including experience in technical leadership or architecture.
- Strong hands-on experience with AWS Data Services, including:
- Amazon S3
- AWS EMR
- AWS Glue
- Amazon Athena
- Amazon Redshift
- AWS DMS
- AWS Lambda
- Amazon RDS
- AWS IAM
- Strong production experience with Apache Airflow.
- Strong hands-on experience with Apache Spark / PySpark.
- Strong SQL skills and experience with data modeling, including layered data architecture, data marts, and enterprise data models.
- Experience with Hasura or a similar GraphQL layer for PostgreSQL-based data/API serving.
- Strong understanding of data lake architecture and enterprise data platforms.
- Proven experience leading engineers and driving technical decisions.
- Hands-on experience with production support, troubleshooting, incident management, and RCA.
- Strong understanding of data platform performance, scalability, reliability, and SLA management.
Good-to-Have Skills :
- Experience with DBT and modern data transformation practices.
- Experience with lakehouse table formats on Amazon S3, such as Apache Iceberg or similar technologies.
- Strong knowledge of Amazon Redshift workload optimization, including :
- Distribution keys
- Sort keys
- Spectrum
- External tables
- Experience with Kafka or other streaming/data ingestion technologies.
- Experience with PostgreSQL / Amazon Aurora as a data-serving layer.
- Experience with Lambda and API Gateway for API-based data serving.
- Experience with enterprise data governance, data quality, security, and compliance.
- Experience in BFSI / Banking / Financial Services / Insurance domain.
Ideal Candidate :
The ideal candidate is a hands-on Data Platform / Data Engineering Lead who can operate at both the architecture and implementation level. You should be comfortable designing an AWS Data Lake from end to end, leading engineers, troubleshooting production issues, and working closely with business and application teams to deliver reliable data products.

Data Engineer
Location: Bengaluru, India (Hybrid)
Employment Type: Full-time
Experience: 3-5 years
Role Overview
What We’re Looking For:
- Bachelor’s degree in Computer Science/Engineering or equivalent experience required.
- Experience designing and shipping cloud services products.
- Experience driving and managing technical and architectural dependencies on AWS Cloud.
- A firm understanding of system architecture, cloud computing, PaaS/SaaS design principles, S3, DynamoDB, RDS mandatory.
- Experience in building or maintaining ETL processes and tools, i.e., AWS Glue or any open-source tool.
- Proven system-level design contribution to a current “Live” (in production / under daily high load) multi-region SaaS or PaaS offering.
- Proven experience with S3, DynamoDB, SQL, and AWS RDS services.
- Proficiency in programming languages such as Python.
- Strong analytical and problem-solving skills.
Required Skills & Experience
- Experience with Python, SQL, and data visualization/exploration tools.
- Familiarity with the AWS ecosystem, specifically S3, DynamoDB, and RDS.
- Communication skills, especially for explaining technical concepts to nontechnical business leaders.
- Ability to work on a dynamic, research-oriented team that has concurrent projects.
- Experience in AWS cost optimization (Savings Plans, Reserved Instances, Spot Instances) and governance frameworks.
- Experience developing solutions using infrastructure orchestration tools (SSM, automation account, Ansible, etc.).
- Excellent leadership, stakeholder management, and communication skills.
What We Offer
- Work with some of the brightest minds in the emerging EV industry.
- Make a tangible impact in reducing carbon emissions and enabling sustainable energy.
- Freedom to suggest, implement, and innovate on systems, processes, and technologies.
- Daily ownership in a high-growth, challenging environment.
- Flexible work environment with hybrid schedules and virtualization options.
- Competitive pay and benefits including health coverage, innovative PTO program, and performance bonuses.
Overview
Planview has one mission: to build the future of connected work, from ideas to impact.
As the global leader in work and resource management, Planview helps organizations accelerate the achievement of what matters most, supporting our customers from need to speed, from passion to progress, and from overhead to optimization. We provide the industry’s most comprehensive solutions designed for strategic planning, portfolio and resource management, Lean and Agile delivery, product portfolio management, capability, technology management (enterprise architecture), innovation management, collaborative work and project management. Our connected platform of solutions underpins the business and digital transformations of more than 4,500 customers globally, including 59 of the Fortune 100.
At Planview, our people connections drive our innovation and success. Our global team of 1400+ work remotely and across our offices including Austin, Vancouver, Hod HaSharon, Bangalore, and Stockholm. We are proud of our world-class, connected culture built on our shared values, that supports our teams to be successful from anywhere.
The Opportunity
Join our product development team at Planview as a Software Engineer 2 and contribute to the Viz Core Team. This role provides an exciting opportunity to support the development of data processing pipelines and APIs that form the backbone of our software solutions. You will work collaboratively to enhance the efficiency of software delivery, developing solutions that help organizations harness and interpret workflow data.
In this role, you will
•Collaborate with senior engineers to create and refine data processing architectures that integrate with various external tools.
•Assist in analyzing problems, defining requirements, and proposing solutions that align with the team’s objectives.
•Contribute to a culture of quality and continuous improvement by providing thoughtful feedback on code and design.
•Learn and apply best practices in cloud architectures and data processing to deliver high-quality software solutions.
•Engage in an environment that values innovation and teamwork, where your contributions will positively impact the delivery of software solutions.
Who we are looking for
The ideal candidate is an emerging professional in cloud software development with a solid foundation in data processing and software systems. You thrive in collaborative environments and are eager to learn and grow within the field.
Preferred qualifications skills, knowledge, and expertise
•Degree in Computer Science, Engineering, or a related field.
•3+ years of experience with any modern programming language, focusing on back-end systems and cloud-based technologies.
•Basic understanding of data structures, algorithms, and software development methodologies.
•Familiarity with cloud platforms, particularly AWS or similar services.
Other beneficial qualifications
•Exposure to big data technologies like Spark or Apache Kafka.
•Experience with Java (v17), Scala, or other JVM programming languages.
•Familiarity with storage tools such as PostgreSQL and MongoDB.
• Understanding of continuous integration/deployment tooling such as GitHub or Jenkins.
• Awareness of Test-Driven Development practices.
• Basic knowledge of containers and orchestration tools like
We’re seeking a highly skilled, execution-focused Senior Backend Engineer with a minimum of 5 years of experience to join our team. This role demands hands-on expertise in building and scaling distributed systems, strong proficiency in Java, and deep knowledge of cloud-native infrastructure. You will be expected to design robust backend services, optimize performance across storage and caching layers, and enable seamless integrations using modern messaging and CI/CD pipelines.
You’ll be working in a high-scale, high-impact environment where reliability, speed, and efficiency are paramount. If you enjoy solving complex engineering challenges and have a passion for distributed systems, this is the right role for you.
Responsibilities—
- Design, develop, and maintain distributed backend systems at scale.
- Write high-performance, production-grade code in Java.
- Architect and optimize storage systems, ensuring efficient query performance and scalable data models.
- Implement caching strategies to reduce latency and improve system throughput.
- Build and manage services leveraging AWS cloud infrastructure.
- Develop resilient messaging pipelines using Kafka (or equivalent) for real-time data processing.
- Define and streamline CI/CD pipelines, ensuring rapid and reliable deployment cycles.
- Collaborate with product managers, frontend engineers, and DevOps to deliver end-to-end solutions.
- Monitor system performance, identify bottlenecks, and apply proactive fixes.
- Drive best practices in software engineering, testing, and code reviews.
Requirements—
- 5+ years of experience in backend engineering, with deep hands-on coding experience.
- Strong proficiency in Java and familiarity with modern frameworks.
- Proven track record in building scalable distributed systems.
- Hands-on expertise with AWS services (e.g., EC2, S3, Lambda, DynamoDB, RDS).
- Solid understanding of messaging systems like Kafka, RabbitMQ, or similar.
- Strong grasp of query performance optimization and storage system design.
- Experience with caching solutions (Redis, Memcached, etc.).
- Familiarity with CI/CD tools (Jenkins, GitHub Actions, GitLab CI, etc.).
- Excellent problem-solving skills and ability to thrive in fast-paced environments.
- Strong communication and collaboration skills, with a proactive mindset.
Benefits—
- Best in class salary: We hire only the best, and we pay accordingly.
- Proximity Talks: Meet other designers, engineers, and product geeks — and learn from experts in the field.
- Keep on learning with a world-class team: Work with the best in the field, challenge yourself constantly, and learn something new every day.
About us—
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.








