Game Designer

Senior Data Engineer (Databricks, BigQuery, Snowflake)

Experience: 8+ Years in Data Engineering

Location: Remote | Onsite (Noida, Gurgaon, Pune, Nagpur, Jaipur, Gandhinagar)

Budget: Open / Competitive

Job Summary:

We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data solutions that support advanced analytics and machine learning initiatives. You will lead the development of reliable, high-performance data systems and collaborate closely with data scientists to enable data-driven decision-making.

In this role, we expect a forward-thinking professional who utilizes AI-augmented development tools (such as Cursor, Windsurf, or GitHub Copilot) to increase engineering velocity and maintain high code standards in a modern enterprise environment.

Key Responsibilities:

• Scalable Pipelines: Design, develop, and optimize end-to-end data pipelines using SQL, Python, and PySpark.
• ETL/ELT Workflows: Build and maintain workflows to transform raw data into structured, analytics-ready datasets.
• ML Integration: Partner with data scientists to deploy and integrate machine learning models into production environments.
• Cloud Infrastructure: Manage and scale data infrastructure within AWS and Azure ecosystems.
• Data Warehousing: Utilize Databricks and Snowflake for big data processing and enterprise warehousing.
• Automation & IaC: Implement workflow orchestration using Apache Airflow and manage infrastructure as code via Terraform.
• Performance Tuning: Optimize data storage, retrieval, and system performance across data warehouse platforms.
• Governance & Compliance: Ensure data quality and security using tools like Unity Catalog or Hive Metastore.
• AI-Augmented Development: Integrate AI tools and LLM APIs into data pipelines and use AI IDEs to streamline debugging and documentation.

Technical Requirements:

• Experience: 8+ years of core Data Engineering experience in large-scale enterprise or consulting environments.
• Languages: Expert proficiency in SQL and Python for complex data processing.
• Big Data: Hands-on experience with PySpark and large-scale distributed computing.
• Architecture: Strong understanding of ETL frameworks, data pipeline architecture, and data warehousing best practices.
• Cloud Platforms: Deep working knowledge of AWS and Azure.
• Modern Tooling: Proven experience with Databricks, Snowflake, and Apache Airflow.
• Infrastructure: Experience with Terraform or similar IaC tools for scalable deployments.
• AI Competency: Proficiency in using AI IDEs (Cursor/Windsurf) and integrating AI/ML models into production data flows.

Preferred Qualifications:

• Exposure to data governance and cataloging tools (e.g., Unity Catalog).
• Knowledge of performance tuning for massive-scale big data systems.
• Familiarity with real-time data processing frameworks.
• Experience in digital transformation and sustainability-focused data projects.

Desired Competencies (Technical/Behavioral Competency)

Must-Have

·       Strong understanding of Kafka concepts, including topics, partitions, consumers, producers, and security.

·       Experience with testing Kafka Connect, Kafka Streams, and other Kafka ecosystem components.

·       API Testing Experience

·       X-RAY and Test Automation Experience

·       Expertise with Postman/SOAP

·       Agile/JIRA/Confluence

·       Strong familiarity such as XML, JSON, CSV, Avro, etc.

·       Strong hands-on SQL, Mongo.

·       Continuous integration and automated testing.

·       Working knowledge and experience of Git.

Good-to-Have

·       Troubleshoot Kafka related issues, Strong in Kafka client configuration and troubleshooting

SN

Responsibility of / Expectations from the Role

1

Engage with the customer to understand the requirements, provide technical solutions, provide value added suggestions

2

Help build and manage the team of Kafka and Java developers in the near future.

Key Responsibilities:

• Analyze and report on quality trends, root causes, and areas for improvement.
• Ensure compliance with internal policies, industry standards, and client specifications.
• Assist in developing and improving quality assurance processes, procedures, and templates.
• Perform audits, inspections, and evaluations of processes and systems.
• Provide feedback and training to staff on quality issues and performance metrics.

Key Skills:

• Strong analytical and problem-solving skills
• Attention to detail
• Excellent written and verbal communication
• Proficiency in using QA/testing tools and software

Your Impact:

•                 Implements Digital Consumer experiences based on a foundation of SFCC (Salesforce commerce cloud) to meet expected quality standards

•                 Ensures functional requirements and high-level solution designs are understood and are translated into detailed technical design

•                 Implement proofs of concept to prove any new technologies, application flows or integration scenarios and identify customizations needed to SFCC platform for meeting client requirements

•                 Guides the performance tuning and scalability of the ecommerce solution

•                 Diagnose and solve technical problems during implementation and support phases

Your Skills & Experience:

•                 4-8 Years with 2+ years of implementing commerce application on any eCommerce platform in last 3 years

•                 If you have not worked in eCommerce platform, we can still discuss with you if you are expert in Java/.Net and understand REST/SOAP webservices.

•                 Demonstrate proficiency in build and release management using CI/CD strategies (leveraging tools like Jenkins and DevOps practices), implement Mocha/Chai unit and functional test cases in order to reduce costs while ensuring quality

•                 Contributes to designing, implementing and documenting the build release process as well as system configuration and deployment characteristics of the applications

•                 Participates in technical walkthroughs/ code reviews of other team members’ components, test plans and results and help them with gaps

•                 Collaborate with architect to define implementation processes and quality gates and standards

•                 Identifies risks and issues, and help monitor them

•                 Guides the development of reference and resource materials

•                 Mentors developers in improving their platform knowledge and development skills

•                 Communicate technical design to the developers and help/guide them in the implementation

•                 Participates in Agile sprints

Set Yourself Apart With:

•                 eCommerce platform certification

•                 Security considerations – OWASP, CSRF, reCAPTCHA etc. – Basic knowledge or experience

•                 Git Development workflow – Proficient

•                 Visual Studio Code IDE or Eclipse IDE - Proficient

•                 Knowledge on Agile methodology and desired tools like Jira, confluence etc.

•                 Scripting/development experience with Node.js, Mocha/Chai

•                 Experience working in any Salesforce cloud environment like SFMC, SFSC etc.

•                 Excellent written, verbal communication and articulation skills & Good team player

•                 Self-starter and self-learner with keen interest to grow

•                 Process orientation and the ability to define and setup processes

•                 Ability to provide necessary coaching to bring team members up to speed on the technology

•                 Ability to prioritize and manage multiple tasks

•                 Excellent and innovative approach to problem solving and finding solutions

•                 Flexible and proactive/self-motivated working style with excellent personal ownership of problem resolution

• Maintain UNIX/Linux Operating System
• Create and maintain environment for running batch jobs associated with daily batch cycle and batch reporting subsystem
• Work with LAN/Network personnel to ensure compatibility with LAN applications and peripheral hardware to provide end users with reliable and stable working environment
• Support and maintain other vendor database software installed on the LAN servers
• Perform User Access Management.
• Experience supporting day-today administration functions including user account management and script creation
• Apply Patches and Upgrades as necessary. Perform tasks for Backup and Recovery Management including High Availability
• Install and configure storage arrays and allocate SAN and NAS storage to different OS platforms and/or administer ZFS storage pools, file system, snapshots, and clones.
• Experience in Linux hypervisors KVM, XEN.
• Good knowledge on open stack administration and service operations.
• Good experience on veritas net backup software.
• Strong experience in public clouds AWS, AZURE, GCP.
• Solid knowledge of protocols such as DNS, HTTP, LDAP, SMTP and SNM
• Good understanding on AWS outposts hybrid environment.
• Good experience in the windows server and OS side.
• Experience in Monitoring platforms like Zabbix, ELK, Grafana.
• Troubleshoot Hardware Issues, Installation, and testing of computer peripherals
• Perform/Implement Security Monitoring and audit to identify any possible security intrusions or breaches.
• Collaborate with other teams and team members to develop automation strategies and deployment processes

Qualifications:

1. Degree in Computer Science, Computer Engineer, or other related courses
2. Minimum of 10 years’ experience
3. In-depth knowledge of Linux: RedHat, CentOS, Debian, etc.
4. In-depth knowledge in Linux virtualization
5. good knowledge of UNIX and LINUX Operating systems, file systems, storage environments, and networking protocols
6. Knowledgeable in Unix, Linux, Unix, Linux shell scripting. Practical scripting skills in Shell, PERL, Batch, Python
7. good DBA skills MySQL, MariaDB, PostgreSQL, Timescale, MongoDB
8. Basic Messaging and Collaboration concepts and tools, Server Virtualization
9. Practical understanding of Networking - routing, subnets, UDP, TCP, IP, and VLANs
10. Backup and Storage Management (Netback up mandatory).
11. Familiar and can identify or interpret crash dump files and core dump files and monitor System Logs