Data Engineer (Pyspark Developer)

Job Title : Software Engineer (.NET, Azure)

Location : Remote

Employment Type : [Full-time/Contract]

Experience Level : 3+ Years

Job Summary :

We are looking for a skilled Software Engineer (.NET, Azure) to develop high-quality, secure, and scalable software solutions. You will collaborate with product owners, security specialists, and engineers to deliver robust applications.

Responsibilities :

• Develop and maintain server-side software using .NET (C#), SQL, and Azure.
• Build and secure RESTful APIs.
• Deploy and manage applications on Azure.
• Ensure version control using Azure DevOps/Git.
• Write clean, maintainable, and scalable code.
• Debug and optimize application performance.

Qualifications :

• 3+ Years of server-side development experience.
• Strong proficiency in .NET, SQL, and Azure.
• Experience with RESTful APIs and DevOps/Git.
• Ability to work collaboratively and independently.
• Familiarity with Scrum methodologies.

Roles & Responsibilities

• Data Engineering Excellence: Design and implement data pipelines using formats like JSON, Parquet, CSV, and ORC, utilizing batch and streaming ingestion.
• Cloud Data Migration Leadership: Lead cloud migration projects, developing scalable Spark pipelines.
• Medallion Architecture: Implement Bronze, Silver, and gold tables for scalable data systems.
• Spark Code Optimization: Optimize Spark code to ensure efficient cloud migration.
• Data Modeling: Develop and maintain data models with strong governance practices.
• Data Cataloging & Quality: Implement cataloging strategies with Unity Catalog to maintain high-quality data.
• Delta Live Table Leadership: Lead the design and implementation of Delta Live Tables (DLT) pipelines for secure, tamper-resistant data management.
• Customer Collaboration: Collaborate with clients to optimize cloud migrations and ensure best practices in design and governance.

Educational Qualifications

• Experience: Minimum 5 years of hands-on experience in data engineering, with a proven track record in complex pipeline development and cloud-based data migration projects.
• Education: Bachelor’s or higher degree in Computer Science, Data Engineering, or a related field.
• Skills
• Must-have: Proficiency in Spark, SQL, Python, and other relevant data processing technologies. Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations. Expertise in on-premises to cloud Spark code optimization and Medallion Architecture.

Good to Have

• Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus).

Soft Skills

• Excellent communication and collaboration skills, with the ability to work effectively with clients and internal teams.
• Certifications
• AWS/GCP/Azure Data Engineer Certification.

Key Responsibilities:

●    Software Development: Iteratively and incrementally design, develop, test and maintain applications and services using C# and .NET.

●    CI/CD Pipelines: Develop, maintain, and optimize continuous integration and continuous delivery pipelines using GitLab, Devtron and Kubernetes.

●    Teamwork and Pair/Mob Programming: Engage with developers, operations, and team members via pair or mob programming sessions to ensure the highest quality product delivery.

●    Containerization: Contribute to the push toward full containerization and zero-downtime deployment goals

●    DevOps Practices: Implement and maintain infrastructure as code (IaC) using tools such as bicep and Terraform.

●    Monitoring and Logging: Implement and manage monitoring, logging, and alerting solutions using tools like OpenTelemetry, Prometheus, Grafana to make our products more supportable.

●    Design and Architecture: Contribute to on-going discussion of our evolving software design and architecture.

●    Cloud Management: Assist in managing and optimizing our private cloud infrastructure (VMWare Tanzu) to ensure high availability, scalability, and efficient resource usage.

●    Security: Implement security best practices and ensure compliance with relevant regulations and standards.

●    Automation: Identify opportunities for automation to improve efficiency, reduce manual efforts and deskilling in deployment, testing and maintenance tasks.

●    Troubleshooting: Diagnose and resolve infrastructure and application issues promptly and effectively.

●    Documentation: Create and maintain comprehensive documentation for applications, infrastructure, processes, and procedures.

●    Continuous Improvement: Advocate for and implement best practices promoting a culture of continuous improvement.

Qualifications:

Minimum of 5 years of experience in software development, particularly with C# and .NET.

Must have skills:

●    Strong knowledge of C# programming language and .NET stack

●    Familiarity with CI/CD tools and practices, including TDD.

●    Understanding of DevOps principles

●    Strong collaboration and communication skills.

Nice to have skills

●    Familiarity with containerization and orchestration tools like Docker and Kubernetes is a plus.

●    Knowledge of infrastructure as code (IaC).

●    Experience in scripting languages such as Python, Bash, or PowerShell is a bonus.

●    Excellent problem-solving skills and attention to detail.

●    Experience with pair/mob programming.

●    Understanding of networking concepts and security best practices.

Role: DevOps Engineer

Exp: 4 - 7 Years

CTC: up to 28 LPA

Key Responsibilities

•   Design, build, and manage scalable infrastructure on cloud platforms (GCP, AWS, Azure, or OCI)

•   Administer and optimize Kubernetes clusters and container runtimes (Docker, containerd)

•   Develop and maintain CI/CD pipelines for multiple services and environments

•   Manage infrastructure as code using tools like Terraform and/or Pulumi

•   Automate operations with Python and shell scripting for deployment, monitoring, and maintenance

•   Ensure high availability and performance of production systems and troubleshoot incidents effectively

•   Monitor system metrics and implement observability best practices using tools like Prometheus, Grafana, ELK, etc.

•   Collaborate with development, security, and product teams to align infrastructure with business needs

•   Apply best practices in cloud networking, Linux administration, and configuration management

•   Support compliance and security audits; assist with implementation of cloud security measures (e.g., firewalls, IDS/IPS, IAM hardening)

•   Participate in on-call rotations and incident response activities

If Interested kindly share your updated resume on 82008 31681

Required Skills/Qualifications: 

∙         B.Tech/MCA in Comput

Required Skills/Qualifications: 

Any Backend Technology is acceptable

∙         B.Tech/MCA in Computer Science or equivalent with 1-3 years’ experience with server-side web application development.

∙         Extensive development experience using LAMP or MEAN stack, RESTful web services or Node.js, HTML and CSS.

∙         Good understanding of Data structures and Relational Databases like MySQL or NoSQL Databases like MongoDB.

∙         Experience working with services in AWS such as EC2, RDS, and ELBs and have knowledge of VPCs.

∙         Experience with server side and client side MVC frameworks, Kafka Ansilble,Jenkins.

∙         Exposure to Continuous Integration (CI) and Continuous Deployment (CD), automated testing and agile development methods.

∙         Understanding of Version Management Tools like GitHub.

er Science or equivalent with 1-3 years’ experience with server-side web application development.

∙         Extensive development experience using LAMP or MEAN stack, RESTful web services or Node.js, HTML and CSS.

∙         Good understanding of Data structures and Relational Databases like MySQL or NoSQL Databases like MongoDB.

∙         Experience working with services in AWS such as EC2, RDS, and ELBs and have knowledge of VPCs.

∙         Experience with server side and client side MVC frameworks, Kafka Ansilble,Jenkins.

∙         Exposure to Continuous Integration (CI) and Continuous Deployment (CD), automated testing and agile development methods.

∙         Understanding of Version Management Tools like GitHub.

Experience:

The candidate should have about 1+ years of experience with design and development in Java/Scala. Experience in algorithm, data-structure, database is mandatory.

Required Skills:

1. Java or Scala
2. Extensive experience in developing web applications. Should possess good Oops knowledge and be aware of enterprise application design patterns.
3. Should have the ability to analyze, design, develop and test complexity of spark job.
4. Basic working knowledge of Unix/Linux.

Good To have: -

1. Python
2. Distributed Computing

About Us

We help our customers to Go Digital at 3x Less Cost and 3x Less Time!

appZUI nGage, our App-as-Service platform, accelerates the digital idea to profit journey. Serving more than 250K consumers in US, UK, Switzerland, Australia, India and other geographies with several exciting digital transformations.

Know more:https://bit.ly/az-godigital">http://bit.ly/az-godigital

Role & Responsibilities

We are looking for a Senior developer/Architect to be part of our growing team. In this role you will be handling

• Design and architecture of various core platform components
• Develop and deliver various products for our customers
• Mentor and develop various team members

Skills & Capabilities

• System design for function, scale, security and maintenance
• Data engineering concepts and practice
• Test driven development
• Server-side skills: Nodejs/.Net/Java, Devops, SQL, microservices
• DevOps & SRE skills: Jenkins, Prometheus, Grafana
• System modelling and visualization skills
• Written & oral communication skills
• Front-end skills: (nice to have) Ionic, Angular/React, Javascript, HTML, CSS

What you can expect from us

• Product engineering culture: hypothesis-develop-test-evolve
• Failures are part of learning and growing
• Organization structure and processes to maximize creativity and openness
• Success is about 5 WINS: Customer, Employee, Supplier, Society and Investor