Azure DevOps Engineer

Job Summary

The Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python. The role involves designing, developing, and implementing data solutions while ensuring they meet the business requirements and align with best practices. (1.) Key Responsibilities

1. Lead and manage end-to-end data engineering projects using azure data factory, azure databricks, sql, oracle pl/sql, and python.

2. Collaborate with stakeholders to gather and understand requirements for data pipelines and analytics solutions.

3. Design and develop etl processes, data models, and data integration solutions.

4. Provide technical guidance and mentorship to the team members.

5. Ensure data quality, data governance, and data security standards are maintained throughout the project lifecycle.

6. Troubleshoot and optimize data pipelines and processes for performance and efficiency.

7. Stay updated on the latest trends and technologies in data engineering and contribute to continuous improvement efforts.

Skill Requirements

1. Proficiency in azure data factory (adf) and azure databricks for building and managing data pipelines.

2. Strong experience with sql and oracle pl/sql for data querying and manipulation.

3. Advanced programming skills in python for scripting and data processing tasks.

4. Knowledge of data modeling, data warehousing concepts, and database design principles.

5. Ability to work in a collaborative team environment and communicate effectively with stakeholders.

6. Strong analytical and problem-solving skills with attention to detail.

7. Experience in data visualization tools and techniques is a plus.

Certifications: Relevant certifications in Azure Data Factory, Azure Databricks, SQL, Oracle PL/SQL, or Python are advantageous.

Skill (Primary)

Data Fabric-Azure-Azure Data Factory (ADF)

Role: Full-Time, Long-Term Required: Docker, GCP, CI/CD Preferred: Experience with ML pipelines

OVERVIEW

We are seeking a DevOps engineer to join as a core member of our technical team. This is a long-term position for someone who wants to own infrastructure and deployment for a production machine learning system. You will ensure our prediction pipeline runs reliably, deploys smoothly, and scales as needed.

The ideal candidate thinks about failure modes obsessively, automates everything possible, and builds systems that run without constant attention.

CORE TECHNICAL REQUIREMENTS

Docker (Required): Deep experience with containerization. Efficient Dockerfiles, layer caching, multi-stage builds, debugging container issues. Experience with Docker Compose for local development.

Google Cloud Platform (Required): Strong GCP experience: Cloud Run for serverless containers, Compute Engine for VMs, Artifact Registry for images, Cloud Storage, IAM. You can navigate the console but prefer scripting everything.

CI/CD (Required): Build and maintain deployment pipelines. GitHub Actions required. You automate testing, building, pushing, and deploying. You understand the difference between continuous integration and continuous deployment.

Linux Administration (Required): Comfortable on the command line. SSH, diagnose problems, manage services, read logs, fix things. Bash scripting is second nature.

PostgreSQL (Required): Database administration basics—backups, monitoring, connection management, basic performance tuning. Not a DBA, but comfortable keeping a production database healthy.

Infrastructure as Code (Preferred): Terraform, Pulumi, or similar. Infrastructure should be versioned, reviewed, and reproducible—not clicked together in a console.

WHAT YOU WILL OWN

Deployment Pipeline: Maintaining and improving deployment scripts and CI/CD workflows. Code moves from commit to production reliably with appropriate testing gates.

Cloud Run Services: Managing deployments for model fitting, data cleansing, and signal discovery services. Monitor health, optimize cold starts, handle scaling.

VM Infrastructure: PostgreSQL and Streamlit on GCP VMs. Instance management, updates, backups, security.

Container Registry: Managing images in GitHub Container Registry and Google Artifact Registry. Cleanup policies, versioning, access control.

Monitoring and Alerting: Building observability. Logging, metrics, health checks, alerting. Know when things break before users tell us.

Environment Management: Configuration across local and production. Secrets management. Environment parity where it matters.

WHAT SUCCESS LOOKS LIKE

Deployments are boring—no drama, no surprises. Systems recover automatically from transient failures. Engineers deploy with confidence. Infrastructure changes are versioned and reproducible. Costs are reasonable and resources scale appropriately.

ENGINEERING STANDARDS

Automation First: If you do something twice, automate it. Manual processes are bugs waiting to happen.

Documentation: Runbooks, architecture diagrams, deployment guides. The next person can understand and operate the system.

Security Mindset: Secrets never in code. Least-privilege access. You think about attack surfaces.

Reliability Focus: Design for failure. Backups are tested. Recovery procedures exist and work.

CURRENT ENVIRONMENT

GCP (Cloud Run, Compute Engine, Artifact Registry, Cloud Storage), Docker, Docker Compose, GitHub Actions, PostgreSQL 16, Bash deployment scripts with Python wrapper.

WHAT WE ARE LOOKING FOR

Ownership Mentality: You see a problem, you fix it. You do not wait for assignment.

Calm Under Pressure: When production breaks, you diagnose methodically.

Communication: You explain infrastructure decisions to non-infrastructure people. You document what you build.

Long-Term Thinking: You build systems maintained for years, not quick fixes creating tech debt.

EDUCATION

University degree in Computer Science, Engineering, or related field preferred. Equivalent demonstrated expertise also considered.

TO APPLY

Include: (1) CV/resume, (2) Brief description of infrastructure you built or maintained, (3) Links to relevant work if available, (4) Availability and timezone.

DevOps Engineer

1.   Should have at least 5+ years of experience

2.   Should have working experience in Docker, Microservices Architecture Application Deployment, GitHub Container Registry, GitHub Actions, Load Balancer, Nginx Web server, 

3.   Should have working expertise in CI/CD tool

4.   Should have working experience with the bash script

5.   Good to have at least one cloud platform services knowledge

DevOps Engineer

Our engineering team is looking for Big-Data DevOps engineers to join the engineering team and help us automate the build, release, packaging and infrastructure provisioning and support processes. The candidate is expected to own the full life-cycle of provisioning, configuration management, monitoring, maintenance and support for cloud as well as on premise deployments.

Responsibilities

• 3-plus years of DevOps experience managing the Big Data application stack including HDFS, YARN, Spark, Hive and Hbase
• Deeper understanding of all the configurations required for installing and maintaining the infrastructure in the long run
• Experience setting up high availability, configuring resource allocation, setting up capacity schedulers, handling data recovery tasks
• Experience with middle-layer technologies including web servers (httpd, ningx),

application servers (Jboss, Tomcat) and database systems (postgres, mysql)

• Experience setting up enterprise security solutions including setting up active directories, firewalls, SSL certificates, Kerberos KDC servers, etc.
• Experience maintaining and hardening the infrastructure by regularly applying required security packages and patches
• Experience supporting on-premise solutions as well as on AWS cloud
• Experience working with and supporting Spark-based applications on YARN
• Experience with one or more automation tools such as Ansible, Terraform, etc
• Experience working with CI/CD tools like Jenkins and various test report and coverage Plugins
• Experience defining and automating the build, versioning and release processes for complex enterprise products
• Experience supporting clients remotely and on-site
• Experience working with and supporting Java- and Python-based tech stacks would be a Plus

Hi,

We are looking for candidate with experience in DevSecOps.

Please find below JD for your reference.

Responsibilities:

Execute shell scripts for seamless automation and system management.

Implement infrastructure as code using Terraform for AWS, Kubernetes, Helm, kustomize, and kubectl.

Oversee AWS security groups, VPC configurations, and utilize Aviatrix for efficient network orchestration.

Contribute to Opentelemetry Collector for enhanced observability.

Implement microsegmentation using AWS native resources and Aviatrix for commercial routes.

Enforce policies through Open Policy Agent (OPA) integration.

Develop and maintain comprehensive runbooks for standard operating procedures.

Utilize packet tracing for network analysis and security optimization.

Apply OWASP tools and practices for robust web application security.

Integrate container vulnerability scanning tools seamlessly within CI/CD pipelines.

Define security requirements for source code repositories, binary repositories, and secrets managers in CI/CD pipelines.

Collaborate with software and platform engineers to infuse security principles into DevOps teams.

Regularly monitor and report project status to the management team.

Qualifications:

Proficient in shell scripting and automation.

Strong command of Terraform, AWS, Kubernetes, Helm, kustomize, and kubectl.

Deep understanding of AWS security practices, VPC configurations, and Aviatrix.

Familiarity with Opentelemetry for observability and OPA for policy enforcement.

Experience in packet tracing for network analysis.

Practical application of OWASP tools and web application security.

Integration of container vulnerability scanning tools within CI/CD pipelines.

Proven ability to define security requirements for source code repositories, binary repositories, and secrets managers in CI/CD pipelines.

Collaboration expertise with DevOps teams for security integration.

Regular monitoring and reporting capabilities.

Site Reliability Engineering experience.

Hands-on proficiency with source code management tools, especially Git.

Cloud platform expertise (AWS, Azure, or GCP) with hands-on experience in deploying and managing applications.

Please send across your updated profile.

1. Proficiency in Python , Django and Other Allied Frameworks;
2. Expert in designing UI/UX interfaces;
3. Expert in testing, troubleshooting, debugging and problem solving;
4. Basic knowledge of SEO;
5. Good communication;
6. Team building and good acumen;
7. Ability to perform; 
8. Continuous learning

Roles and Responsibilities 

• Primary stakeholder collaborating with Dir Engineering on software/infrastructure architecture, monitoring/alerting framework and all other architectural level technical issues
• Design and manage implementation of Silvermine’s high performance, scalable, extensible and resilient microservices application stack based of existing, partially migrated monolithic application and for new product development.  Includes:
  • Utilizing either ECS Fargate (no EC2 clusters) or EKS as the orchestration framework – to be tested up to a minimum of 100k concurrent users
  • Exploring, designing and implementing use of on demand compute (Lambda) where appropriate
  • Scalable and redundant data architecture supporting microservices design principles
  • A scalable reverse proxy layer to isolate microservices from managing network connections
  • Utilizing CDN capabilities to offload origin load via an intelligent caching strategy
  • Leveraging best in breed AWS service offerings to enable team to focus on application stack instead of application scaffolding while minimizing operational complexity and cost
  • Monitoring and optimizing of stack for
• Security and monitoring
  • Leverage AWS and 3rd party services to monitor the application stack and data; secure them from DDOS attacks and security breaches; and alert the team in the vent of an incident
  • Using APM and logging tools:
    • Monitor application stack and infrastructure component performance
    • Proactively detect, triage and mitigate stack performance issues
    • Alert upon exception events
    • Provide triaging tools for debugging and Root Cause Analysis.
• Enhance the CI/CD pipeline to support automated testing, a resilient deployment model (e.g., blue-green, canary) and 100% rollback support (including the data layer)
• Development a comprehensive, supportable, repeatable IAC implementation using CloudFormation or Terraform
• Take a leadership role and exhibit expertise in the development of standards, architectural governance, design patterns, best practices and optimization of existing architecture.
• Partner with teams and leaders to provide strategic consultation for business process design/optimization, creating strategic technology road maps, performing rapid prototyping and implementing technical solutions to accelerate the fulfillment of the business strategic vision.
• Staying up to date on emerging technologies (AI, Automation, Cloud etc.) and trends with a clear focus on productivity, ease of use and fit-for-purpose, by researching, testing, and evaluating.
• Providing POCs and product implementation guidelines.
• Applying imagination and innovation by creating, inventing, and implementing new or better approaches, alternatives and breakthrough ideas that are valued by customers within the function.
• Assessing current state of solutions, defining future state needs, identifying gaps and recommending new technology solutions and strategic business execution improvements.
• Overseeing and facilitating the evaluation and selection technology, product standards and the design of standard configurations/implementation patterns.
• Partnering with other architects and solution owners to create standards and set strategies for the enterprise.
• Communicating directly with business colleagues on applying digital workplace technologies to solve identified business challenges.

Skills Required:

• Good mentorship skills to coach and guide the team on AWS DevOps.
• Jenkins, Python, Pipeline as Code, Cloud Formation Templates and Terraform.
• Experience with Dockers, Containers, Lambda and Fargate is a must
• Experience with CI/CD and Release management
• Strong proficiency in PowerShell scripting
• Demonstrable expertise in Java
• Familiarity with REST APIs

Qualifications:

• Minimum of 5 years of relevant experience in Devops.
• Bachelors or Masters in Computer Science or equivalent degree.
• AWS Certifications is added advantage

Rules & Responsibilities:

• Design, implement and maintain all AWS infrastructure and services within a managed service environment
• Should be able to work on 24 X 7 shifts for support of infrastructure.
• Design, Deploy and maintain enterprise class security, network and systems management applications within an AWS environment
• Design and implement availability, scalability, and performance plans for the AWS managed service environment
• Continual re-evaluation of existing stack and infrastructure to maintain optimal performance, availability and security
• Manage the production deployment and deployment automation
• Implement process and quality improvements through task automation
• Institute infrastructure as code, security automation and automation or routine maintenance tasks
• Experience with containerization and orchestration tools like docker, Kubernetes
• Build, Deploy and Manage Kubernetes clusters thru automation
• Create and deliver knowledge sharing presentations and documentation for support teams
• Learning on the job and explore new technologies with little supervision
• Work effectively with onsite/offshore teams

Qualifications:

• Must have Bachelor's degree in Computer Science or related field and 4+ years of experience in IT
• Experience in designing, implementing, and maintaining all AWS infrastructure and services
• Design and implement availability, scalability, and performance plans for the AWS managed service environment
• Continual re-evaluation of existing stack and infrastructure to maintain optimal performance, availability, and security
• Hands-on technical expertise in Security Architecture, automation, integration, and deployment
• Familiarity with compliance & security standards across the enterprise IT landscape
• Extensive experience with Kubernetes and AWS(IAM, Route53, SSM, S3, EFS, EBS, ELB, Lambda, CloudWatch, CloudTrail, SQS, SNS, RDS, Cloud Formation, DynamoDB)
• Solid understanding of AWS IAM Roles and Policies
• Solid Linux experience with a focus on web (Apache Tomcat/Nginx)
• Experience with automation/configuration management using Terraform\Chef\Ansible or similar.
• Understanding of protocols/technologies like Microservices, HTTP/HTTPS, SSL/TLS, LDAP, JDBC, SQL, HTML
• Experience in managing and working with the offshore teams
• Familiarity with CI/CD systems such as Jenkins, GitLab CI
• Scripting experience (Python, Bash, etc.)
• AWS, Kubernetes Certification is preferred
• Ability to work with and influence Engineering teams

Key Responsibilities :

Look through the various DevOps Tools/Technologies and identify the strengths and provide direction to the DevOps automation team

Out-of-box thought process on the DevOps Automation Platform implementation

Expose various tools and technologies and do POC on integration of the these tools

Evaluate Backend API's for various DevOps tools

Perform code reviews keep in context of RASUI

Mentor the team on the various E2E integrations

Be Liaison in evangelizing the automation solution currently implemented

Bring in various DevOps best Practices/Principles and participate in adoption with various app teams

Must have:

Should possess Bachelors/Masters in computer science with minimum of 8+ years of experience

Should possess minimum 3 years of strong experience in DevOps

Should possess expertise in using various DevOps tools libraries and API's (Jenkins/JIRA/AWX/Nexus/GitHub/BitBucket/SonarQube)

Should possess expertise in optimizing the DevOps stack ( Containers/Kubernetes/Monitoring )

2+ Experience in creating solutions and translate to the development team

Should have strong understanding of OOPs, SDLC (Agile Safe standards)

Proficient in Python , with a good knowledge of its ecosystems (IDEs and Frameworks)

Proficient in various cloud platforms (Azure/AWS/Google cloud platform)

Proficient in various DevOps offerings (Pivotal/OpenStack/Azure DevOps