
Profile: AWS Data Engineer
Mode- Hybrid
Experience- 5+7 years
Locations - Bengaluru, Pune, Chennai, Mumbai, Gurugram
Roles and Responsibilities
- Design and maintain ETL pipelines using AWS Glue and Python/PySpark
- Optimize SQL queries for Redshift and Athena
- Develop Lambda functions for serverless data processing
- Configure AWS DMS for database migration and replication
- Implement infrastructure as code with CloudFormation
- Build optimized data models for performance
- Manage RDS databases and AWS service integrations
- Troubleshoot and improve data processing efficiency
- Gather requirements from business stakeholders
- Implement data quality checks and validation
- Document data pipelines and architecture
- Monitor workflows and implement alerting
- Keep current with AWS services and best practices
Required Technical Expertise:
- Python/PySpark for data processing
- AWS Glue for ETL operations
- Redshift and Athena for data querying
- AWS Lambda and serverless architecture
- AWS DMS and RDS management
- CloudFormation for infrastructure
- SQL optimization and performance tuning

Similar jobs
Design, test, and build scalable backend python services
Closely collaborate with marketing and product teams to build innovative, robust and
easy-to-use features that serve.
Developing high quality code based on detailed designs that cater to the product
requirements.
Responsible for troubleshooting, testing and maintaining the core product software and
databases to ensure strong optimisation and functionality
Required Skills:
Degree in Computer Science, Software Engineering or equivalent.
Minimum 3+ years experience in software development.
Expertise in Python 3.7, Django 2.2+ and REST APIs.
Willingness to learn and ability to flourish in a dynamic, high-growth, entrepreneurial
environment
Hands-on, self-starter, capable of working independently
Skills Required:
- Good experience with programming language Python
- Strong experience in Docker.
- Good knowledge with any of the Cloud Platform like Azure.
- Must be comfortable working in a Linux environment.
- Must have exposure into IOT domain and its protocols ((Zigbee & BLE ,LoRa,Modbus)
- Must be a good team player.
- Strong Communication Skills

- Experience building and managing large scale data/analytics systems.
- Have a strong grasp of CS fundamentals and excellent problem solving abilities. Have a good
understanding of software design principles and architectural best practices.
- Be passionate about writing code and have experience coding in multiple languages, including at least
one scripting language, preferably Python.
- Be able to argue convincingly why feature X of language Y rocks/sucks, or why a certain design decision
is right/wrong, and so on.
- Be a self-starter—someone who thrives in fast paced environments with minimal ‘management’.
- Have exposure and working knowledge in AI environment with Machine learning experience
- Have experience working with multiple storage and indexing technologies such as MySQL, Redis,
MongoDB, Cassandra, Elastic.
- Good knowledge (including internals) of messaging systems such as Kafka and RabbitMQ.
- Use the command line like a pro. Be proficient in Git and other essential software development tools.
- Working knowledge of large-scale computational models such as MapReduce and Spark is a bonus.
- Exposure to one or more centralized logging, monitoring, and instrumentation tools, such as Kibana,
Graylog, StatsD, Datadog etc
- 3 - 5 years of progressive IT experience with a minimum of 1+ years in building Ruby on Rails Web applications and RESTful API endpoints
- Proficient with scripting languages, specifically Python and Bash
- Experience in Object Oriented Java Script.
- Must have Linux command line experience.
- Good experience in developing and integrating reliable REST web services.
- Possess good communication, organizational and creative thinking skills.
- Multi-tasking individual that likes to work both in a team environment and individually
- Excellent technical abilities, leadership, decision making, and adaptability to new technology
- Good knowledge of Agile processes and practice
- Team player
3+ years of experience with development in JAVA technology.
Strong Java Basics
SpringBoot or Spring MVC
Hands-on experience with Relational Databases (SQL query or Hibernate) + Mongo (JSON parsing)
Proficient in REST API development
Messaging Queue (RabitMQ or Kafka)
Microservices
Any Caching Mechanism
Good at problem-solving
Good to Have Skills:
4+ years of experience in using Java/J2EE tech stacks
Good understanding of data structures and algorithms.
Excellent analytical and problem-solving skills.
Ability to work in a fast-paced internet start-up environment.
Experience in technical mentorship/coaching is highly desirable.
Understanding of AI/ML algorithms is a plus.
BASIC QUALIFICATIONS
- At least 8+ years of IT experience most of which will have been in helping the design and implementation of the software suite/module.
- 2+ years on any Cloud Platform (AWS, Azure, Google etc).
- Bachelor’s degree in Information Science / Information Technology, Computer Science, Engineering, Mathematics, Physics, or a related field.
- Strong verbal and written communication skills, with the ability to work effectively across internal and external organizations.
- Strong programming skills in Java.
- Strong hands-on experience in integrating multiple databases like Oracle, SQL Server, PostgreSQL etc.
- Deep hands-on experience in the design, development and deployment of business software at scale.
- Customer facing skills to represent AWS well within the customer’s environment and drive discussions with senior personnel regarding trade-offs, best practices, project management and risk mitigation
- Leading/Involved in highly-available and fault-tolerant enterprise and web-scale software applications.
- Experience in performance optimization techniques
- High end Troubleshooting and Communication skills.
- Proven experience with software development life cycle (SDLC) and agile/iterative methodologies required
PREFERRED QUALIFICATIONS
- Implementing experience with primary AWS services (EC2, ELB, RDS, Lambda, API Gateway Route53 & S3).
- AWS Solutions Architect Certified
- Experience in programming languages like Java/Python.
- Demonstrated ability to think strategically about business, product, and technical challenges
- Integration of AWS cloud services with on-premise technologies from Microsoft, IBM, Oracle, HP, SAP etc.
- Experience with IT compliance and risk management requirements (eg. security, privacy, SOX, HIPAA etc.).
- Extended travel to customer locations may be required to sell and deliver professional services as needed
Experience Required -2 years to 6 years
Location -A-45 Atrium of Quark city Mohali
You will be responsible for reviewing requirements, optimizing code, writing documentation,
writing tests with proper coverage, and following all the best standards.
Requirements:
- Django
- Flask
- Django REST Framework
- Pytest
- Docker
Diagnal Technologies is a fully owned subsidiary of Diagnal Inc, Hong Kong. It’s a product company focusing on media and entertainment clients, creating Video on Demand apps for their customers. We are a team of OTT video solution specialists helping our customers build the new digital media industry in Asia Pacific and beyond. We create viable revenue generating video streaming products, enable our customers to gain business and operational insights through big data analytics and empower them to engage with their users more effectively through targeted marketing campaigns. Started its Development Center in Technopark Phase III campus and focusing on Indian and International clients.
For further details please refer our website
LinkedIN :
Job Description :
- 2 or more years of experience of server side web systems engineering and middleware in a project services or product engineering organisation
- Hands-on development experience using JavaScript on the Server Side with Node.js and associated frameworks.
- Published sites/systems that are now online
- Self motivation and ability to manage your own time to get the job done at the high international quality levels we expect
- An engineering Degree in computer science or equivalent practical experience
- A solid understanding of networking, storage, operating/browser systems engineering as well as application/software systems performance optimisation and third party systems integration
- Hands on experience with Amazon Web Services
- Prior experience working within the Media, E-commerce, Telecommunications or similar large scale consumer facing industry
First round will be a Machine test and Aptitude, sharing it with this mail, which has to be completed in 2 days.This will be followed by technical discussion, CEO round and HR discussion.Hiring process will take 5-7 working days post completion of M/C test






