Data Engineer - AWS

We are looking for a skilled and motivated Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) data pipelines. The role involves working with various GCP services, implementing data ingestion and transformation logic, and ensuring data quality and consistency across systems.

Key Responsibilities:

• Design, develop, test, and maintain scalable ETL data pipelines using Python.
• Work extensively on Google Cloud Platform (GCP) services such as:
• Dataflow for real-time and batch data processing
• Cloud Functions for lightweight serverless compute
• BigQuery for data warehousing and analytics
• Cloud Composer for orchestration of data workflows (based on Apache Airflow)
• Google Cloud Storage (GCS) for managing data at scale
• IAM for access control and security
• Cloud Run for containerized applications
• Perform data ingestion from various sources and apply transformation and cleansing logic to ensure high-quality data delivery.
• Implement and enforce data quality checks, validation rules, and monitoring.
• Collaborate with data scientists, analysts, and other engineering teams to understand data needs and deliver efficient data solutions.
• Manage version control using GitHub and participate in CI/CD pipeline deployments for data projects.
• Write complex SQL queries for data extraction and validation from relational databases such as SQL Server, Oracle, or PostgreSQL.
• Document pipeline designs, data flow diagrams, and operational support procedures.
• 
  

Required Skills:

• 4–8 years of hands-on experience in Python for backend or data engineering projects.
• Strong understanding and working experience with GCP cloud services (especially Dataflow, BigQuery, Cloud Functions, Cloud Composer, etc.).
• Solid understanding of data pipeline architecture, data integration, and transformation techniques.
• Experience in working with version control systems like GitHub and knowledge of CI/CD practices.
• Strong experience in SQL with at least one enterprise database (SQL Server, Oracle, PostgreSQL, etc.).

Location: Chennai

Ideal Candidates for this Role should have:

• Minimum 5 years experience in testing.
• The candidate should have strong experience in Rest Assured, API Testing and Java.
• Proficiency in API testing tools like Postman
• Good knowledge in automation testing using Selenium WebDriver with Java.
• The candidate should be able to write a program to retrieve/parse the Json
• Good understanding of REST API types (GET, PUT, POST, DELETE) and its working.
• Good understanding of HTTP, JSON protocol and syntax
• Good knowledge in automation testing using Selenium WebDriver with Java.
  Be able to work as an individual contributor and be an Expert at writing automated scripts and be 100% hands-on
  Should have automated 1000s of Test cases at various complexity levels and workflows
• Design, implementation, and delivery of scalable, maintainable, configurable and robust test automation frameworks.
• Ability to analyze and translate requirements and development stories into test scripts to ensure complete test coverage.
• Should have strong knowledge of continuous integration tools like Hudson and Jenkins.
• Strong understanding of testing and automation best practices.
• Proven experience in functional, regression and cross-browser testing.
• Willingness to learn new technologies, approaches and test tools

InViz is Bangalore Based Startup helping Enterprises simplifying the Search and Discovery experiences for both their end customers as well as their internal users. We use state-of-the-art technologies in Computer Vision, Natural Language Processing, Text Mining, and other ML techniques to extract information/concepts from data of different formats- text, images, videos and make them easily discoverable through simple human-friendly touchpoints. 

TSDE - Data 

Data Engineer:

• Should have total 3-6 Yrs of experience in Data Engineering.
• Person should have experience in coding data pipeline on GCP. 
• Prior experience on Hadoop systems is ideal as candidate may not have total GCP experience. 
• Strong on programming languages like Scala, Python, Java. 
• Good understanding of various data storage formats and it’s advantages. 
• Should have exposure on GCP tools to develop end to end data pipeline for various scenarios (including ingesting data from traditional data bases as well as integration of API based data sources). 
• Should have Business mindset to understand data and how it will be used for BI and Analytics purposes. 
• Data Engineer Certification preferred 

Skills:

Full Stack Developer

Developed Gaming Application

Developed 3D Gaming Application

Basic knowledge of C, C++, MySQL, Mongo db

• 3 years Experience in Ruby on Rails
• Minimum 3 years in MongoDB / PostgreSQL
• Must be from Product based companies   

• Develop system and program specifications to meet client needs
• Analyze the impact of new systems or system changes on existing technology
• Design and maintain database structures to support system requirements
• Build and configure CRM forms, views, dashboards, and workflows
• Create and maintain schema and code to support system/program requirements
• Adhere to company standards, policies, and procedures
• Create and maintain appropriate user and technical documentation
• Provide client support, as needed
• Work with clients to resolve problems in a timely manner and escalate as appropriate
• Participate in company initiatives and committees

• Bachelor’s degree or higher degree in Computer Engineering, Computer Science, Information Technology, or any related field.
• Experience with CRM 2013/2015/2016, D365 CE modules such as Finance/Supply Chain/Sales/Marketing/Service/Field Services and CRM Portal
• Microsoft Dynamics Sales and Customer Engagement applications (CRM) product experience
• Professional CRM certification OR agree to achieve appropriate certification in an agreed-upon timeframe
• Experience implementing and customizing Dynamics CRM and Dynamics 365
• Demonstrated analytical, problem-solving, organizational, interpersonal, communication skills
• The ability to handle multiple tasks, simultaneously
• The ability to work independently or as part of a team
• Knowledge in Azure technologies like API, Azure Functions, Logic Apps development is advantageous