Data Engineer - AWS

- Modeling complex problems, discovering insights, and identifying opportunities through the use of statistical, algorithmic, mining, and visualization techniques

- Experience working with business understanding the requirement, creating the problem statement, and building scalable and dependable Analytical solutions

- Must have hands-on and strong experience in Python

- Broad knowledge of fundamentals and state-of-the-art in NLP and machine learning

- Strong analytical & algorithm development skills

- Deep knowledge of techniques such as Linear Regression, gradient descent, Logistic Regression, Forecasting, Cluster analysis, Decision trees, Linear Optimization, Text Mining, etc

- Ability to collaborate across teams and strong interpersonal skills

Skills

- Sound theoretical knowledge in ML algorithm and their application

- Hands-on experience in statistical modeling tools such as R, Python, and SQL

- Hands-on experience in Machine learning/data science

- Strong knowledge of statistics

- Experience in advanced analytics / Statistical techniques – Regression, Decision trees, Ensemble machine learning algorithms, etc

- Experience in Natural Language Processing & Deep Learning techniques 

- Pandas, NLTK, Scikit-learn, SpaCy, Tensorflow

Job Description-

Responsibilities:

* Work on real-world computer vision problems

* Write robust industry-grade algorithms

* Leverage OpenCV, Python and deep learning frameworks to train models.

* Use Deep Learning technologies such as Keras, Tensorflow, PyTorch etc.

* Develop integrations with various in-house or external microservices.

* Must have experience in deployment practices (Kubernetes, Docker, containerization, etc.) and model compression practices

* Research latest technologies and develop proof of concepts (POCs).

* Build and train state-of-the-art deep learning models to solve Computer Vision related problems, including, but not limited to:

* Segmentation

* Object Detection

* Classification

* Objects Tracking

* Visual Style Transfer

* Generative Adversarial Networks

* Work alongside other researchers and engineers to develop and deploy solutions for challenging real-world problems in the area of Computer Vision

* Develop and plan Computer Vision research projects, in the terms of scope of work including formal definition of research objectives and outcomes

* Provide specialized technical / scientific research to support the organization on different projects for existing and new technologies

Skills:

* Object Detection

* Computer Science

* Image Processing

* Computer Vision

* Deep Learning

* Artificial Intelligence (AI)

* Pattern Recognition

* Machine Learning

* Data Science

* Generative Adversarial Networks (GANs)

* Flask

* SQL

Responsibilities:

• Data science model review, run the code refactoring and optimization, containerization, deployment, versioning, and monitoring of its quality.
• Design and implement cloud solutions, build MLOps on the cloud (preferably AWS)
• Work with workflow orchestration tools like Kubeflow, Airflow, Argo, or similar tools
• Data science models testing, validation, and test automation.
• Communicate with a team of data scientists, data engineers, and architects, and document the processes.

Eligibility:

• Rich hands-on experience in writing object-oriented code using python
• Min 3 years of MLOps experience (Including model versioning, model and data lineage, monitoring, model hosting and deployment, scalability, orchestration, continuous learning, and Automated pipelines)
• Understanding of Data Structures, Data Systems, and software architecture
• Experience in using MLOps frameworks like Kubeflow, MLFlow, and Airflow Pipelines for building, deploying, and managing multi-step ML workflows based on Docker containers and Kubernetes.
• Exposure to deep learning approaches and modeling frameworks (PyTorch, Tensorflow, Keras, etc. )

We are a rapidly expanding global technology partner, that is looking for a highly skilled Senior (Python) Data Engineer to join their exceptional Technology and Development team. The role is in Kolkata. If you are passionate about demonstrating your expertise and thrive on collaborating with a group of talented engineers, then this role was made for you!

At the heart of technology innovation, our client specializes in delivering cutting-edge solutions to clients across a wide array of sectors. With a strategic focus on finance, banking, and corporate verticals, they have earned a stellar reputation for their commitment to excellence in every project they undertake.

We are searching for a senior engineer to strengthen their global projects team. They seek an experienced Senior Data Engineer with a strong background in building Extract, Transform, Load (ETL) processes and a deep understanding of AWS serverless cloud environments.

As a vital member of the data engineering team, you will play a critical role in designing, developing, and maintaining data pipelines that facilitate data ingestion, transformation, and storage for our organization.

Your expertise will contribute to the foundation of our data infrastructure, enabling data-driven decision-making and analytics.

Key Responsibilities:

• ETL Pipeline Development: Design, develop, and maintain ETL processes using Python, AWS Glue, or other serverless technologies to ingest data from various sources (databases, APIs, files), transform it into a usable format, and load it into data warehouses or data lakes.
• AWS Serverless Expertise: Leverage AWS services such as AWS Lambda, AWS Step Functions, AWS Glue, AWS S3, and AWS Redshift to build serverless data pipelines that are scalable, reliable, and cost-effective.
• Data Modeling: Collaborate with data scientists and analysts to understand data requirements and design appropriate data models, ensuring data is structured optimally for analytical purposes.
• Data Quality Assurance: Implement data validation and quality checks within ETL pipelines to ensure data accuracy, completeness, and consistency.
• Performance Optimization: Continuously optimize ETL processes for efficiency, performance, and scalability, monitoring and troubleshooting any bottlenecks or issues that may arise.
• Documentation: Maintain comprehensive documentation of ETL processes, data lineage, and system architecture to ensure knowledge sharing and compliance with best practices.
• Security and Compliance: Implement data security measures, encryption, and compliance standards (e.g., GDPR, HIPAA) as required for sensitive data handling.
• Monitoring and Logging: Set up monitoring, alerting, and logging systems to proactively identify and resolve data pipeline issues.
• Collaboration: Work closely with cross-functional teams, including data scientists, data analysts, software engineers, and business stakeholders, to understand data requirements and deliver solutions.
• Continuous Learning: Stay current with industry trends, emerging technologies, and best practices in data engineering and cloud computing and apply them to enhance existing processes.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
• Proven experience as a Data Engineer with a focus on ETL pipeline development.
• Strong proficiency in Python programming.
• In-depth knowledge of AWS serverless technologies and services.
• Familiarity with data warehousing concepts and tools (e.g., Redshift, Snowflake).
• Experience with version control systems (e.g., Git).
• Strong SQL skills for data extraction and transformation.
• Excellent problem-solving and troubleshooting abilities.
• Ability to work independently and collaboratively in a team environment.
• Effective communication skills for articulating technical concepts to non-technical stakeholders.
• Certifications such as AWS Certified Data Analytics - Specialty or AWS Certified DevOps Engineer are a plus.

Preferred Experience:

• Knowledge of data orchestration and workflow management tools
• Familiarity with data visualization tools (e.g., Tableau, Power BI).
• Previous experience in industries with strict data compliance requirements (e.g., insurance, finance) is beneficial.

What You Can Expect:

- Innovation Abounds: Join a company that constantly pushes the boundaries of technology and encourages creative thinking. Your ideas and expertise will be valued and put to work in pioneering solutions.

- Collaborative Excellence: Be part of a team of engineers who are as passionate and skilled as you are. Together, you'll tackle challenging projects, learn from each other, and achieve remarkable results.

- Global Impact: Contribute to projects with a global reach and make a tangible difference. Your work will shape the future of technology in finance, banking, and corporate sectors.

They offer an exciting and professional environment with great career and growth opportunities. Their office is located in the heart of Salt Lake Sector V, offering a terrific workspace that's both accessible and inspiring. Their team members enjoy a professional work environment with regular team outings. Joining the team means becoming part of a vibrant and dynamic team where your skills will be valued, your creativity will be nurtured, and your contributions will make a difference. In this role, you can work alongside some of the brightest minds in the industry.

If you're ready to take your career to the next level and be part of a dynamic team that's driving innovation on a global scale, we want to hear from you.

Apply today for more information about this exciting opportunity.

Onsite Location: Kolkata, India (Salt Lake Sector V)

Skills and requirements

• Experience analyzing complex and varied data in a commercial or academic setting.
• Desire to solve new and complex problems every day.
• Excellent ability to communicate scientific results to both technical and non-technical team members.

Desirable

• A degree in a numerically focused discipline such as, Maths, Physics, Chemistry, Engineering or Biological Sciences..
• Hands on experience on Python, Pyspark, SQL
• Hands on experience on building End to End Data Pipelines.
• Hands on Experience on Azure Data Factory, Azure Data Bricks, Data Lake - added advantage
• Hands on Experience in building data pipelines.
• Experience with Bigdata Tools, Hadoop, Hive, Sqoop, Spark, SparkSQL
• Experience with SQL or NoSQL databases for the purposes of data retrieval and management.
• Experience in data warehousing and business intelligence tools, techniques and technology, as well as experience in diving deep on data analysis or technical issues to come up with effective solutions.
• BS degree in math, statistics, computer science or equivalent technical field.
• Experience in data mining structured and unstructured data (SQL, ETL, data warehouse, Machine Learning etc.) in a business environment with large-scale, complex data sets.
• Proven ability to look at solutions in unconventional ways. Sees opportunities to innovate and can lead the way.
• Willing to learn and work on Data Science, ML, AI.

• Cloud: GCP
• Must have: BigQuery, Python, Vertex AI
• Nice to have Services: Data Plex
• Exp level: 5-10 years.
• Preferred Industry (nice to have): Manufacturing – B2B sales