Data Engineer

Location: Bangalore / Pune (Remote)

Employment type:  Full time

Permanent website: www.amazech.com  

 Qualifications:   

               B.E./B.Tech/M.E./M.Tech in Computer Science, Information Technology, Electrical or Electronic with good academic background.   

Experience and Required Skill Sets:  

·        Minimum 5 years of hands on experience with Azure Data Lake, Azure Data Factory, SQL Data Warehouse,Azure Blob, Azure Storage Explorer 

·        Experience in Data warehouse/analytical systems using Azure Synapse. 

·        Proficient in creating Azure Data Factory pipelines for ETL processing; copy activity, custom Azure development etc. 

·        Knowledge of Azure Data Catalog, Event Grid, Service Bus, SQL, Purview and Synapse 

·        Good technical knowledge in Microsoft SQL Server BI Suite (ETL, Reporting, Analytics, Dashboards) using SSIS, SSAS, SSRS, Power BI 

·        Design and develop batch and real-time streaming of data loads to data warehouse systems 

  Other Requirements: 

·        A Bachelor's or Master's degree (Engineering or computer related degree preferred) 

·        Strong understanding of Software Development Life Cycles including Agile/Scrum  

  Responsibilities:   

·        Ability to create complex, enterprise-transforming applications that meet and exceed client expectations.   

·        Responsible for bottom line. Strong project management abilities. Ability to encourage team to stick to timelines.  

·        Should demonstrate strong client interfacing capabilities like emails and calls with clients in the US/UK

• Leveraging your deep knowledge to provide technical leadership to take projects from zero to completion
• Architecting, building and maintaining scalable data pipelines and accessing patterns related to permissions and security
• Researching, evaluating and utilising new technologies/tools/frameworks centred around high-volume data processing
• Involving in building and deploying large scale data processing pipelines in a production environment
• Working with data scientists and other engineers to develop data pipelines for model development and productization
• Identifying gaps and implementing solutions for data security, quality and automation of processes
• Providing inputs on right tool options and model designs for use cases
• Identifying gaps and implementing solutions for data security, quality and automation of processes
• Designing scalable implementations of the models developed by our Data Scientists

Desired Candidate Profile

• 3+ years strong programming experience in PySpark and Python
• Knowledge in Python, SQL, Spark (Pyspark)
• Exposure to AWS/ Azure cloud tools and services like S3 Athena, Apache Nifi, Apache Airflow
• Analytical and problem-solving skills
• Knowledge in Scrum & code sharing Tech: Git, Jira
• Experience related to processing frameworks such as Spark, Spark Streaming, Hive, Sqoop, Kafka etc
• Deep understanding of measuring and ensuring data quality at scale and the required tooling to monitor and optimise the performance of our data pipelines
• Experience building data pipelines and data-centric applications using distributed storage platforms and shipping data production pipelines sourcing data from a diverse array of sources    

• Product Analytics: This is the first and most obvious role of the Product Analyst. At this capacity, the Product Analyst is responsible for the development and delivery of tangible consumer benefits through the product or service of the business.
• In addition, in this capacity, the Product Analyst is also responsible for measuring and monitoring the product or service’s performance as well as presenting product-related consumer, market, and competitive intelligence.
• Product Strategy: As a member of the Product team, the Product Analyst is responsible for the development and proposal of product strategies.
• Product Management Operations: The Product Analyst also has the obligation to respond in a timely manner to all requests and inquiries for product information or changes. He also performs the initial product analysis in order to assess the need for any requested changes as well as their potential impact.
• At this capacity, the Product Analyst also undertakes financial modeling on the products or services of the business as well as of the target markets in order to bring about an understanding of the relations between the product and the target market. This information is presented to the Marketing Manager and other stakeholders, when necessary.
• Additionally, the Product Analyst produces reports and makes recommendations to the Product Manager and Product Marketing Manager to be used as guidance in decision-making pertaining to the business’s new as well as existent products.
• Initiative: In this capacity, the Product Analyst ensures that there is a good flow of communication between the Product team and other teams. The Product Analyst ensures this by actively participating in team meetings and keeping everyone up to date.
• Pricing and Development: The Product Analyst has the responsibility to monitor the market, competitor activities, as well as any price movements and make recommendations that will be used in key decision making. In this function, the Product Analyst will normally liaise with other departments such as the credit/risk in the business in order to enhance and increase the efficiency of effecting price changes in accordance with market shifts.
• Customer/Market Intelligence: The Product Analyst has the obligation to drive consumer intelligence through the development of external and internal data sources that improve the business’s understanding of the product’s market, competitor activities, and consumer activities.
• In the performance of this role, the Product Analyst develops or adopts research tools, sources, and methods that further support and contribute to the business’s product.

We are looking for an ML engineer (Neuroscience) and would like them to

• Build ML algorithms on MRI/ Medical data sets
• Design and build intelligent agents using continual learning and deep reinforcement learning techniques
• Study and transform data science prototypes
• Design machine learning systems
• Research and implement appropriate ML algorithms and tools
• Develop machine learning applications according to requirements
• Select appropriate datasets and data representation methods
• Run machine learning tests and experiments
• Perform statistical analysis and fine-tune using test results
• Train and retrain systems when necessary
• Extend existing ML libraries and frameworks
• Keep abreast of developments in the field
• Collaborate on technical proposals to grow and define artificial intelligence research for intelligent systems

Must have:

• Knowledge of MRI image processing and inferencing
• Hands-on machine learning expertise with an intensive knowledge of hyperparameter optimization, statistical assumption, and implications
• Experience in deep learning models
• Excellent Python programming skills for ML coding
• Understanding of ML integration with our software
• Good understanding of neuroscience/ clinical data

Bonus if you:

• Are enthusiastic about all things brain science and/or mental well-being
• We are a company that highly values the ability to communicate well. We all take turns at the blog roster, so writing experience and/or enthusiasm is appreciated
• Our core values encourage empathy and innovation, you are gold if you share these

• Excellent working knowledge on Data Warehousing /Data Migration activity using an ETL tool.
• Strong Data Integration, PostgreSQL/Oracle Database skills, Shell Scripting, Python programming, and development know-how.
• Hands-on experience in working with and generating XML documents.
• Good analytical and business process understanding capability.
• Familiarized with Data Models, Source-Target Data Mapping, Transactional, and Master Data concepts.
• Well-experienced in High level/Detailed design, Performance tuning of ETL jobs.
• Very good communication skills, interpersonal skills, stakeholder management skills, self-motivated, quick learner, team player.
• Exposure to After Sales Business Domain is highly preferred.
• Experience using HP ALM, Jira for ticketing.
• Experience release management

• Total Experience of 7-10 years and should be interested in teaching and research
• 3+ years’ experience in data engineering which includes data ingestion, preparation, provisioning, automated testing, and quality checks.
• 3+ Hands-on experience in Big Data cloud platforms like AWS and GCP, Data Lakes and Data Warehouses
• 3+ years of Big Data and Analytics Technologies. Experience in SQL, writing code in spark engine using python, scala or java Language. Experience in Spark, Scala
• Experience in designing, building, and maintaining ETL systems
• Experience in data pipeline and workflow management tools like Airflow
• Application Development background along with knowledge of Analytics libraries, opensource Natural Language Processing, statistical and big data computing libraries
• Familiarity with Visualization and Reporting Tools like Tableau, Kibana.
• Should be good at storytelling in Technology