Data Engineer

1. Strong communication skills are essential, as the selected candidate will be responsible for leading a team of two in the future.
2. Proficiency in SQL.
3. Expertise in Data Modelling.
4. Experience with Azure Data Factory (ADF).
5. Competence in Power BI.
6. SQL – Should be strong in Data Modeling , Tables Design and SQL Queries.
7. ADF – Must have hands-on experience in ADF pipelines and its set-up from End-to-End in Azure including subscriptions, IR and Resource Group creations.
8. Power BI – Hands-on knowledge in Power BI reports including documentation and follow existing standards.

We are #hiring for AWS Data Engineer expert to join our team

Job Title: AWS Data Engineer

Experience: 5 Yrs to 10Yrs

Location: Remote

Notice: Immediate or Max 20 Days

Role: Permanent Role

Skillset: AWS, ETL, SQL, Python, Pyspark, Postgres DB, Dremio.

Job Description:

 Able to develop ETL jobs.

Able to help with data curation/cleanup, data transformation, and building ETL pipelines.

Strong Postgres DB exp and knowledge of Dremio data visualization/semantic layer between DB and the application is a plus.

Sql, Python, and Pyspark is a must.

Communication should be good

• Partnering with internal business owners (product, marketing, edit, etc.) to understand needs and develop custom analysis to optimize for user engagement and retention
• Good understanding of the underlying business and workings of cross functional teams for successful execution
• Design and develop analyses based on business requirement needs and challenges.
• Leveraging statistical analysis on consumer research and data mining projects, including segmentation, clustering, factor analysis, multivariate regression, predictive modeling, etc.
• Providing statistical analysis on custom research projects and consult on A/B testing and other statistical analysis as needed. Other reports and custom analysis as required.
• Identify and use appropriate investigative and analytical technologies to interpret and verify results.
• Apply and learn a wide variety of tools and languages to achieve results
• Use best practices to develop statistical and/ or machine learning techniques to build models that address business needs.

• 2 - 4 years  of relevant experience in Data science.
• Preferred education: Bachelor's degree in a technical field or equivalent experience.
• Experience in advanced analytics, model building, statistical modeling, optimization, and machine learning algorithms.
• Machine Learning Algorithms: Crystal clear understanding, coding, implementation, error analysis, model tuning knowledge on Linear Regression, Logistic Regression, SVM, shallow Neural Networks, clustering, Decision Trees, Random forest, XGBoost, Recommender Systems, ARIMA and Anomaly Detection. Feature selection, hyper parameters tuning, model selection and error analysis, boosting and ensemble methods.
• Strong with programming languages like Python and data processing using SQL or equivalent and ability to experiment with newer open source tools.
• Experience in normalizing data to ensure it is homogeneous and consistently formatted to enable sorting, query and analysis.
• Experience designing, developing, implementing and maintaining a database and programs to manage data analysis efforts.
• Experience with big data and cloud computing viz. Spark, Hadoop (MapReduce, PIG, HIVE).
• Experience in risk and credit score domains preferred.

Location:  Pune/Nagpur,Goa,Hyderabad/Bangalore

Job Requirements:

• 9 years and above of total experience preferably in bigdata space.
• Creating spark applications using Scala to process data.
• Experience in scheduling and troubleshooting/debugging Spark jobs in steps.
• Experience in spark job performance tuning and optimizations.
• Should have experience in processing data using Kafka/Pyhton.
• Individual should have experience and understanding in configuring Kafka topics to optimize the performance.
• Should be proficient in writing SQL queries to process data in Data Warehouse.
• Hands on experience in working with Linux commands to troubleshoot/debug issues and creating shell scripts to automate tasks.
• Experience on AWS services like EMR.

4 - 8 overall experience.

• 1-2 years’ experience in Azure Data Factory - schedule Jobs in Flows and ADF Pipelines, Performance Tuning, Error logging etc..
• 1+ years of experience with Power BI - designing and developing reports, dashboards, metrics and visualizations in Powe BI.
• (Required) Participate in video conferencing calls - daily stand-up meetings and all day working with team members on cloud migration planning, development, and support.
• Proficiency in relational database concepts & design using star, Azure Datawarehouse, and data vault.
• Requires 2-3 years of experience with SQL scripting (merge, joins, and stored procedures) and best practices.
• Knowledge on deploying and run SSIS packages in Azure.
• Knowledge of Azure Data Bricks.
• Ability to write and execute complex SQL queries and stored procedures.

Key deliverables for the Data Science Engineer would be to help us discover the information hidden in vast amounts of data, and help us make smarter decisions to deliver even better products. Your primary focus will be on applying data mining techniques, doing statistical analysis, and building high-quality prediction systems integrated with our products.

What will you do?

• You will be building and deploying ML models to solve specific business problems related to NLP, computer vision, and fraud detection.
• You will be constantly assessing and improving the model using techniques like Transfer learning
• You will identify valuable data sources and automate collection processes along with undertaking pre-processing of structured and unstructured data
• You will own the complete ML pipeline - data gathering/labeling, cleaning, storage, modeling, training/testing, and deployment.
• Assessing the effectiveness and accuracy of new data sources and data gathering techniques.
• Building predictive models and machine-learning algorithms to apply to data sets.
• Coordinate with different functional teams to implement models and monitor outcomes.
•  
• Presenting information using data visualization techniques and proposing solutions and strategies to business challenges

We would love to hear from you if :

• You have 2+ years of experience as a software engineer at a SaaS or technology company
• Demonstrable hands-on programming experience with Python/R Data Science Stack
• Ability to design and implement workflows of Linear and Logistic Regression, Ensemble Models (Random Forest, Boosting) using R/Python
• Familiarity with Big Data Platforms (Databricks, Hadoop, Hive), AWS Services (AWS, Sagemaker, IAM, S3, Lambda Functions, Redshift, Elasticsearch)
• Experience in Probability and Statistics, ability to use ideas of Data Distributions, Hypothesis Testing and other Statistical Tests.
• Demonstrable competency in Data Visualisation using the Python/R Data Science Stack.
• Preferable Experience Experienced in web crawling and data scraping
• Strong experience in NLP. Worked on libraries such as NLTK, Spacy, Pattern, Gensim etc.
• Experience with text mining, pattern matching and fuzzy matching

• Brand new Macbook
• Stock Options
• Health Insurance
• Unlimited Sick Leaves
• Passion Fund (Invest in yourself or your passion project)
• Wind Down

1. Data Steward :

Data Steward will collaborate and work closely within the group software engineering and business division. Data Steward has overall accountability for the group's / Divisions overall data and reporting posture by responsibly managing data assets, data lineage, and data access, supporting sound data analysis. This role requires focus on data strategy, execution, and support for projects, programs, application enhancements, and production data fixes. Makes well-thought-out decisions on complex or ambiguous data issues and establishes the data stewardship and information management strategy and direction for the group. Effectively communicates to individuals at various levels of the technical and business communities. This individual will become part of the corporate Data Quality and Data management/entity resolution team supporting various systems across the board.

Primary Responsibilities:

• Responsible for data quality and data accuracy across all group/division delivery initiatives.
• Responsible for data analysis, data profiling, data modeling, and data mapping capabilities.
• Responsible for reviewing and governing data queries and DML.
• Accountable for the assessment, delivery, quality, accuracy, and tracking of any production data fixes.
• Accountable for the performance, quality, and alignment to requirements for all data query design and development.
• Responsible for defining standards and best practices for data analysis, modeling, and queries.
• Responsible for understanding end-to-end data flows and identifying data dependencies in support of delivery, release, and change management.
• Responsible for the development and maintenance of an enterprise data dictionary that is aligned to data assets and the business glossary for the group responsible for the definition and maintenance of the group's data landscape including overlays with the technology landscape, end-to-end data flow/transformations, and data lineage.
• Responsible for rationalizing the group's reporting posture through the definition and maintenance of a reporting strategy and roadmap.
• Partners with the data governance team to ensure data solutions adhere to the organization’s data principles and guidelines.
• Owns group's data assets including reports, data warehouse, etc.
• Understand customer business use cases and be able to translate them to technical specifications and vision on how to implement a solution.
• Accountable for defining the performance tuning needs for all group data assets and managing the implementation of those requirements within the context of group initiatives as well as steady-state production.
• Partners with others in test data management and masking strategies and the creation of a reusable test data repository.
• Responsible for solving data-related issues and communicating resolutions with other solution domains.
• Actively and consistently support all efforts to simplify and enhance the Clinical Trial Predication use cases.
• Apply knowledge in analytic and statistical algorithms to help customers explore methods to improve their business.
• Contribute toward analytical research projects through all stages including concept formulation, determination of appropriate statistical methodology, data manipulation, research evaluation, and final research report.
• Visualize and report data findings creatively in a variety of visual formats that appropriately provide insight to the stakeholders.
• Achieve defined project goals within customer deadlines; proactively communicate status and escalate issues as needed.

Additional Responsibilities:

• Strong understanding of the Software Development Life Cycle (SDLC) with Agile Methodologies
• Knowledge and understanding of industry-standard/best practices requirements gathering methodologies.
• Knowledge and understanding of Information Technology systems and software development.
• Experience with data modeling and test data management tools.
• Experience in the data integration project • Good problem solving & decision-making skills.
• Good communication skills within the team, site, and with the customer

Knowledge, Skills and Abilities

• Technical expertise in data architecture principles and design aspects of various DBMS and reporting concepts.
• Solid understanding of key DBMS platforms like SQL Server, Azure SQL
• Results-oriented, diligent, and works with a sense of urgency. Assertive, responsible for his/her own work (self-directed), have a strong affinity for defining work in deliverables, and be willing to commit to deadlines.
• Experience in MDM tools like MS DQ, SAS DM Studio, Tamr, Profisee, Reltio etc.
• Experience in Report and Dashboard development
• Statistical and Machine Learning models
• Python (sklearn, numpy, pandas, genism)
• Nice to Have:
• 1yr of ETL experience
• Natural Language Processing
• Neural networks and Deep learning
• xperience in keras,tensorflow,spacy, nltk, LightGBM python library

Interaction :  Frequently interacts with subordinate supervisors.

Education : Bachelor’s degree, preferably in Computer Science, B.E or other quantitative field related to the area of assignment. Professional certification related to the area of assignment may be required

Experience :  7 years of Pharmaceutical /Biotech/life sciences experience, 5 years of Clinical Trials experience and knowledge, Excellent Documentation, Communication, and Presentation Skills including PowerPoint