Data Analyst - SQL

Title:  Data Engineer – Snowflake

Location: Mysore (Hybrid model)

Exp-2-8 yrs

Type: Full Time

Walk-in date: 25th Jan 2023 @Mysore 

Job Role: We are looking for an experienced Snowflake developer to join our team as a Data Engineer who will work as part of a team to help design and develop data-driven solutions that deliver insights to the business. The ideal candidate is a data pipeline builder and data wrangler who enjoys building data-driven systems that drive analytical solutions and building them from the ground up. You will be responsible for building and optimizing our data as well as building automated processes for production jobs. You will support our software developers, database architects, data analysts and data scientists on data initiatives

Key Roles & Responsibilities:

• Use advanced complex Snowflake/Python and SQL to extract data from source systems for ingestion into a data pipeline.
• Design, develop and deploy scalable and efficient data pipelines.
• Analyze and assemble large, complex datasets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements. For example: automating manual processes, optimizing data delivery, re-designing data platform infrastructure for greater scalability.
• Build required infrastructure for optimal extraction, loading, and transformation (ELT) of data from various data sources using AWS and Snowflake leveraging Python or SQL technologies.
• Monitor cloud-based systems and components for availability, performance, reliability, security and efficiency
• Create and configure appropriate cloud resources to meet the needs of the end users.
• As needed, document topology, processes, and solution architecture.
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies

Qualifications & Experience

Qualification & Experience Requirements:

• Bachelor's degree in computer science, computer engineering, or a related field.
• 2-8 years of experience working with Snowflake
• 2+ years of experience with the AWS services.
• Candidate should able to write the stored procedure and function in Snowflake.
• At least 2 years’ experience in snowflake developer.
• Strong SQL Knowledge.
• Data injection in snowflake using Snowflake procedure.
• ETL Experience is Must (Could be any tool)
• Candidate should be aware of snowflake architecture.
• Worked on the Migration project
• DW Concept (Optional)
• Experience with cloud data storage and compute components including lambda functions, EC2s, containers.
• Experience with data pipeline and workflow management tools: Airflow, etc.
• Experience cleaning, testing, and evaluating data quality from a wide variety of ingestible data sources
• Experience working with Linux and UNIX environments.
• Experience with profiling data, with and without data definition documentation

• Familiar with Git
• Familiar with issue tracking systems like JIRA (Project Management Tool) or Trello.
• Experience working in an agile environment.

Desired Skills:

• Experience in Snowflake. Must be willing to be Snowflake certified in the first 3 months of employment.
• Experience with a stream-processing system: Snowpipe
• Working knowledge of AWS or Azure
• Experience in migrating from on-prem to cloud systems

Required Skill Set-

Project experience in any of the following - Data Management,

Database Development, Data Migration or Data Warehousing.

• Expertise in SQL, PL/SQL.

Role and Responsibilities -

• Work on a complex data management program for multi-billion dollar

customer

• Work on customer projects related to data migration, data

integration

•No Troubleshooting

• Execution of data pipelines, perform QA, project documentation for

project deliverables

• Perform data profiling, data cleansing, data analysis for migration

data

• Participate and contribute in project meeting

• Experience in data manipulation using Python preferred

• Proficient in using Excel, PowerPoint

 -Perform other tasks as per project requirements.

• The ideal candidate is adept at using large data sets to find opportunities for product and process optimization and using models to test the effectiveness of different courses of action.
• Mine and analyze data from company databases to drive optimization and improvement of product development, marketing techniques and business strategies.
• Assess the effectiveness and accuracy of new data sources and data gathering techniques.
• Develop custom data models and algorithms to apply to data sets.
• Use predictive modeling to increase and optimize customer experiences, revenue generation, ad targeting and other business outcomes.
• Develop company A/B testing framework and test model quality.
• Develop processes and tools to monitor and analyze model performance and data accuracy.

• Experience using statistical languages (R, Python, SQL, etc.) to manipulate data and draw insights from large data sets.
• Experience working with and creating data architectures.
• Looking for someone with 3-7 years of experience manipulating data sets and building statistical models
• Has a Bachelor's, Master's in Computer Science or another quantitative field
• Knowledge and experience in statistical and data mining techniques :
• GLM/Regression, Random Forest, Boosting, Trees, text mining,social network analysis, etc.
• Experience querying databases and using statistical computer languages :R, Python, SQL, etc.
• Experience creating and using advanced machine learning algorithms and statistics: regression, simulation, scenario analysis, modeling, clustering, decision trees,neural networks, etc.
• Experience with distributed data/computing tools: Map/Reduce, Hadoop, Hive, Spark, Gurobi, MySQL, etc.
• Experience visualizing/presenting data for stakeholders using: Periscope, Business Objects, D3, ggplot, etc.

Responsibilities

• Design and implement Azure BI infrastructure, ensure overall quality of delivered solution 
• Develop analytical & reporting tools, promote and drive adoption of developed BI solutions 
• Actively participate in BI community 
• Establish and enforce technical standards and documentation 
• Participate in daily scrums  
• Record progress daily in assigned Devops items 

Ideal Candidates should have

• 5 + years of experience in a similar senior business intelligence development position 
• To be successful in the role you will require a high level of expertise across all facets of the Microsoft BI stack and prior experience in designing and developing well-performing data warehouse solutions 
• Demonstrated experience using development tools such as Azure SQL database, Azure Data Factory, Azure Data Lake, Azure Synapse, and Azure DevOps. 
• Experience with development methodologies including Agile, DevOps, and CICD patterns 
• Strong oral and written communication skills in English 
• Ability and willingness to learn quickly and continuously 
• Bachelor's Degree in computer science 

Job Description

This requirement is to service our client which is a leading big data technology company that measures what viewers consume across platforms to enable marketers make better advertising decisions. We are seeking a Senior Data Operations Analyst to mine large-scale datasets for our client. Their work will have a direct impact on driving business strategies for prominent industry leaders. Self-motivation and strong communication skills are both must-haves. Ability to work in a fast-paced work environment is desired.

Problems being solved by our client: 

Measure consumer usage of devices linked to the internet and home networks including computers, mobile phones, tablets, streaming sticks, smart TVs, thermostats and other appliances. There are more screens and other connected devices in homes than ever before, yet there have been major gaps in understanding how consumers interact with this technology. Our client uses a measurement technology to unravel dynamics of consumers’ interactions with multiple devices.

Duties and responsibilities:

• The successful candidate will contribute to the development of novel audience measurement and demographic inference solutions. 
• Develop, implement, and support statistical or machine learning methodologies and processes. 

• Build, test new features and concepts and integrate into production process
• Participate in ongoing research and evaluation of new technologies
• Exercise your experience in the development lifecycle through analysis, design, development, testing and deployment of this system
• Collaborate with teams in Software Engineering, Operations, and Product Management to deliver timely and quality data. You will be the knowledge expert, delivering quality data to our clients

Qualifications:

• 3-5 years relevant work experience in areas as outlined below
• Experience in extracting data using SQL from large databases
• Experience in writing complex ETL processes and frameworks for analytics and data management. Must have experience in working on ETL tools.

• Master’s degree or PhD in Statistics, Data Science, Economics, Operations Research, Computer Science, or a similar degree with a focus on statistical methods. A Bachelor’s degree in the same fields with significant, demonstrated professional research experience will also be considered. 
• Programming experience in scientific computing language (R, Python, Julia) and the ability to interact with relational data (SQL, Apache Pig, SparkSQL). General purpose programming (Python, Scala, Java) and familiarity with Hadoop is a plus.  
• Excellent verbal and written communication skills. 
• Experience with TV or digital audience measurement or market research data is a plus. 
• Familiarity with systems analysis or systems thinking is a plus. 

• Must be comfortable with analyzing complex, high-volume and high-dimension data from varying sources
• Excellent verbal, written and computer communication skills
• Ability to engage with Senior Leaders across all functional departments
• Ability to take on new responsibilities and adapt to changes

Title: Data Engineer (Azure) (Location: Gurgaon/Hyderabad)

Salary: Competitive as per Industry Standard

We are expanding our Data Engineering Team and hiring passionate professionals with extensive

knowledge and experience in building and managing large enterprise data and analytics platforms. We

are looking for creative individuals with strong programming skills, who can understand complex

business and architectural problems and develop solutions. The individual will work closely with the rest

of our data engineering and data science team in implementing and managing Scalable Smart Data

Lakes, Data Ingestion Platforms, Machine Learning and NLP based Analytics Platforms, Hyper-Scale

Processing Clusters, Data Mining and Search Engines.

What You’ll Need:

• 3+ years of industry experience in creating and managing end-to-end Data Solutions, Optimal

Data Processing Pipelines and Architecture dealing with large volume, big data sets of varied

data types.

• Proficiency in Python, Linux and shell scripting.
• Strong knowledge of working with PySpark dataframes, Pandas dataframes for writing efficient pre-processing and other data manipulation tasks.
  ● Strong experience in developing the infrastructure required for data ingestion, optimal

extraction, transformation, and loading of data from a wide variety of data sources using tools like Azure Data Factory,  Azure Databricks (or Jupyter notebooks/ Google Colab) (or other similiar tools).

• Working knowledge of github or other version control tools.
• Experience with creating Restful web services and API platforms.
• Work with data science and infrastructure team members to implement practical machine

learning solutions and pipelines in production.

• Experience with cloud providers like Azure/AWS/GCP.
• Experience with SQL and NoSQL databases. MySQL/ Azure Cosmosdb / Hbase/MongoDB/ Elasticsearch etc.
• Experience with stream-processing systems: Spark-Streaming, Kafka etc and working experience with event driven architectures.
• Strong analytic skills related to working with unstructured datasets.

Good to have (to filter or prioritize candidates)

• Experience with testing libraries such as pytest for writing unit-tests for the developed code.
• Knowledge of Machine Learning algorithms and libraries would be good to have,

implementation experience would be an added advantage.

• Knowledge and experience of Datalake, Dockers and Kubernetes would be good to have.
• Knowledge of Azure functions , Elastic search etc will be good to have.

• Having experience with model versioning (mlflow) and data versioning will be beneficial
• Having experience with microservices libraries or with python libraries such as flask for hosting ml services and models would be great.

• Create and maintain optimal data pipeline architecture,
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across national boundaries through multiple data centers and AWS regions.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
• Work with data and analytics experts to strive for greater functionality in our data systems.

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

• Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with AWS cloud services: EC2, EMR, RDS, Redshift
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.