Job Description:
As Azure Lead Data Engineer, you will have to take over the key activities of designing, coding, and implementing data solutions in the platform. With a focus on leveraging DBT for data transformation and modelling, as well as expertise in MDM tools, you will play a pivotal role in architecting scalable and performant data pipelines and warehouses. You will collaborate closely with cross-functional teams to understand business requirements, architect data solutions, and ensure successful project delivery. You will be leading a team of skilled engineers who will collectively deliver scalable, highly dependable data solutions that can cater to the customers.
Responsibilities:
- Lead the design, development, and implementation of data solutions on the Microsoft Azure platform.
- Architect data pipelines, data warehouses, and data lakes using Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Blob Storage.
- Design and implement ETL processes to extract, transform, and load data from various sources into Azure data platforms, utilizing DBT for data transformation.
- Develop scalable and efficient data models to support analytics, reporting, and machine learning initiatives, with a strong emphasis on using DBT for modelling.
- Lead performance optimization efforts to ensure the efficient processing of large volumes of data.
- Mentor and coach junior team members, providing guidance on best practices, technical expertise, and professional development.
- Collaborate with stakeholders to understand business requirements and translate them into technical solutions.
- Stay abreast of emerging technologies and industry trends in data engineering and cloud computing.
Qualifications:
- BE Computer Science or a related field.
- ~10 years of experience in data engineering, designing, and implementing data solutions on the Microsoft Azure platform.
- Deep understanding of Azure services such as Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Blob Storage.
- Proficiency in DBT (Data Build Tool) for data transformation and modelling.
- Experience working with any Master Data Management (MDM) tools.
- Experience with data governance and metadata management tools such as Azure Purview or similar.
- Proficiency in programming languages such as Python, Scala, or Java.
- Experience with big data technologies such as Hadoop, Spark, and Kafka.
- Strong leadership skills with the ability to lead and mentor a team of engineers.
About Intraedge Technologies
Similar jobs
Responsibilities
-
Deliver full-cycle Tableau development projects, from business needs assessment and data discovery, through solution design, to delivery to client.
-
Enable our clients and ourselves to answer questions and develop data-driven insights through Tableau.
-
Provide technical leadership and support across all aspects of Tableau development and use, from data specification development, through DataMart development, to supporting end-user dashboards and reports.
-
Administrate Tableau Server by creating sites, add/remove users, and provide the appropriate level access for users.
-
Strategize and ideate the solution design. Develop UI mock-ups, storyboards, flow diagrams, conceptual diagrams, wireframes, visual mockups, and interactive prototypes.
-
Develop best practices guidelines for Tableau data processing and visualization. Use these best practices to quickly deliver functionality across the client base and internal users.
Qualifications
-
Degree in a highly-relevant analytical or technical field, such as statistics, data science, or business analytics.
· 5+ years as a Tableau developer and administrator.
· Extensive experience with large data sets, statistical analyses, and visualization as well as hands-on experience on tools (SQL, Tableau, Power BI).
· Ability to quickly learn and take responsibility to deliver.
- Provide insights based on data to business teams
- Develop framework, solutions and recommendations for business problems
- Build ML models for predictive solutions
- Use advance data science techniques to build business solutions
- Automation / Optimization of new/existing models ensuring smooth,timely and accurate execution with lowest possible TAT.
- Design & maintenance of response tracking, measurement, and comparison of success parameters of various projects.
- Ability to handle large volumes of data with ease using multiple software like Python ,R etc
Experience in modeling techniques and hands on experience in building Logistic regression models, Random Forrest, K-mean Cluster, NLP, Decision tree, Boosting techniques etc
- Good at data interpretation and reasoning skills
Responsibilities:
- Data science model review, run the code refactoring and optimization, containerization, deployment, versioning, and monitoring of its quality.
- Design and implement cloud solutions, build MLOps on the cloud (preferably AWS)
- Work with workflow orchestration tools like Kubeflow, Airflow, Argo, or similar tools
- Data science models testing, validation, and test automation.
- Communicate with a team of data scientists, data engineers, and architects, and document the processes.
Eligibility:
- Rich hands-on experience in writing object-oriented code using python
- Min 3 years of MLOps experience (Including model versioning, model and data lineage, monitoring, model hosting and deployment, scalability, orchestration, continuous learning, and Automated pipelines)
- Understanding of Data Structures, Data Systems, and software architecture
- Experience in using MLOps frameworks like Kubeflow, MLFlow, and Airflow Pipelines for building, deploying, and managing multi-step ML workflows based on Docker containers and Kubernetes.
- Exposure to deep learning approaches and modeling frameworks (PyTorch, Tensorflow, Keras, etc. )
- A Natural Language Processing (NLP) expert with strong computer science fundamentals and experience in working with deep learning frameworks. You will be working at the cutting edge of NLP and Machine Learning.
Roles and Responsibilities
- Work as part of a distributed team to research, build and deploy Machine Learning models for NLP.
- Mentor and coach other team members
- Evaluate the performance of NLP models and ideate on how they can be improved
- Support internal and external NLP-facing APIs
- Keep up to date on current research around NLP, Machine Learning and Deep Learning
Mandatory Requirements
- Any graduation with at least 2 years of demonstrated experience as a Data Scientist.
Behavioral Skills
Strong analytical and problem-solving capabilities.
- Proven ability to multi-task and deliver results within tight time frames
- Must have strong verbal and written communication skills
- Strong listening skills and eagerness to learn
- Strong attention to detail and the ability to work efficiently in a team as well as individually
Technical Skills
Hands-on experience with
- NLP
- Deep Learning
- Machine Learning
- Python
- Bert
Preferred Requirements
- Experience in Computer Vision is preferred
Is your next career move to work in a team which uses data, reporting and analytical skills to help answer business questions to make DAZN a data-driven company?
DAZN is a tech-first sport streaming platform that reaches millions of users every week. We are challenging a traditional industry and giving power back to the fans. Our new Hyderabad tech hub will be the engine that drives us forward to the future. We’re pushing boundaries and doing things no-one has done before. Here, you have the opportunity to make your mark and the power to make change happen - to make a difference for our customers. When you join DAZN you will work on projects that impact millions of lives thanks to your critical contributions to our global products
This is the perfect place to work if you are passionate about technology and want an opportunity to use your creativity to help grow and scale a global range of IT systems, Infrastructure, and IT Services. Our cutting-edge technology allows us to stream sports content to millions of concurrent viewers globally across multiple platforms and devices. DAZN’s Cloud based architecture unifies a range of technologies in order to deliver a seamless user experience and support a global user base and company infrastructure.
This role will be based in our brand-new Hyderabad office. Join us in India’s beautiful “City of Pearls” and bring your ambition to life.
Responsibilities:
- Communicate with different stakeholders such as Ad Tech Engineers and Product Owners
- Should be able to extensively work in Google Analytics and strong SQL knowledge is expected.
- Strong analytical skills
Key Competencies:
- 4-8 years of experience as Data Analyst
- Advanced Microsoft Excel Skills
- Strong command on Google Analytics
- Reporting platform UI experience (Tableau, Looker, etc)
- Experience with VAST tags, pixels trackers, etc.
- Experience with DSPs & third-party ad platforms (GAM, YoSpace, etc)
At DAZN, we bring ambition to life. We are innovators, game-changers and pioneers. So, if you want to push boundaries and make an impact, DAZN is the place to be.
As part of our team, you'll have the opportunity to make your mark and the power to make change happen. We're doing things no-one has done before, giving fans and customers access to sport anytime, anywhere. We're using world-class technology to transform sports and revolutionise the industry and we're not going to stop.
Responsibilities
- Understanding the business requirements so as to formulate the problems to solve and restrict the slice of data to be explored.
- Collecting data from various sources.
- Performing cleansing, processing, and validation on the data subject to analyze, in order to ensure its quality.
- Exploring and visualizing data.
- Performing statistical analysis and experiments to derive business insights.
- Clearly communicating the findings from the analysis to turn information into something actionable through reports, dashboards, and/or presentations.
Skills
- Experience solving problems in the project’s business domain.
- Experience with data integration from multiple sources
- Proficiency in at least one query language, especially SQL.
- Working experience with NoSQL databases, such as MongoDB and Elasticsearch.
- Working experience with popular statistical and machine learning techniques, such as clustering, linear regression, KNN, decision trees, etc.
- Good scripting skills using Python, R or any other relevant language
- Proficiency in at least one data visualization tool, such as Matplotlib, Plotly, D3.js, ggplot, etc.
- Great communication skills.
Experience with Agile development and software such as Azure DevOps or JIRA. Product
Owner certification is a plus
Experience with global teams
Bachelors required. CS degree preferred
1. Expert in deep learning and machine learning techniques,
2. Extremely Good in image/video processing,
3. Have a Good understanding of Linear algebra, Optimization techniques, Statistics and pattern recognition.
Then u r the right fit for this position.
Only a solid grounding in computer engineering, Unix, data structures and algorithms would enable you to meet this challenge. 7+ years of experience architecting, developing, releasing, and maintaining large-scale big data platforms on AWS or GCP Understanding of how Big Data tech and NoSQL stores like MongoDB, HBase/HDFS, ElasticSearch synergize to power applications in analytics, AI and knowledge graphs Understandingof how data processing models, data location patterns, disk IO, network IO, shuffling affect large scale text processing - feature extraction, searching etc Expertise with a variety of data processing systems, including streaming, event, and batch (Spark, Hadoop/MapReduce) 5+ years proficiency in configuring and deploying applications on Linux-based systems 5+ years of experience Spark - especially Pyspark for transforming large non-structured text data, creating highly optimized pipelines Experience with RDBMS, ETL techniques and frameworks (Sqoop, Flume) and big data querying tools (Pig, Hive) Stickler of world class best practices, uncompromising on the quality of engineering, understand standards and reference architectures and deep in Unix philosophy with appreciation of big data design patterns, orthogonal code design and functional computation models |
(Hadoop, HDFS, Kafka, Spark, Hive)
Overall Experience - 8 to 12 years
Relevant exp on Big data - 3+ years in above
Salary: Max up-to 20LPA
Job location - Chennai / Bangalore /
Notice Period - Immediate joiner / 15-to-20-day Max
The Responsibilities of The Senior Data Engineer Are:
- Requirements gathering and assessment
- Breakdown complexity and translate requirements to specification artifacts and story boards to build towards, using a test-driven approach
- Engineer scalable data pipelines using big data technologies including but not limited to Hadoop, HDFS, Kafka, HBase, Elastic
- Implement the pipelines using execution frameworks including but not limited to MapReduce, Spark, Hive, using Java/Scala/Python for application design.
- Mentoring juniors in a dynamic team setting
- Manage stakeholders with proactive communication upholding TheDataTeam's brand and values
A Candidate Must Have the Following Skills:
- Strong problem-solving ability
- Excellent software design and implementation ability
- Exposure and commitment to agile methodologies
- Detail oriented with willingness to proactively own software tasks as well as management tasks, and see them to completion with minimal guidance
- Minimum 8 years of experience
- Should have experience in full life-cycle of one big data application
- Strong understanding of various storage formats (ORC/Parquet/Avro)
- Should have hands on experience in one of the Hadoop distributions (Hortoworks/Cloudera/MapR)
- Experience in at least one cloud environment (GCP/AWS/Azure)
- Should be well versed with at least one database (MySQL/Oracle/MongoDB/Postgres)
- Bachelor's in Computer Science, and preferably, a Masters as well - Should have good code review and debugging skills
Additional skills (Good to have):
- Experience in Containerization (docker/Heroku)
- Exposure to microservices
- Exposure to DevOps practices - Experience in Performance tuning of big data applications