Data Analyst

We are seeking an experienced Senior Data Platform Engineer to join our team. The ideal candidate should have extensive experience with Pyspark, Airflow, Presto, Hive, Kafka and Debezium, and should be passionate about developing scalable and reliable data platforms.

Responsibilities:

• Design, develop, and maintain our data platform architecture using Pyspark, Airflow, Presto, Hive, Kafka, and Debezium.
• Develop and maintain ETL processes to ingest, transform, and load data from various sources into our data platform.
• Work closely with data analysts, data scientists, and other stakeholders to understand their requirements and design solutions that meet their needs.
• Implement and maintain data governance policies and procedures to ensure data quality, privacy, and security.
• Continuously monitor and optimize the performance of our data platform to ensure scalability, reliability, and cost-effectiveness.
• Keep up-to-date with the latest trends and technologies in the field of data engineering and share knowledge and best practices with the team.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 5+ years of experience in data engineering or related fields.
• Strong proficiency in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium.
• Experience with data warehousing, data modeling, and data governance.
• Experience working with large-scale distributed systems and cloud platforms (e.g., AWS, GCP, Azure).
• Strong problem-solving skills and ability to work independently and collaboratively.
• Excellent communication and interpersonal skills.

If you are a self-motivated and driven individual with a passion for data engineering and a strong background in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium, we encourage you to apply for this exciting opportunity. We offer competitive compensation, comprehensive benefits, and a collaborative work environment that fosters innovation and growth.

Job Sector: IT, Software

Job Type: Permanent

Location: Chennai

Experience: 10 - 20 Years

Salary: 12 – 40 LPA

Education: Any Graduate

Notice Period: Immediate

Key Skills: Python, Spark, AWS, SQL, PySpark

Contact at triple eight two zero nine four two double seven

Job Description:

Requirements

• Minimum 12 years experience
• In depth understanding and knowledge on distributed computing with spark.
• Deep understanding of Spark Architecture and internals
• Proven experience in data ingestion, data integration and data analytics with spark, preferably PySpark.
• Expertise in ETL processes, data warehousing and data lakes.
• Hands on with python for Big data and analytics.
• Hands on in agile scrum model is an added advantage.
• Knowledge on CI/CD and orchestration tools is desirable.
• AWS S3, Redshift, Lambda knowledge is preferred

1. Minimum 2.5 years of experience as a Python Developer.
2. Minimum 2.5 years of experience in any framework like Django/Flask/Fast API
3. Minimum 2.5 years of experience in SQL/ Postgress
4. Minimum 2.5 years of experience in Git/Gitlab/Bit-Bucket
5. Minimum 2+ years of experience in deployment (CICD with Jenkins)
6. Minimum 2.5 years of experience in any cloud like AWS/GCP/Azure

Responsibilities:

• Design and develop strong analytics system and predictive models
• Managing a team of data scientists, machine learning engineers, and big data specialists
• Identify valuable data sources and automate data collection processes
• Undertake pre-processing of structured and unstructured data
• Analyze large amounts of information to discover trends and patterns
• Build predictive models and machine-learning algorithms
• Combine models through ensemble modeling
• Present information using data visualization techniques
• Propose solutions and strategies to business challenges
• Collaborate with engineering and product development teams

Requirements:

• Proven experience as a seasoned Data Scientist
• Good Experience in data mining processes
• Understanding of machine learning and Knowledge of operations research is a value addition
• Strong understanding and experience in R, SQL, and Python; Knowledge base with Scala, Java, or C++ is an asset
• Experience using business intelligence tools (e. g. Tableau) and data frameworks (e. g. Hadoop)
• Strong math skills (e. g. statistics, algebra)
• Problem-solving aptitude
• Excellent communication and presentation skills
• Experience in Natural Language Processing (NLP)
• Strong competitive coding skills
• BSc/BA in Computer Science, Engineering or relevant field; graduate degree in Data Science or other quantitative field is preferred

Job ID: RP100

Work Location: Remote

Required Experience: 4 to 7 years

Job Description

• Must have Google Cloud Big Query experience
• Strong experience with data analysis, data modeling and governance, with excellent analytical and problem-solving abilities
• Good knowledge of Data Warehouses, data flow ETL pipelines
• Design, configuration/administration of database software in Cloud platform.
• Monitoring, Troubleshooting, and Performance tuning the DB objects.
• Experience on Table Partition, Clustered Table, Materialized View, External Tables etc.

Anyone RDBMS technologies

• Good experience in DB design with knowledge of ER Diagram, PK/FK, Stored procedure, Function, Triggers, and Indexes.
• Understanding the requirement of the App team and creating the necessary DB objects by following the best practices.
• Managing logins and database users, as well as database roles, application roles, and other security principles within the database.
• Deep knowledge about Indexes, Performance tuning, and Complex SQL Query patterns.
• Monitoring, Tuning, and Troubleshooting the database-related issues.

About Us:

Mobile programming LLC is a US-based digital transformation company. We help enterprises transform ideas into innovative and intelligent solutions, governing the Internet of Things, Digital Commerce, Business Intelligence Analytics, and Cloud Programming. Bring your challenges to us, we will give you the smartest solutions. From conceptualizing and engineering to advanced manufacturing, we help customers build and scale products fit for the global marketplace.

Mobile programming LLC has offices located in Los Angeles, San Jose, Glendale, San Diego, Phoenix, Plano, New York, Fort Lauderdale, and Boston. Mobile programming is SAP Preferred Vendor, Apple Adjunct Partner, Google Empaneled Mobile Vendor, and Microsoft Gold Certified Partner.

• Adept at Machine learning techniques and algorithms.

Feature selection, dimensionality reduction, building and

• optimizing classifiers using machine learning techniques
• Data mining using state-of-the-art methods
• Doing ad-hoc analysis and presenting results
• Proficiency in using query languages such as N1QL, SQL

Experience with data visualization tools, such as D3.js, GGplot,

• Plotly, PyPlot, etc.

Creating automated anomaly detection systems and constant tracking

• of its performance
• Strong in Python is a must.
• Strong in Data Analysis and mining is a must
• Deep Learning, Neural Network, CNN, Image Processing (Must)

Building analytic systems - data collection, cleansing and

• integration

Experience with NoSQL databases, such as Couchbase, MongoDB,

Cassandra, HBase

Do you have a passion for computer vision and deep learning problems? We are looking for someone who thrives on collaboration and wants to push the boundaries of what is possible today! Material Depot (materialdepot.in) is on a mission to be India’s largest tech company in the Architecture, Engineering and Construction space by democratizing the construction ecosystem and bringing stakeholders onto a common digital platform. Our engineering team is responsible for developing Computer Vision and Machine Learning tools to enable digitization across the construction ecosystem. The founding team includes people from top management consulting firms and top colleges in India (like BCG, IITB), and have worked extensively in the construction space globally and is funded by top Indian VCs.

Our team empowers Architectural and Design Businesses to effectively manage their day to day operations. We are seeking an experienced, talented Data Scientist to join our team. You’ll be bringing your talents and expertise to continue building and evolving our highly available and distributed platform.

Our solutions need complex problem solving in computer vision that require robust, efficient, well tested, and clean solutions. The ideal candidate will possess the self-motivation, curiosity, and initiative to achieve those goals. Analogously, the candidate is a lifelong learner who passionately seeks to improve themselves and the quality of their work. You will work together with similar minds in a unique team where your skills and expertise can be used to influence future user experiences that will be used by millions.

• Extensive knowledge in machine learning and deep learning techniques
• Solid background in image processing/computer vision
• Experience in building datasets for computer vision tasks
• Experience working with and creating data structures / architectures
• Proficiency in at least one major machine learning framework
• Experience visualizing data to stakeholders
• Ability to analyze and debug complex algorithms
• Good understanding and applied experience in classic 2D image processing and segmentation
• Robust semantic object detection under different lighting conditions
• Segmentation of non-rigid contours in challenging/low contrast scenarios
• Sub-pixel accurate refinement of contours and features
• Experience in image quality assessment
• Experience with in depth failure analysis of algorithms
• Highly skilled in at least one scripting language such as Python or Matlab and solid experience in C++
• Creativity and curiosity for solving highly complex problems
• Excellent communication and collaboration skills
• Mentor and support other technical team members in the organization
• Create, improve, and refine workflows and processes for delivering quality software on time and with carefully calculated debt
• Work closely with product managers, customer support representatives, and account executives to help the business move fast and efficiently through relentless automation.

• You’re part of an agile, multidisciplinary team.
• You bring your own unique skill set to the table and collaborate with others to accomplish your team’s goals.
• You prioritize your work with the team and its product owner, weighing both the business and technical value of each task.
• You experiment, test, try, fail, and learn continuously.
• You don’t do things just because they were always done that way, you bring your experience and expertise with you and help the team make the best decisions.

• Strong knowledge of and experience with the functional programming paradigm.
• Experience conducting code reviews, providing feedback to other engineers.
• Great communication skills and a proven ability to work as part of a tight-knit team.

We are actively seeking a Senior Data Engineer experienced in building data pipelines and integrations from 3rd party data sources by writing custom automated ETL jobs using Python. The role will work in partnership with other members of the Business Analytics team to support the development and implementation of new and existing data warehouse solutions for our clients. This includes designing database import/export processes used to generate client data warehouse deliverables.