Data Pipeline

About TensorIoT

TensorIoT is an AWS Advanced Consulting Partner. We help companies realize the value and efficiency of the AWS ecosystem. From building PoCs and MVPs to production-ready applications, we are tackling complex business problems every day and developing solutions to drive customer success.

TensorIoT's founders helped build world-class IoT and AI platforms at AWS and Google and are now creating solutions to simplify the way enterprises incorporate edge devices and their data into their day-to-day operations. Our mission is to help connect devices and make them intelligent. Our founders firmly believe in the transformative potential of smarter devices to enhance our quality of life, and we're just getting started!

TensorIoT is proud to be an equal-opportunity employer. This means that we are committed to diversity and inclusion and encourage people from all backgrounds to apply. We do not tolerate discrimination or harassment of any kind, and make our hiring decisions based solely on qualifications, merit, and business needs at the time.

You have:

• Study and transform data science prototypes.
• Research and implement appropriate ML algorithms and tools.
• Data exploratory analysis
• Feature engineering
• Implement basic ETL pipelines with database/data lake tools
• Conduct scientific research on the latest technologies and ML models
• Run machine-learning tests and experiments.
• Perform statistical analysis and fine-tuning using test results.
• Extend existing ML libraries and frameworks.
• Present scientific research methodologies to the audience

Machine Learning Engineer responsibilities include:

• Designing and developing machine learning and deep learning systems
• Running machine learning tests and experiments
• Implementing appropriate ML algorithms

Must have/Requirements.

·      Proven experience as a Machine Learning Engineer or similar role

·      Must have experience with integrating applications and platforms with cloud technologies (i.e., AWS)

·      Docker containers knowledge

·      Know about distributed training tools

·      Experience with GPU acceleration (i.e., CUDA and cuDNN)

·      Create feature engineering pipelines to process high-volume, multi-dimensional, unstructured (audio, video, NLP) data at scale.

·      Knowledge of server-less architectures (e.g., Lambda, Kinesis, Glue).

·      Understanding of end-to-end ML project lifecycle.

·      Must have experience with Data Science tools and frameworks (i.e., Python, Scikit, NLTK, NumPy, Pandas, TensorFlow, Kera’s, R, Spark, PyTorch).

·      Experience with cloud-native technologies, microservices design, and REST APIs.

·      Knowledge of data query and data processing tools (i.e., SQL)

·      Deep knowledge of Math, Probability, Statistics, and Algorithms

·      Strong understanding of image recognition & computer vision.

·      Must have 4-8 years of experience.

·      Excellent communication skills

·      Ability to work in a team.

·      BSc or Master in Computer Science, Mathematics, or a similar field;

· Ph.D. degree is a plus.

Job Title

Data Analyst

Job Brief

The successful candidate will turn data into information, information into insight and insight into business decisions.

Data Analyst Job Duties

Data analyst responsibilities include conducting full lifecycle analysis to include requirements, activities and design. Data analysts will develop analysis and reporting capabilities. They will also monitor performance and quality control plans to identify improvements.

Responsibilities

● Interpret data, analyze results using statistical techniques and provide ongoing reports.

● Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality.

● Acquire data fromprimary orsecondary data sources andmaintain databases/data systems.

● Identify, analyze, and interpret trends orpatternsin complex data sets.

● Filter and “clean” data by reviewing computerreports, printouts, and performance indicatorsto locate and correct code problems.

● Work withmanagementto prioritize business and information needs.

● Locate and define new processimprovement opportunities. 

Requirements

● Proven working experienceas aData Analyst or BusinessDataAnalyst.

● Technical expertise regarding data models, database design development, data mining and segmentation techniques.

● Strong knowledge of and experience with reporting packages (Business Objects etc), databases (SQL etc), programming (XML, Javascript, or ETL frameworks).

● Knowledge of statistics and experience using statistical packages for analyzing datasets (Excel, SPSS, SAS etc).

● Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.

● Adept atqueries,reportwriting and presenting findings.

Job Location SouthDelhi, New Delhi 

We are seeking an experienced Senior Data Platform Engineer to join our team. The ideal candidate should have extensive experience with Pyspark, Airflow, Presto, Hive, Kafka and Debezium, and should be passionate about developing scalable and reliable data platforms.

Responsibilities:

• Design, develop, and maintain our data platform architecture using Pyspark, Airflow, Presto, Hive, Kafka, and Debezium.
• Develop and maintain ETL processes to ingest, transform, and load data from various sources into our data platform.
• Work closely with data analysts, data scientists, and other stakeholders to understand their requirements and design solutions that meet their needs.
• Implement and maintain data governance policies and procedures to ensure data quality, privacy, and security.
• Continuously monitor and optimize the performance of our data platform to ensure scalability, reliability, and cost-effectiveness.
• Keep up-to-date with the latest trends and technologies in the field of data engineering and share knowledge and best practices with the team.

Requirements:

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 5+ years of experience in data engineering or related fields.
• Strong proficiency in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium.
• Experience with data warehousing, data modeling, and data governance.
• Experience working with large-scale distributed systems and cloud platforms (e.g., AWS, GCP, Azure).
• Strong problem-solving skills and ability to work independently and collaboratively.
• Excellent communication and interpersonal skills.

If you are a self-motivated and driven individual with a passion for data engineering and a strong background in Pyspark, Airflow, Presto, Hive, Datalake, and Debezium, we encourage you to apply for this exciting opportunity. We offer competitive compensation, comprehensive benefits, and a collaborative work environment that fosters innovation and growth.

Our Client enables enterprises in their digital transformation journey by offering Consulting & Implementation Services related to Data Analytics &Enterprise Performance Management (EPM).

Job Location : Noida

Position – Azure Solution Architect

Notice period- Immediate to 60 days

Experience –  6+

Job Description:

Your Role and Responsibilities

• Able to drive the technology design meetings, propose technology design and architecture
• Experienced in the design and delivery of enterprise level Highly Available solutions
• Work closely with project management teams to successfully monitor progress of implementation
• Collaborate with Pre-sales team on RFP
• Provide detailed specifications for proposed solutions
• Experienced to Migrate applications to cloud
• Experienced to create Data Lake and Data warehouse solutions
• Ability to implement the solution as per technical requirements
• Identity, authentication, security, privacy, and compliance including Active Directory modern Application Architecture (Queue’s, Micro-Services, Containers etc)

Required Technical and Professional Expertise

• Project management and leadership skills are essential.
• 4+ years of experience developing IT and cloud infrastructure (MS Azure, GPC).
• Working knowledge of MS Azure technology stack and related technology (ie. Data Factory, Data Flow, Synapse Analytics, Synapse ML, Gen2 storage, etc.).
• Master's degree in Computer Science or Software Engineering preferred.
• Current understanding of best practices regarding system security measures.
• Experience in software engineering and design architecture.
• Positive outlook in meeting challenges and working to a high level.
• Advanced understanding of business analysis techniques and processes.
• Good to have Azure ML experience

Preferred Technical and Professional Experience

• MS Azure Certification: Fundamentals, Solution Architect, Data Engineer

Data Modeler JD: - 

1. Understand and translate business needs into dimension models supporting long-term solutions

2. Experience on building models on ERwin or similar tools.

3. Experience and understanding on dimensional data model, customer 360 and Entity relationship model.

4. Work with the Development team to implement data strategies, build data flows and develop conceptual data models.

5. Create logical and physical data models using best practices to ensure high data quality and reduced redundancy

6. Optimize and update logical and physical data models to support new and existing projects

7. Maintain conceptual, logical, and physical data models along with corresponding metadata

8. Develop best practices for standard naming conventions and coding practices to ensure consistency of data models

9. Recommend opportunities for reuse of data models in new environments

10. Perform reverse engineering of physical data models from databases and SQL scripts

11. Evaluate models and physical databases for variances and discrepancies

12. Validate business data objects for accuracy and completeness

13. Analyze data-related system integration challenges and propose appropriate solutions

14. Develop data models according to company standards

15. Guide System Analysts, Engineers, Programmers and others on project limitations and capabilities, performance requirements and interfaces

16. Good to have Home appliance/Retail domain knowledge and Azure Synapse.

Job Functions: Information Technology 

Employment Type - Full-time 

Thank you!

About us

SteelEye is the only regulatory compliance technology and data analytics firm that offers transaction reporting, record keeping, trade reconstruction, best execution and data insight in one comprehensive solution. The firm’s scalable secure data storage platform offers encryption at rest and in flight and best-in-class analytics to help financial firms meet regulatory obligations and gain competitive advantage.

The company has a highly experienced management team and a strong board, who have decades of technology and management experience and worked in senior positions at many leading international financial businesses. We are a young company that shares a commitment to learning, being smart, working hard and being honest in all we do and striving to do that better each day. We value all our colleagues equally and everyone should feel able to speak up, propose an idea, point out a mistake and feel safe, happy and be themselves at work.

Being part of a start-up can be equally exciting as it is challenging. You will be part of the SteelEye team not just because of your talent but also because of your entrepreneurial flare which we thrive on at SteelEye. This means we want you to be curious, contribute, ask questions and share ideas. We encourage you to get involved in helping shape our business. What you'll do

What you will do?

• Deliver plugins for our python based ETL pipelines.
• Deliver python services for provisioning and managing cloud infrastructure.
• Design, Develop, Unit Test, and Support code in production.
• Deal with challenges associated with large volumes of data.
• Manage expectations with internal stakeholders and context switch between multiple deliverables as priorities change.
• Thrive in an environment that uses AWS and Elasticsearch extensively.
• Keep abreast of technology and contribute to the evolution of the product.
• Champion best practices and provide mentorship.

What we're looking for

• Python 3.
• Python libraries used for data (such as pandas, numpy).
• AWS.
• Elasticsearch.
• Performance tuning.
• Object Oriented Design and Modelling.
• Delivering complex software, ideally in a FinTech setting.
• CI/CD tools.
• Knowledge of design patterns.
• Sharp analytical and problem-solving skills.
• Strong sense of ownership.
• Demonstrable desire to learn and grow.
• Excellent written and oral communication skills.
• Mature collaboration and mentoring abilities.

What will you get?

• This is an individual contributor role. So, if you are someone who loves to code and solve complex problems and build amazing products and not worry about anything else, this is the role for you.
• You will have the chance to learn from the best in the business who have worked across the world and are technology geeks.
• Company that always appreciates ownership and initiative. If you are someone who is full of ideas, this role is for you.

• Gathering project requirements from customers and supporting their requests.
• Creating project estimates and scoping the solution based on clients’ requirements.
• Delivery on key project milestones in line with project Plan/ Budget.
• Establishing individual project plans and working with the team in prioritizing production schedules.
• Communication of milestones with the team and to clients via scheduled work-in-progress meetings
• Designing and documenting product requirements.
• Possess good analytical skills - detail-orientemd
• Be familiar with Microsoft applications and working knowledge of MS Excel
• Knowledge of MIS Reports & Dashboards
• Maintaining strong customer relationships with a positive, can-do attitude

Must Have Skills:

• Good experience in Pyspark - Including Dataframe core functions and Spark SQL
• Good experience in SQL DBs - Be able to write queries including fair complexity.
• Should have excellent experience in Big Data programming for data transformation and aggregations
• Good at ELT architecture. Business rules processing and data extraction from Data Lake into data streams for business consumption.
• Good customer communication.
• Good Analytical skills

Technology Skills (Good to Have):

• Building and operationalizing large scale enterprise data solutions and applications using one or more of AZURE data and analytics services in combination with custom solutions - Azure Synapse/Azure SQL DWH, Azure Data Lake, Azure Blob Storage, Spark, HDInsights, Databricks, CosmosDB, EventHub/IOTHub.
• Experience in migrating on-premise data warehouses to data platforms on AZURE cloud. 
• Designing and implementing data engineering, ingestion, and transformation functions
• Azure Synapse or Azure SQL data warehouse
• Spark on Azure is available in HD insights and data bricks