Overview

The Data Engineer will build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse. They will develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-architect existing data loading processes and data structures.

This is a hybrid position.

Responsibilities

• Design and develop ELT / ETL to load data from internal sources into operational data stores, Azure data lake and data warehouse.
• Create APIs, data structures and schemas to store and retrieve data.
• Create and maintain data mappings, data definitions, architecture and data flow diagrams.
• Implement solutions to validate and test data quality.
• Build proof-of-concepts to determine viability of new processes and technologies.
• Deploy and manage code in non-prod and prod environments.
• Troubleshoot data related issues and fix defects.
• Optimize and performance tune databases and queries.
• Participate in code reviews and train other team members.
• Document code and create code templates based on industry best practices.

Qualifications

• Bachelor’s Degree and/or equivalent experience required.
• Minimum 5 years of experience working as a Data Engineer, ETL Engineer, Data/ETL Architect or similar roles.

Skills

• 5+ years of solid continuous experience in Python.
• 3+ years of continuous experience with Airflow.
• 3+ years of experience with Azure Data Factory (ADF) or similar.
• 3+ years of experience working with relational databases: Oracle, SQL Server, PostgreSQL, or similar.
• 3+ years of experience working with NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar
• 3+ years of experience writing SQL code.
• 3+ years of experience in Kimball Dimensional Modeling (Star-Schema).
• 2+ years of experience in columnar databases: Snowflake, Azure Synapse, or similar.
• 2+ years of Azure / AWS experience.