We are offering a contract to permanent employment opportunity in the non-profit industry, located in New York, New York. The Big Data Engineer will be tasked with optimizing the design and architecture of data pipelines, working closely with our Agile/Scrum teams, and leveraging their knowledge of Python, Sequel, and Cloud technologies.

Key responsibilities:

• Design and optimize data pipelines, ensuring efficient data flow and storage
• Collaborate with Agile/Scrum teams to deliver high-quality projects on schedule
• Utilize Python, Sequel, and Cloud technologies in day-to-day tasks and projects
• Apply knowledge of OLTP and OLAP database design to enhance system performance
• Develop and maintain data models, both normalized and dimensional, to support various business needs
• Implement ETL and ELT architectures for real and near-real-time scenarios
• Apply data architecture implementation patterns to optimize data flow and storage
• Use version control (GIT) processes to manage code versions and history
• Set up automated pipeline deployments, particularly in Azure, to streamline delivery
• Stay updated on the latest industry trends and technologies to continuously improve our data engineering practices.• Demonstrated proficiency in building and optimizing data pipelines, architectures, and data sets.
• Proven ability in data modeling and data architecture.
• Experience in Agile Scrum methodologies.
• Proficiency in Azure SQL Database.
• Advanced skills in Python programming language.
• Proficiency in Oracle database management.
• Demonstrated experience in Cloud engineering.
• A degree in Computer Science, IT, or a related field is preferred.
• Previous work experience in non-profit sectors is a plus.
• Excellent verbal and written communication skills.
• Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy.
• Ability to work collaboratively in a team environment.
• Strong problem-solving skills and the ability to think critically and strategically.
• Adaptable and able to learn quickly in a fast-paced environment.
• Ability to maintain confidentiality and handle sensitive information with discretion.