Job Description
We are looking for a Data Engineer to be responsible for building and maintaining the infrastructure that supports the organization’s data architecture.
The role involves creating and managing data pipelines using Airflow for data extraction, processing, and loading, ensuring their maintenance, monitoring, and stability.
The engineer will work closely with data analysts and end-users to provide accessible and reliable data
Main Tasks and Responsibilities:
- Responsible for maintaining the infrastructure that supports the current data architecture
- Responsible for creating data pipelines in Airflow for data extracting, processing, and loading
- Responsible for data pipelines maintenance, monitoring, and stability
- Responsible for providing data access to data analysts and end-users
- Responsible for DevOps infrastructure
- Responsible for deploying Airflow dags to production environment using DevOps tools
- Responsible for code and query optimization
- Responsible for code review
- Responsible for improving the current data architecture and DevOps processes
- Responsible for delivering data in useful and appealing ways to users
- Responsible for performing and documenting analysis, review, and study on specified regulatory topics
- Responsible for understanding business change and requirement needs, assess the impact and the cost
