Lead Data Engineer
Outside IR35
£500–£525 per day
3 Month Initial Contract
We're looking for an experienced Lead Data Engineer to join a high-profile project. This is a hybrid role, requiring 2 days per week onsite in London, with the remainder worked remotely.
What we're looking for:
- Proven experience leading data engineering teams, with a strong track record of mentoring and developing junior engineers.
- Expert knowledge of Python and Airbyte CDK, with experience building custom data connectors (a key focus of the role).
- Strong experience developing ELT pipelines using PySpark in Databricks within a Medallion Architecture.
- Experience with Terraform and deploying Airbyte connections and Databricks assets using GitHub Actions.
- Confident working with JIRA, GitHub, and VS Code.
Desirable:
- Experience with Databricks Delta Live Tables (DLT).
- Knowledge of Databricks Asset Bundles.