£600 per day
Outside Spy
Leeds, UK
Lead Databricks Engineer Sector: Finance Contract: 6 months | Outside IR35 Rate : £600 Location: Leeds, with 2 days per week on site The role We are looking for a hands-on Lead Databricks Engineer to lead the delivery and improvement of data pipelines within an Azure Databricks environment. The platform uses a medallion architecture across its tables and views, with a strong focus on Python, reliable data processing and engineering quality. You will combine technical leadership with development, helping the team make sound design decisions, investigate loosely defined requirements and deliver well-tested solutions. Key responsibilities Lead the design, development and optimisation of Databricks pipelines using Python, PySpark and SQL. Build and maintain Delta Live Tables pipelines, managed tables and views across bronze, silver and gold layers. Define and implement data reconciliation and quality checks, investigating discrepancies between source...