Outside Spy
Nottingham, UK
Location: UK-based (travel to Nottingham up to 2 days per month) Rate: £600 per day (Outside IR35) Clearance: BPSS Contract Length: Until October 2026 Start: ASAP We're looking for a Senior Data Engineer to support delivery of a modern Azure-based data platform, with Azure Databricks as the core engineering and processing layer. You'll be responsible for building scalable, production-grade data pipelines and ensuring strong engineering discipline across the platform. Responsibilities Design, build, and maintain data pipelines using Azure Databricks (Apache Spark) Develop production-quality PySpark code following software engineering best practices Use SQL (Spark SQL/Databricks SQL) for transformations and analytics-ready data preparation Engineer reliable batch and near-real-time ingestion workflows Implement orchestration patterns using Azure Data Factory Optimise Spark workloads for performance and cost efficiency Contribute to...