26 azure databricks data engineer jobs found

Senior Data Engineer - Azure, Databricks, Datafactory, Phinsys Tandem, Insurance Our sister company Stellarmann is seeking an experienced Data Engineer to join their team, who are building out a new data platform for processing Finance and Actuarial data. You will be working as part of a talented team building a cutting edge data platform using Azure Databricks, Datafactory and Python integrating with Phinsys Tandem. We're seeking someone with: Proven experience as a Data Engineer Strong hands on experience with Azure Databricks and Azure Data Factory Experience in the Lloyd's/London Market (specialty insurance) Experience working with Phinsys Tandem - essential Proficiency in Python Experience working with Medallion architecture/tiered medallion Lakehouse Experience using Databricks Notebooks for data pipelines This is a 6 month initial engagement, outside IR35 with scope to extend beyond the initial term. If this opportunity aligns with your...

Role: Data Engineer (Azure Databricks Contract) Day Rate: £250-£300 (Outside IR35) Duration: 2 months (Initially) Location: Fully Remote (UK-based candidates only) Start Date: 5th January 2026 This role will focus on Azure Databricks, revamping Salesforce integrations, modernising data pipelines, and building a new integration layer to replace legacy systems. Key Responsibilities: Lead development and optimisation of Azure Databricks pipelines and workflows. Work with Azure Data Factory and Azure Data Streaming to ingest, transform, and deliver data in real time. Collaborate with developers to make C# modifications as part of integration improvements. Rebuild and enhance data flows between Salesforce and other systems. Design and implement a new integration layer to replace old estate connections. Key Skills & Experience: Proven experience as a Databricks Specialist in enterprise environments. Strong knowledge of Azure Data...

Sibling is seeking a Freelance Data Engineer to architect a green-energy analytics platform. This is a hands-on, high-scale role focused on processing global telemetry data. Bio Architect & Code: Build robust, scalable ETL pipelines using PySpark and Databricks. Optimize: Refactor existing Spark jobs for maximum performance and cost-efficiency. Integrate: Deploy production-grade solutions within Azure (Unity Catalog, Delta Lake). Collaborate: Act as the technical bridge between Data Science and Business stakeholders. Expertise: Deep, recent experience in PySpark and large-scale Databricks environments. Cloud: Strong Azure experience (ADF, ADLS Gen2). DevOps: Proven track record with CI/CD, Git, and data engineering best practices. Compliance: Must have the Right to Work in the UK (No sponsorship available). Contract Details: Rate: Competitive Market Day Rate. Timeline: Kick-off in April, running through year-end. Assessment: Confirmed Outside...

Our sister company Stellarmann is seeking an experienced Data Engineer to support a London Market Insurance client in building a modern Azure data environment. The Data Engineer will work alongside the wider team in building a modern Data Lakehouse with three tiers. We're seeking someone with: Proven experience as a Data Engineer Strong hands on experience with Azure Databricks and Azure Data Factory Experience in the Lloyd's/London Market (specialty insurance) Proficiency in Python Experience with SQL Server Exposure to DevOps automation and deployment pipelines (YAML) Experience working with Medallion architecture/tiered medallion Lakehouse Strong understanding of Delta Lakes and Delta Tables Experience using Databricks Notebooks for data pipelines The initial engagement will run until early September 2026 and the role is paying a competitive day rate outside IR35, offering hybrid working in the London office (2 days per week). If this...

Senior Power BI Consultant - Outside IR35 Location: Central London (2-3 days on-site) Contract: 6-12 months, Outside IR35 Industry: Oil & Gas/Commodity Trading Role Summary A leading energy & commodity trading business is looking for a Senior Power BI Consultant to support ongoing analytics and data-modelling work. The role blends hands-on BI development with business engagement. Key Responsibilities * Build and optimise Power BI dashboards, semantic models, and data flows. * Lead data modelling across complex trading datasets. * Translate business requirements into technical specifications for offshore developers. * Ensure best practice in DAX, SQL, and BI design. * Work within the organisation's Databricks environment (platform familiarity expected). * Provide leadership, guidance, and clear communication to both business and technical teams. Required Skills * Expert-level Power BI, DAX, and data modelling. * Strong SQL and experience with large datasets. *...

Role: Junior Azure Data Engineer Length: 6 Month Contract (Scope to Extend) Location: 3 Days per Week Onsite Rate: £350 - £370 p/d Outside IR35 A data-driven Specialty Insurer is seeking a hands-on Junior Azure Data Engineer to join their established cloud-native data team on an initial 6-month contract, with strong scope for extension. The organisation has invested significantly in building out a modern Databricks-based data platform over recent years. The focus now shifts to driving value from that investment, enhancing, optimising and further developing the platform to support advanced analytics and business insight. You will have demonstrated experience building & optimizing pipelines using ADF, with strong Python & SQL coding ability and demonstratable experience working on Databricks platform. Responsibilities: Enhance and develop an existing Azure Databricks data platform to improve performance, scalability and usability Design, build...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £400pd-£440pd (Outside IR35) Contract: 6 months initial We are seeking a highly skilled Data Engineer to support a Material Spend Project. You will play a crucial role in extracting, transforming, and analysing large data sets from multiple sources, including API integrations and ServiceNow, to drive actionable insights and support strategic decision-making. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture API Integration experience Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data manipulation, transformation, and quality...

I'm currently recruiting for a Azure Platform Engineer role with a market-leading analytics consultancy. Required skills: 5+ years professional experience in Platform engineering roles Terraform for Azure infrastructure as code GitHub Actions & CI/CD principles Azure Private Link & Private Link Service Expert knowledge of Databricks & Unity catalog for data governance (Must have) Azure Policy & compliance enforcement Identity & access management (OAuth, federated credentials) Azure security best practices (Defense in Depth, BCDR, high availability) Location - Remote (UK only) Daily rate - £450 (outside IR35) Contract length - 10 weeks (possible extension) Start date - 1st April Sponsorship is not available

Data Engineer l Informatica l Data governance l SQL Database l Databricks l Stakeholder management l integration l Financial Services I £500-£525 per day Outside IR35 3-6-month initial Hybrid Remote - London One of my consulting partners is kicking off a major data governance and integration programme for a leading insurance client, and we're looking for three strong Data Engineers with deep experience implementing and optimising Informatica solutions end-to-end from both on prem and cloud systems. Core Skills Required Extensive Informatica experience (IICS / PowerCenter / IDMC) across design, build, integration, and deployment Strong background in Data Governance , including: Enterprise metadata management Data lineage setup and optimisation Data quality rules, profiling, and stewardship workflows Policy framework implementation Governance operating model support Experience working with Informatica Data Quality (IDQ) , Axon , or similar governance...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £400pd-£440pd (Outside IR35) Contract: 6 months initial We are seeking a highly skilled Data Engineer to support a Material Spend Project. You will play a crucial role in extracting, transforming, and analysing large data sets from multiple sources, including API integrations and ServiceNow, to drive actionable insights and support strategic decision-making. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture API Integration experience Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data manipulation, transformation, and quality...

Data Engineer l Informatica l Data governance l SQL Database l Databricks l Stakeholder management l integration l Financial Services I £500- £525 per day | Outside IR35 | 3-6-month initial | Hybrid Remote - London One of my consulting partners is kicking off a major data governance and integration programme for a leading insurance client, and we 're looking for three strong Data Engineers with deep experience implementing and optimising Informatica solutions end-to-end from both on prem and cloud systems. Core Skills Required Extensive Informatica experience (IICS / PowerCenter / IDMC) across design, build, integration, and deployment Strong background in Data Governance , including: Enterprise metadata management Data lineage setup and optimisation Data quality rules, profiling, and stewardship workflows Policy framework implementation Governance operating model support Experience working with Informatica...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £400pd- £440pd (Outside IR35) Contract: 6 months initial We are seeking a highly skilled Data Engineer to support a Material Spend Project. You will play a crucial role in extracting, transforming, and analysing large data sets from multiple sources, including API integrations and ServiceNow, to drive actionable insights and support strategic decision-making. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture API Integration experience Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data...

Our sister company Stellarmann is seeking an experienced Data Engineer to support a London Market Insurance client in building a modern Azure data environment. The Data Engineer will work alongside the wider team in building a modern Data Lakehouse with three tiers. We're seeking someone with: Proven experience as a Data Engineer Strong hands on experience with Azure Databricks and Azure Data Factory Experience in the Lloyd's/London Market (specialty insurance) Proficiency in Python Experience with SQL Server Exposure to DevOps automation and deployment pipelines (YAML) Experience working with Medallion architecture/tiered medallion Lakehouse Strong understanding of Delta Lakes and Delta Tables Experience using Databricks Notebooks for data pipelines The initial engagement will run until early September 2026 and the role is paying a competitive day rate outside IR35, offering hybrid working in the London office (2 days per week). If this...

Role Overview We are seeking a skilled Data Engineer with strong expertise in Microsoft Azure and Databricks to design, build, and optimize scalable data pipelines and analytics solutions. You will play a key role in building modern data platforms that support business intelligence, advanced analytics, and machine learning initiatives. This role requires hands-on experience with cloud-native data architectures, distributed processing, and best practices in data engineering. Key Responsibilities Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, and related Azure services Build and optimize data workflows using Apache Spark within Databricks Develop and maintain data lakes and lakehouse architectures using Azure Data Lake Storage Implement data modeling solutions for analytics and reporting Ensure data quality, governance, and security best practices Collaborate with data analysts, data scientists, and business...

Platform Engineer - Outside IR35 - Remote About the Role We are seeking a highly skilled Platform Engineer with strong expertise in Microsoft Azure and Azure Databricks to design, build, and maintain scalable, secure, and high-performing data platforms. You will play a critical role in enabling data engineering, analytics, and AI workloads by developing robust cloud-native infrastructure and data services. This role sits at the intersection of cloud infrastructure, DevOps, and data engineering, ensuring platform reliability, performance, and governance across the organization. Key Responsibilities Design, implement, and maintain cloud-native platform solutions in Microsoft Azure Build, optimize, and manage data platforms using Azure Databricks Develop Infrastructure as Code (IaC) using tools such as Terraform, Bicep, or ARM templates Implement CI/CD pipelines for data and platform deployments Ensure high availability, scalability, security, and cost...

Data Engineer | Outside IR35 | £400 - £500 | 6 months | Hybrid Nottingham We’re looking for a highly skilled Data Engineer to join a growing data team supporting a large-scale modern data platform project. Working 3 days a week onsite on the outskirts of Nottingham, you'll be a key contributor in evolving the organisation’s data capability, focusing on best‑practice engineering, clean architecture, and high‑value BI delivery.Key Responsibilities Design, build and optimise ETL/ELT pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python. Work hands‑on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI. Implement best practices for data quality, governance, lineage and automation. Collaborate with cross‑functional teams including analysts, product owners and business stakeholders. Support performance tuning, cost optimisation and...

Data Engineer | Outside IR35 | £400 - £500 | 6 months | Hybrid Nottingham We’re looking for a highly skilled Data Engineer to join a growing data team supporting a large-scale modern data platform project. Working 3 days a week onsite on the outskirts of Nottingham, you'll be a key contributor in evolving the organisation’s data capability, focusing on best‑practice engineering, clean architecture, and high‑value BI delivery. Key Responsibilities Design, build and optimise ETL/ELT pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python . Work hands‑on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI . Implement best practices for data quality, governance, lineage and automation. Collaborate with cross‑functional teams including analysts, product owners and business stakeholders. Support performance...

Data Engineer | Outside IR35 | £400 - £500 | 6 months | Hybrid Nottingham We're looking for a highly skilled Data Engineer to join a growing data team supporting a large-scale modern data platform project. Working 3 days a week onsite on the outskirts of Nottingham, you'll be a key contributor in evolving the organisation's data capability, focusing on best practice engineering, clean architecture, and high value BI delivery. Key Responsibilities Design, build and optimise ETL/ELT pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python . Work hands on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI . Implement best practices for data quality, governance, lineage and automation. Collaborate with cross functional teams including analysts, product owners and business stakeholders. Support performance tuning,...

Data Engineer | Outside IR35 | £400 - £500 | 6 months | Hybrid Nottingham We're looking for a highly skilled Data Engineer to join a growing data team supporting a large-scale modern data platform project. Working 3 days a week onsite on the outskirts of Nottingham, you 'll be a key contributor in evolving the organisation's data capability, focusing on best-practice engineering, clean architecture, and high-value BI delivery. Key Responsibilities Design, build and optimise ETL/ELT pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python . Work hands-on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI . Implement best practices for data quality, governance, lineage and automation. Collaborate with cross-functional teams including analysts, product owners and business...

Contract Length: Initial 6 months (extension likely) Day Rate: £500 per day (Outside IR35)Location: Hybrid in South Wales Role OverviewWe are seeking an experienced Senior Data Engineer contractor to support and extend the Azure Databricks Lakehouse platform, ensuring the reliable delivery of new data pipelines alongside ongoing operational stability.This is a hands‑on engineering role focused on implementation, optimisation and support of Delta Lake pipelines across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework. Key Responsibilities Develop and maintain data pipelines in Azure Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark‑driven CDC frameworks Implement and maintain Slowly Changing Dimension (SCD Type 2) logic Engineer time‑aware joins and dimensional transformations Support optimisation of Spark workloads and Delta tables Solid...