39 senior data engineer jobs found in London

We are seeking a Senior Data Engineer / Developer to join a major Liquidity Reporting programme, delivering regulatory reporting solutions for large UK insurers. This is a hands-on role focused on building and enhancing a Databricks -based data engineering framework. You will design and develop scalable PySpark pipelines covering data ingestion, validation, transformation, and mapping, while creating reusable libraries and ensuring high engineering standards. You will also provide technical leadership, conducting code reviews, guiding offshore teams, and ensuring delivery quality aligns with architectural and regulatory requirements. Key Requirements: Strong experience with Databricks and PySpark Solid Azure data platform experience Proven ability to build reusable frameworks and libraries Experience leading code quality and working with offshore teams Desirable: Experience in regulatory reporting or financial services Exposure to Tagetik...

Senior Data Engineer Location: London (Hybrid Working) Type: Contract Pay Rate: £350-£400 (Outside IR35) We're looking for a Senior Data Engineer to join a high-performing data team responsible for delivering scalable, reliable, and innovative data solutions that power analytics and decision-making across a global streaming and digital ecosystem. This is an opportunity to work with modern cloud technologies, large-scale datasets, and a collaborative engineering culture where you'll help shape the future of data platforms used by millions of customers worldwide. What You'll Be Doing Design, build, and support scalable data models and distributed ETL pipelines using modern big data technologies. Collaborate with stakeholders across multiple business functions to understand data challenges and translate requirements into robust technical solutions. Champion DataOps principles, engineering best practices, and continuous improvement initiatives....

Contract – Senior Data Engineer Contract : 6 month sRate : £750 - £800 PD (Outside IR35 )Location : London / Hybrid x2 days onsit e Overvi ewWe are seeking an experienced Senior Data Engineer to join a major data programme transforming how organisations and industry partners access and use health data across the UK. You will design, build and optimise scalable data platforms and pipelines that enable secure, trusted and efficient access to health data. You will collaborate closely with architects, product teams, analysts and engineers to deliver high-quality data solutions that support research, innovation and operational decision-making. This role is ideal for a hands-on Data Engineer who enjoys solving complex data challenges, working with large datasets and building modern cloud-based data solutions within regulated environment s. Role & Responsibilit iesData Engineering & Platform Developm entDesign, build and maintain scalable data...

Senior Data Engineer | Outside IR35 | £450- £550 per day | Hybrid | London We are seeking a Senior Data Engineer/Developer to join a major Liquidity Reporting programme, delivering regulatory reporting solutions for large UK insurers. This is a hands-on role focused on building and enhancing a Databricks -based data engineering framework. You will design and develop scalable PySpark pipelines covering data ingestion, validation, transformation, and mapping, while creating reusable libraries and ensuring high engineering standards. You will also provide technical leadership, conducting code reviews, guiding offshore teams, and ensuring delivery quality aligns with architectural and regulatory requirements. Key Requirements: Strong experience with Databricks and PySpark Solid Azure data platform experience Proven ability to build reusable frameworks and libraries Experience leading code quality and working with offshore teams Desirable: Experience in...

Senior Data Engineer | Outside IR35 | £450- £550 per day | Hybrid | London We are seeking a Senior Data Engineer / Developer to join a major Liquidity Reporting programme, delivering regulatory reporting solutions for large UK insurers. This is a hands-on role focused on building and enhancing a Databricks -based data engineering framework. You will design and develop scalable PySpark pipelines covering data ingestion, validation, transformation, and mapping, while creating reusable libraries and ensuring high engineering standards. You will also provide technical leadership, conducting code reviews, guiding offshore teams, and ensuring delivery quality aligns with architectural and regulatory requirements. Key Requirements: Strong experience with Databricks and PySpark Solid Azure data platform experience Proven ability to build reusable frameworks and libraries Experience leading code quality and working with offshore...

Senior Data Engineer - Contract - Outside IR35 Hybrid working, Central London Up to £650 per day FinTech | London We’re partnered with a fast-growing FinTech at the heart of Private Market Investments, looking to scale their Data function. What They’re Looking For Strong grounding in computer science fundamentals and data modelling Commercial experience with more than one programming language (Python, Scala, Java, C++) (and strong SQL skills) Experience with streaming and data processing (e.g. Kafka, Spark, Athena or similar) Hands-on experience with Azure Familiarity with data containerisation, ingestion, and orchestration Clear communicator who can explain technical decisions to non-technical stakeholders Why Join? Work on genuinely difficult engineering problems Exposure to large, complex systems in real production environments A culture built by engineers, for engineers Hybrid working, Central London Up to £650 per day If you’re...

Senior ML Platform Engineer London 5 Days a Week In Office £550- £700 Per Day, Outside IR35 Immediate Start The Company They are a technically ambitious organisation building advanced machine learning systems tightly integrated with real-world systems. Their platform supports the full lifecycle from data collection through to large-scale training and deployment. Engineering quality, reliability, and thoughtful system design are core to how they operate. The Role and Deliverables Own and evolve the operator-facing platform, including web UIs, backend services, and CLI tooling used for live data collection and dataset workflows. Design and maintain robust recording and orchestration systems that scale across robots, sensors, and lab environments. Collaborate with the ML team on dataset builders, ingestion pipelines, and infrastructure supporting large-scale distributed training. Partner with robotics engineers on ROS2 integration,...

Data Engineer - Contract - Outside IR35 - Wembley Hybrid This position will focus on improving and maturing the existing data platform, developing scalable data structures and pipelines, and supporting the wider business in maximising the value of its data. The environment has been established within Microsoft Fabric for several years, however there remains significant opportunity to improve data warehousing structures, data modelling, governance and reporting capabilities. The successful candidate will work across the full data lifecycle, from ingestion and transformation through to reporting and security, while collaborating closely with Data Analysts and business stakeholders. This is a hands-on role suited to someone who enjoys building practical, maintainable solutions and can balance strong technical capability with a business-focused approach. Key Responsibilities Design, develop and maintain data pipelines within Microsoft Fabric. Build and enhance...

Senior Data Engineer - Contract (outside IR35) - up to £550 per day We’re hiring a Senior Data Engineer to join a leading global organisation, helping to build and enhance a modern data platform within the insurance sector. NB London Market Insurance experience is essential. If you do not have this experience, you will not be eligible for this role. The role You’ll design and develop data pipelines using ETL/ELT, ensuring strong standards in testing, reconciliation, and documentation. You’ll contribute to data modelling, support data delivery, and build APIs/services for stakeholders. You’ll also collaborate across teams, promote best practices, and mentor junior colleagues in an Agile environment. Key requirements London Market Insurance experience (must-have) Advanced T-SQL, SSIS, Azure Data Factory Power BI/SSRS, MDX/DAX Strong data modelling Experience with Azure & on-prem SQL Azure DevOps, data APIs, solid documentation/testing practices...

Data Engineer £550-600/day outside IR35 6-month contract Remote working Working with a FTSE 100 client who have a requirement for a hands-on Senior Data Engineer. The role will be involved in designing and delivering data models, integrations, orchestration pipelines, rule engines and contributing to the platform/infrastructure - MUST have Terraform experience. Data Engineer, key skills: MUST have experience using Terraform to build and manage IAC Proven experience with AWS (eg S3, Lambda, Glue etc) Background in building data pipelines, integrations and transformation layers Strong hands-on SQL experience Data modelling Data quality rules and validation frameworks Snowflake is massively desirable Data Engineer, AWS, Terraform, Snowflake Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and...

Data Engineer £550-600/day outside IR35 6-month contract Remote working Working with a FTSE 100 client who have a requirement for a hands-on Senior Data Engineer. The role will be involved in designing and delivering data models, integrations, orchestration pipelines, rule engines and contributing to the platform/infrastructure - MUST have Terraform experience. Data Engineer, key skills: MUST have experience using Terraform to build and manage IAC Proven experience with AWS (E.g S3, Lambda, Glue etc) Background in building data pipelines, integrations and transformation layers Strong hands-on SQL experience Data modelling Data quality rules and validation frameworks Snowflake is massively desirable Data Engineer, AWS, Terraform, Snowflake Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have...

Lead Platform Engineer Python | Cloud | Terraform | IAM | DevOps | Infrastructure-as-Code (IaC) Rate: Circa £700–£800 per day (Outside IR35) Contract Length: 6 Months Location: London / Hybrid (2 days per week) Overview We are seeking an experienced Principal / Lead Engineer to join a highly ambitious organisation building a large-scale, mission-critical data and technology platform. This is a unique opportunity to join at the earliest stages of a major programme and play a key role in shaping its technical foundation. Working closely with the CTO and a small team of senior engineers, you will lead the design and implementation of core infrastructure, establish engineering standards and practices, and make architectural decisions that will have long-term strategic impact. If you enjoy solving complex technical challenges, influencing technology strategy, and creating robust engineering foundations, this role offers an exceptional opportunity to make a...

AI Engineer - LLM & Agentic Systems London - Hybrid. Circa £500 – £675 per day (and negotiable DOE) Contract (Outside IR35) Mid-level, Senior & Lead. Agentic AI / Machine Learning. Company: Our client is a globally established B2B information and professional services business, operating across multiple high-value industry sectors. They have established data science and machine learning engineering teams already delivering in production, and are now expanding their AI capability significantly across the global organisation. This is a role for people who ship. If you've built LLM-powered systems and agentic workflows that are live in production - not demos, not prototypes - and you want to do that at genuine scale, this is worth applying for. What you'll be working on as an AI Engineer - LLM & Agentic Systems: Designing and building AI agents and agentic workflows - combining LLMs, RAG, reasoning, memory and tool orchestration to...

AI Engineer | Hybrid | London | Outside IR35 | 6+ Months Cititec are working with a global financial markets organisation on a large scale data and platform transformation. They are looking for a Senior AI Engineer to help accelerate the migration of a complex data estate from Oracle to a Microsoft centric architecture across Azure, Cosmos DB, Microsoft Fabric and Azure Data Factory. The role will focus on applying AI and GenAI to improve developer productivity and engineering throughput, particularly across data migration, code generation and developer tooling. This includes deploying AI agents and AI assisted development tools within the Azure ecosystem. This is a high impact role operating across a large engineering environment, helping scale modern engineering practices and AI enabled development across a significant developer base.

Role Summary Wagglyn is looking for an experienced HFT DevOps: infrastructure engineer to work with the existing engineering team on the deployment, operation and production readiness of a low-latency FX trading platform. This is a hands-on contract role for someone who has worked with real production infrastructure, HFT systems, data-centre deployments or high-availability backend platforms. It is not a generic cloud-only DevOps role. Scope The contractor will help prepare, operate and support the infrastructure required to run Wagglyn’s MVP trading stack across our production sites, cloud/UAT environments and interim infrastructure. The role will involve working with the engineering team to document the current environment, define deployment and access processes, support the migration from interim to physical infrastructure, and put in place the monitoring, runbooks and controls needed for production operation. Profile ·     Senior hands-on...

AI Engineer Hybrid London Outside IR35 6+ Months Cititec are working with a global financial markets organisation on a large scale data and platform transformation. They are looking for a Senior AI Engineer to help accelerate the migration of a complex data estate from Oracle to a Microsoft centric architecture across Azure, Cosmos DB, Microsoft Fabric and Azure Data Factory. The role will focus on applying AI and GenAI to improve developer productivity and engineering throughput, particularly across data migration, code generation and developer tooling. This includes deploying AI agents and AI assisted development tools within the Azure ecosystem. This is a high impact role operating across a large engineering environment, helping scale modern engineering practices and AI enabled development across a significant developer base.

AI Engineer - LLM & Agentic Systems London - Hybrid. Circa £500 - £675 per day (and negotiable DOE) Contract (Outside IR35) Mid-level, Senior & Lead. Agentic AI/Machine Learning. Company: Our client is a globally established B2B information and professional services business, operating across multiple high-value industry sectors. They have established data science and machine learning engineering teams already delivering in production, and are now expanding their AI capability significantly across the global organisation. This is a role for people who ship. If you've built LLM-powered systems and agentic workflows that are live in production - not demos, not prototypes - and you want to do that at genuine scale, this is worth applying for. What you'll be working on as an AI Engineer - LLM & Agentic Systems: Designing and building AI agents and agentic workflows - combining LLMs, RAG, reasoning, memory and tool orchestration to solve real,...

Job Information Date Opened 05/06/2026 Job Type Contract Industry Government & Public Sector Work Experience 5+ years Salary 400-500 Outside IR35 City London Province Greater London Country United Kingdom Postal Code EC1A Job Description About Scrumconnect Consulting Scrumconnect Consulting is a leading UK digital and technology consultancy specialising in delivering impactful, user-centred services for the public sector. We work with central government departments, agencies, and local authorities to build digital products and services that improve outcomes for citizens. With a team of engineers, architects, delivery professionals, and designers, we bring deep expertise in digital transformation, cloud technologies, and agile delivery. We are committed to engineering excellence, innovation, and creating an inclusive environment where everyone can thrive. Role Overview We are seeking an experienced Azure...

Senior AI Engineer | Agentic AI / Machine Learning | UK | Contract Conquer AI is hiring Senior AI Engineers. We’re looking for hands-on engineers who can build practical AI systems that solve real operational problems for enterprise clients. This is an Outside IR35 contract working on greenfield AI projects for enterprise clients. This is not a pure ML research role. It’s applied, product-minded, and delivery-focused. You’ll work on systems that extract information, reason over documents, automate workflows, connect into business tools, support decision-making, and reduce manual effort for client teams. You’ll be close to clients, close to the problem, and close to the code. What you’ll work on: Agentic AI workflows LLM-powered applications RAG and knowledge systems Document extraction and classification Workflow automation API and system integrations Client-facing prototypes and production builds Internal Conquer AI product development What...

Senior Software Engineer | :£950/day Outside IR35 | 6-months initial contract | London Industry: Trading Location: London - 4 days in office Job Type: Contract - 6-month initial Our client is building the next generation of execution and order management (EMS/OMS) infrastructure for global commodities trading. You will join a small, fast-moving engineering team designing and delivering a cloud-native trading platform from the ground up, tightly integrated with risk, pricing, and market data systems to support high-performance trading environments. This is a senior, hands-on role for an engineer who wants end-to-end ownership — from low-level protocol integration through to distributed execution services and trader-facing tooling. What You’ll Be Doing Design and build core components of a modern EMS/OMS platform, including order routing, order lifecycle management, venue connectivity, and execution analytics. Implement and maintain FIX protocol and...