54 aws data engineer jobs found

Remote (UK based candidates only) Initial 6-Month Contract Outside IR35 Possibility for extension We're working with a consulting client who's partnering with a leading financial services organisation looking to hire an AWS Data Engineer to help build and enhance their cloud-based data platform. Key Skills Required Strong AWS data engineering experience Hands-on expertise with DBT and PySpark Proven Python development skills Experience with Apache Airflow for orchestration Strong knowledge of Amazon Redshift Experience with Apache Iceberg and modern data lake architectures Familiarity with OpenSearch for search and analytics solutions Exposure to Knowledge Graph technologies and semantic data modelling CI/CD experience using CircleCI Responsibilities Design and develop scalable data pipelines and ETL processes Build and optimise cloud-native data solutions on AWS Develop data models and transformations using DBT and PySpark...

AWS Data Engineer - Migration We're seeking a strong & hands-on AWS Data Engineering contractor, ASAP start, for an urgent replacement contract Rate: Up to £500-525 per day, Outside IR35 Working Patterns: Remote-first, London (up to 1 day p/w on-site). Contract: 3-6 months (likley extension) Sector: Financial Services About: We're supporting a major cloud transformation programme focused on migrating a large-scale Legacy ETL estate consiting of several thousand batch jobs from Ab Initio to a modern, serverless AWS data platform. This is a hands-on Senior Data Engineer role where you'll play a key part in defining migration patterns and delivering production-ready AWS Glue solutions. Person Requirments: Strong AWS Data Engineer - ideally AWS Certified. Idelaly, you'll bring proven experience delivering Ab Initio AWS Glue (PySpark) migration programmes Strong hands-on AWS Data Engineering expertise, particularly AWS Glue , S3 , Athena and...

AWS Data Engineer Contract: 6 Months Rate: £400 per day IR35 Status: Outside IR35 Location: Remote/Hybrid Overview We are seeking an experienced AWS Data Engineer to join a growing consultancy on an initial 6-month contract. You'll be responsible for designing, building, and optimising modern data platforms and pipelines, enabling analytics, reporting, and AI-driven initiatives across the business. This role is ideal for someone with strong AWS experience and a background working with modern data tooling including dbt, PySpark, Airflow, Redshift, Iceberg, OpenSearch, Knowledge Graphs, Python, and CircleCI . Key Responsibilities Design and develop scalable ETL/ELT data pipelines using Python and PySpark Build and maintain data transformation frameworks using dbt Develop and orchestrate workflows using Apache Airflow Deliver high-performance data warehouse solutions within Amazon Redshift Support the implementation and optimisation of...

URGENT | AWS Data Engineer - Migration We're seeking a strong & hands-on AWS Data Engineering contractor, ASAP start, for an urgent replacement contract Rate: Up to £500-525 per day, Outside IR35 Working Patterns: Remote-first, London (up to 1 day p/w on-site). Contract: 3-6 months (likely extension) Sector: Financial Services About: We're supporting a major cloud transformation programme focused on migrating a large-scale legacy ETL estate consisting of several thousand batch jobs from Ab Initio to a modern, serverless AWS data platform. This is a hands-on Senior Data Engineer role where you'll play a key part in defining migration patterns and delivering production-ready AWS Glue solutions. Person Requirements: Strong AWS Data Engineer - ideally AWS Certified. Ideally, you'll bring proven experience delivering Ab Initio → AWS Glue (PySpark) migration programmes Strong hands-on AWS Data Engineering expertise, particularly AWS Glue ,...

AWS Data Engineer - MigrationWe're seeking a strong & hands-on AWS Data Engineering contractor, ASAP start, for an urgent replacement contract  Rate: Up to £500-525 per day, Outside IR35  Working Patterns: Remote-first, London (up to 1 day p/w on-site).  Contract: 3-6 months (likley extension) Sector: Financial Services About:  We're supporting a major cloud transformation programme focused on migrating a large-scale legacy ETL estate consiting of several thousand batch jobs from Ab Initio to a modern, serverless AWS data platform. This is a hands-on Senior Data Engineer role where you'll play a key part in defining migration patterns and delivering production-ready AWS Glue solutions.  Person Requirments: Strong AWS Data Engineer - ideally AWS Certified.  Idelaly, you'll bring proven experience delivering Ab Initio → AWS Glue (PySpark) migration programmes Strong hands-on AWS Data Engineering expertise, particularly AWS Glue, S3, Athena and serverless data platforms....

AWS Data Engineer - Migration We're seeking a strong & hands-on AWS Data Engineering contractor, ASAP start, for an urgent replacement contract Rate: Up to £500-525 per day, Outside IR35 Working Patterns: Remote-first, London (up to 1 day p/w on-site). Contract: 3-6 months (likely extension) Sector: Financial Services About: We're supporting a major cloud transformation programme focused on migrating a large-scale Legacy ETL estate consiting of several thousand batch jobs from Ab Initio to a modern, serverless AWS data platform. This is a hands-on Senior Data Engineer role where you'll play a key part in defining migration patterns and delivering production-ready AWS Glue solutions. Person Requirments: Strong AWS Data Engineer - ideally AWS Certified. Idelaly, you'll bring proven experience delivering Ab Initio AWS Glue (PySpark) migration programmes Strong hands-on AWS Data Engineering expertise, particularly AWS Glue , S3 , Athena and...

AWS Data Engineer - Migration We 're seeking a strong & hands-on AWS Data Engineering contractor, ASAP start, for an urgent replacement contract Rate: Up to £500-525 per day, Outside IR35 Working Patterns: Remote-first, London (up to 1 day p/w on-site). Contract: 3-6 months (likley extension) Sector: Financial Services About: We 're supporting a major cloud transformation programme focused on migrating a large-scale legacy ETL estate consiting of several thousand batch jobs from Ab Initio to a modern, serverless AWS data platform. This is a hands-on Senior Data Engineer role where you 'll play a key part in defining migration patterns and delivering production-ready AWS Glue solutions. Person Requirments: Strong AWS Data Engineer - ideally AWS Certified. Idelaly, you 'll bring proven experience delivering Ab Initio AWS Glue (PySpark) migration programmes Strong hands-on AWS Data Engineering expertise, particularly AWS...

AWS Data Engineer Contract: 6 months Rate: £400 per day IR35 Status: Outside IR35 Location: Remote/Hybrid Overview We are seeking an experienced AWS Data Engineer to join a growing consultancy on an initial 6-month contract. You'll be responsible for designing, building, and optimising modern data platforms and pipelines, enabling analytics, reporting, and AI-driven initiatives across the business. This role is ideal for someone with strong AWS experience and a background working with modern data tooling including dbt, PySpark, Airflow, Redshift, Iceberg, OpenSearch, Knowledge Graphs, Python, and CircleCI . Key Responsibilities Design and develop scalable ETL/ELT data pipelines using Python and PySpark Build and maintain data transformation frameworks using dbt Develop and orchestrate workflows using Apache Airflow Deliver high-performance data warehouse solutions within Amazon Redshift Support the implementation and optimisation of Apache Iceberg...

Remote (UK based candidates only) | Initial 6-Month Contract | Outside IR35 | Possibility for extension We 're working with a consulting client who 's partnering with a leading financial services organisation looking to hire an AWS Data Engineer to help build and enhance their cloud-based data platform. Key Skills Required Strong AWS data engineering experience Hands-on expertise with DBT and PySpark Proven Python development skills Experience with Apache Airflow for orchestration Strong knowledge of Amazon Redshift Experience with Apache Iceberg and modern data lake architectures Familiarity with OpenSearch for search and analytics solutions Exposure to Knowledge Graph technologies and semantic data modelling CI/CD experience using CircleCI Responsibilities Design and develop scalable data pipelines and ETL processes Build and optimise cloud-native data solutions on AWS Develop data...

AWS Data Engineer Contract: 6 Months Rate: £400 per day IR35 Status: Outside IR35 Location: Remote / Hybrid Overview We are seeking an experienced AWS Data Engineer to join a growing consultancy on an initial 6-month contract. You 'll be responsible for designing, building, and optimising modern data platforms and pipelines, enabling analytics, reporting, and AI-driven initiatives across the business. This role is ideal for someone with strong AWS experience and a background working with modern data tooling including dbt, PySpark, Airflow, Redshift, Iceberg, OpenSearch, Knowledge Graphs, Python, and CircleCI . Key Responsibilities Design and develop scalable ETL/ELT data pipelines using Python and PySpark Build and maintain data transformation frameworks using dbt Develop and orchestrate workflows using Apache Airflow Deliver high-performance data warehouse solutions within Amazon Redshift Support...

Remote (UK based candidates only) | Initial 6-Month Contract | Outside IR35 | Possibility for extension We're working with a consulting client who's partnering with a leading financial services organisation looking to hire an AWS Data Engineer to help build and enhance their cloud-based data platform. Key Skills Required Strong AWS data engineering experience Hands-on expertise with DBT and PySpark Proven Python development skills Experience with Apache Airflow for orchestration Strong knowledge of Amazon Redshift Experience with Apache Iceberg and modern data lake architectures Familiarity with OpenSearch for search and analytics solutions Exposure to Knowledge Graph technologies and semantic data modelling CI/CD experience using CircleCI Responsibilities Design and develop scalable data pipelines and ETL processes Build and optimise cloud-native data solutions on AWS Develop data models and transformations using DBT and PySpark Automate...

Senior Data Engineer – AWS / Lakehouse Contract: 3 months | Fully Remote | Outside IR35 We are seeking an experienced Data Engineer for a three-month project delivering cloud-based data engineering and integration solutions. This is a hands-on role requiring strong AWS expertise , practical experience with Apache Iceberg and Apache Spark , and a solid understanding of unstructured data processing and integration patterns . You will work closely with technical and delivery teams to build reliable data pipelines, develop lakehouse capabilities and integrate data across systems. Key responsibilities Design, build and optimise data pipelines using AWS Glue, S3, Lambda and Apache Spark . Develop and maintain lakehouse solutions using Apache Iceberg , supporting efficient data ingestion, transformation and access. Build workflows for ingesting, processing and transforming unstructured data . Develop integrations using API Gateway, Lambda and...

Senior Data Engineer - AWS/Lakehouse Contract: 3 months | Fully Remote | Outside IR35 We are seeking an experienced Data Engineer for a three-month project delivering cloud-based data engineering and integration solutions. This is a hands-on role requiring strong AWS expertise , practical experience with Apache Iceberg and Apache Spark , and a solid understanding of unstructured data processing and integration patterns . You will work closely with technical and delivery teams to build reliable data pipelines, develop lakehouse capabilities and integrate data across systems. Key responsibilities Design, build and optimise data pipelines using AWS Glue, S3, Lambda and Apache Spark . Develop and maintain lakehouse solutions using Apache Iceberg , supporting efficient data ingestion, transformation and access. Build workflows for ingesting, processing and transforming unstructured data . Develop integrations using API Gateway, Lambda and EventBridge ,...

Senior Contract Data Engineer (6-month contract) | Amazon Bedrock Project Location: London, FULLY REMOTE! Contract Duration : 6 months, initial (extension is possible, but hasn 't been confirmed) Day Rate : £575-600 per day, Outside IR35 Sector: Entertainment / Digital I have an exciting new contract opportunity for a Senior AWS Data Engineer to play a key role in developing a Gen AI platform which will leverage Amazon Bedrock AgentCore. The successful candidate will be responsible for designing and implementing natural language-to-SQL capabilities, which will enable the business to query complex data using conversational AI to receive accurate summaries, supporting developers & Senior Stakeholders. Responsibilities: - Design & implement scalable data solutions which fully support Amazon Bedrock AgentCore applications - Develop and optimise NL to SQL workflows across the enterprise data platform - Create AI-driven...

Senior Python Engineer – AI & Microservices £450 PD to £600 PD (Outside IR35) | 12 month Contract | 2 days onsite in London (Hybrid) | ASAP Start Our trusted technology partner is developing innovative products and is looking for an experienced Senior Python Engineer / Back End Developer to join a fast-moving team. You will develop Python microservices, scalable services, voice chat, and data pipelines, using AI technologies including ASR, TTS, and LLMs. The ideal candidate will have strong Python skills, a pragmatic delivery-focused approach and experience working within either a SaaS, Digital-Rich, Assessment, Subs cription or Ed-tech environment.  Key Responsibilities: Develop and maintain scalable Python services, microservices, voice chat, and data pipelines. Work with the Tech Lead and wider team on architecture, technical decisions, and solution design. Collaborate with product, data science, QA, design, and subject matter experts...

We are seeking 3 experienced SC Cleared Data Engineers with strong expertise in AWS Glue to work for our client in the defence sector. You will support the design, development, and optimisation of cloud-based data integration and analytics solutions. You will play a key role in building scalable ETL pipelines, data lakes, and data processing frameworks within AWS, enabling the organisation to deliver high-quality data products and insights. We are looking for people with extensive commercial experience with AWS Glue, developing Glue Jobs using PySpark and Python, AWS Glue Data Catalog, Crawlers, Workflows and triggers, Amazon S3 data lake implementation and management, AWS IAM roles, policies and security best practice. You will also need experience with several of the following: Amazon Redshift, AWS Lambda, Amazon Athena, Amazon RDS, Amazon EMR, Amazon EventBridge, Amazon Kinesis, AWS Step Functions, Amazon SQS/SNS and AWS CloudWatch. These roles are outside IR35 and...

Senior Python Engineer - AI & Microservices £450 PD to £600 PD (Outside IR35) | 12 month Contract | 2 days onsite in London (Hybrid) | ASAP Start Our trusted technology partner is developing innovative products and is looking for an experienced Senior Python Engineer/Back End Developer to join a fast-moving team. You will develop Python microservices, scalable services, voice chat, and data pipelines, using AI technologies including ASR, TTS, and LLMs. The ideal candidate will have strong Python skills, a pragmatic delivery-focused approach and experience working within either a SaaS, Digital-Rich, Assessment, Subscription or Ed-tech environment. Key Responsibilities: Develop and maintain scalable Python services, microservices, voice chat, and data pipelines. Work with the Tech Lead and wider team on architecture, technical decisions, and solution design. Collaborate with product, data science, QA, design, and subject matter experts to deliver new features....

Costello & Reyes Group Limited have been engaged by its client a global defence firm to identify a SailPoint Consultant/Engineer - Design & Delivery who holds current SC level Clearance. Role Overview We are seeking an experienced SC Cleared SailPoint Consultant to take a leading, hands-on role in the design, configuration, implementation and delivery of SailPoint Identity Governance and Administration (IGA) capabilities within a secure enterprise environment. This is not purely an architecture or advisory position. The successful consultant must be capable of taking requirements from initial discovery and solution design through to hands-on configuration, integration, testing, deployment and transition into operational service. The consultant will work closely with Security, IAM, Infrastructure, Application, Service Management and business stakeholders to translate identity and access management requirements into practical SailPoint solutions. The role...

ML ENGINEER - SC CLEARED - OUTSIDE IR35 - CENTRAL LONDON (1 DAY ONSITE) - 6 MONTH CONTRACT TechShack is recruiting an ML Engineer for a 6 month contract, outside IR35, supporting a UK Government department on an AI-enabled data and business systems programme. Central London, on site 1 day per week. This role requires active SC clearance. Candidates who do not meet this criteria cannot be considered. THE PROGRAMME The programme covers two key deliverables: a new MCP (Model Context Protocol) server enabling an internal AI assistant to securely query business data in natural language, and an AI-powered MVP automating the capture, summarisation and submission of business information from Outlook and Teams. KEY RESPONSIBILITIES - Design and implement a new MCP server using the organisation's existing data platform/API - Enable natural language querying of business datasets via an internal AI assistant - Integrate relevant datasets based on defined user and business requirements...

I'm working with a client who is scaling their Databricks platform to support AI and GenAI use cases, and they need people who can make sure it's governed properly from the start. This is about hands-on Databricks delivery, not just certifications. This is a remote first contract but you must be based in the UK. This is outside IR35. What they're building Unity Catalog as the single governance layer across data, ML models and AI assets Governed GenAI and agent workloads using Mosaic AI and Agent Bricks Secure, auditable AI pipelines from ingestion to model serving, all within the Databricks platform Databricks project experience we're looking for Unity Catalog implementations or migrations from Hive metastore, including metastore design, catalog and schema structure, and workspace and environment separation Fine-grained access control in Unity Catalog: row filters, column masks, and attribute-based access control (ABAC) with governed tags Unity Catalog lineage...