Senior Data Engineer at Primus Connect Ltd, United Kingdom, £Contract Rate

Databricks Platform Buildout

Senior Data Engineer

Outside IR35

Fully Remote

£500 - £575 per day

Overview & Objective

We are seeking 3x Senior Data Engineers for a Databricks Platform Buildout

project which is part of a bigger engagement.

The main project focus is to review the group-level Databricks platform foundations to confirm readiness for replication and then rollout across country-level platforms. These Engineers will participate in the rollout and replication phase.

Requirements

• 4+ Years Data Engineering Experience,
• 5+ years of consulting experience
• Minimum 2-3 projects delivered with hands-on experience in Databricks
• Experience in Spark and/or Hadoop,
• Familiarity with Databricks multi-hop pipeline architecture

Technical Stack and Required Capabilities

• Python, SQL
• Strong Databricks knowledge (PySpark, SQL, Delta Lake, Unity Catalog)
• Azure
• ETL/ELT development and data pipeline orchestration (eg, Databricks Workflows, DLT, Airflow, ADF, Glue, and Step Functions.)
• Github (creating branches, pull requests, reviewing PRs, solving merge conflicts)
• Apache Spark
• CI/CD (Azure DevOps/Terraform/Bicep