Senior Data Engineer at Primus Connect Ltd, United Kingdom, £Contract Rate

Contract Description

Databricks Platform Buildout

Senior Data Engineer

Outside IR35

Fully Remote

£500 - £575 per day

Overview & Objective

We are seeking 3x Senior Data Engineers for a Databricks Platform Buildout

project which is part of a bigger engagement.

The main project focus is to review the group-level Databricks platform foundations to confirm readiness for replication and then rollout across country-level platforms. These Engineers will participate in the rollout and replication phase.

Requirements

  • 4+ Years Data Engineering Experience,
  • 5+ years of consulting experience
  • Minimum 2-3 projects delivered with hands-on experience in Databricks
  • Experience in Spark and/or Hadoop,
  • Familiarity with Databricks multi-hop pipeline architecture

Technical Stack and Required Capabilities

  • Python, SQL
  • Strong Databricks knowledge (PySpark, SQL, Delta Lake, Unity Catalog)
  • Azure
  • ETL/ELT development and data pipeline orchestration (eg, Databricks Workflows, DLT, Airflow, ADF, Glue, and Step Functions.)
  • Github (creating branches, pull requests, reviewing PRs, solving merge conflicts)
  • Apache Spark
  • CI/CD (Azure DevOps/Terraform/Bicep