Data Engineer - TWE46775 at twentyAI, City Of London, £700 per day

Data Engineer – Databricks, Azure, Python, SQL - 12 Month Contract - Outside IR35 - Up to £700 p/d (dependent on experience)

A well-established private equity firm with close to two decades' track record and preparing to launch its largest flagship fund to date is hiring a Data Engineer to build its first Databricks environment from the ground up.

This is a rare opportunity to own the technical delivery of a firm's core data platform end-to-end: partnering directly with stakeholders across Finance, Portfolio Operations, General Partners, and Compliance to unlock the data quality and visibility the business needs ahead of its next fund launch.

The Role

You'll design and build the firm's first Databricks instance, standing up a medallion architecture (bronze/silver/gold) on Azure and integrating it with the firm's existing portfolio administration and data intelligence platforms. The environment you build will become the firm's golden data source.

What We're Looking For

• Proven, hands-on delivery of Databricks / Delta Lake builds - ACID transactions, time travel, schema evolution, and performance tuning (OPTIMIZE, Z-ordering, VACUUM)
• Strong command of medallion architecture and PySpark, lazy evaluation, partitioning, broadcast joins, and diagnosing partition skew
• Solid Azure data platform experience; familiarity with Azure Analysis Services or a comparable semantic layer is a plus
• Confidence engaging senior, non-technical stakeholders and translating between technical and business needs
• Experience working within Private Equity is highly desirable - wider Financial Services is accepted

*Suitable candidates will be contacted within 24 hours