Data Scientist/ML Engineer at Primus Connect Ltd, London, 4 Months, £500-£545 per day

Databricks + MLOps + Recommendation Systems + Entity Matching

Role: Data Scientist

Rate: £500-545 GBP per day

Outside IR35

4 months initial

London - Hybrid

Role Summary

Senior data scientist/ML engineer to design, build, and productionise ML capabilities on a Databricks platform buildout for an international events company. The role covers three interconnected workstreams: MLOps foundation, recommendation engine (exhibitors and sessions for event attendees), and single customer view (identity resolution across sparse conference registration data).

The role includes MLOps framework design and implementation, recommendation approach experimentation and delivery, tiered identity resolution (deterministic + fuzzy matching), a human-in-the-loop review workflow via Databricks Apps, plus documentation and knowledge transfer to the internal team.

This is a hands-on delivery role: responsible for both defining the ML approach and implementing it end-to-end. Sets ML standards along the way (MLOps patterns, feature engineering conventions, model lifecycle, matching approach) so the customer team can operate and extend what's delivered.

Seniority

Senior. Owns the ML workstreams end-to-end: design, experimentation, implementation, and productionisation. Works alongside the delivery team's architects and data engineers, but is the primary hands-on contributor for the ML deliverables.

Experience

• 5+ years in applied ML or data science with strong hands-on production delivery
• Prior end-to-end ownership of at least one ML delivery covering design, implementation, and productionisation
• Prior production experience with MLOps on Databricks (or a directly comparable platform)
• Prior production experience building recommendation systems or embedding-based retrieval
• Prior production experience with entity resolution/fuzzy matching pipelines
• Prior customer-facing or consulting experience, including working directly with customer teams and handing over what's been built
• Comfortable running knowledge transfer sessions and leaving a customer team capable of operating independently

Must-Have Technical SkillsMLOps on Databricks

Hands-on experience designing and operating end-to-end MLOps on Databricks: MLflow (tracking, model registry, promotion), Databricks Feature Store, model serving patterns, monitoring, drift detection, and feedback ingestion. Able to design MLOps patterns that are reusable across current and future models.

Recommendation Systems

Practical experience building recommendation or matching systems in production, including feature engineering, candidate selection, and evaluation. Familiarity with embedding-based similarity and vector search, ideally with Databricks Vector Search. Comfort experimenting with alternative approaches and selecting the best fit through hypothesis-driven iteration rather than defaulting to a single technique.

Entity Resolution/Fuzzy Matching

Prior implementation of tiered matching pipelines (deterministic + fuzzy) in a production setting. Comfort with string similarity techniques (Jaro-Winkler, Levenshtein, Soundex, or equivalent) and blocking strategies for scale. Realistic judgment about achievable match quality given sparse, inconsistent source data. Able to design for meaningful gains without overpromising.

Python and SQL

Strong hands-on Python (PySpark, pandas, ML libraries) and SQL. Able to build production ML pipelines that run reliably on Databricks.

Databricks and Delta Lake

Working experience with Delta Lake, Unity Catalog, and Lakeflow Jobs. Able to reason about ML workload cost and performance: cluster sizing, serverless vs classic compute, feature computation cadence, and inference patterns (batch vs online).

CI/CD for ML

Experience with Git-based CI/CD for ML pipelines: automated training runs, model validation, and promotion between environments. Familiarity with Declarative Automation Bundles, GitHub Actions or equivalent.

Databricks Apps (nice-to-have but valued)

Experience building lightweight UIs on Databricks Apps for human-in-the-loop workflows.

Delivery Responsibilities

• Designs and implements the MLOps framework end-to-end: model tracking, versioning, promotion, serving, monitoring, and feedback ingestion
• Designs, experiments with, and implements the recommendation approach for exhibitors and sessions based on attendee preferences
• Designs and implements the tiered matching pipeline (deterministic + fuzzy) for the single customer view, including feature engineering and the feedback loop from human review
• Sets ML standards along the way (MLOps patterns, feature engineering conventions, model lifecycle, matching approach) that the customer team can follow
• Owns documentation quality: MLOps reference architecture, feature specifications, matching rules, model runbooks
• Runs knowledge transfer sessions that leave the internal team capable of operating ML pipelines, extending models, and iterating on matching logic without external help