£600 - £750 per day
Outside Spy
London, UK
Data/AI Engineer (Databricks) - Outside of IR35Contract We're taking an AI decision-support product from proof-of-concept to full production on Databricks and need a hands-on engineer to build, run and own it. It analyses data across ~1,000 sites, uses an LLM (Anthropic Claude) to generate per-site recommendations, and includes a conversational agent that lets managers question their own data. Delivery role, not advisory - you'll fill our gap in Databricks engineering, LLM pipelines and agent development, and build the capability in-house. Initial 6 Month Contract (Ext) & Remote Working What you'll do Build ingestion pipelines (PySpark) into a governed medallion layer in Unity Catalog. Productionise the LLM pipeline - run Claude across the estate on a schedule, with retries and validation gates. Build and tune the conversational agent - grounding, guardrails, benchmarking. Own evaluation (prompt versioning, MLflow) and CI/CD across dev/UAT/prod. Essential...