Technical Director, Data Engineering

About the role

Diligent is the AI leader in governance, risk and compliance (GRC) SaaS solutions, helping more than 1 million users and 700,000 board members to clarify risk and elevate governance. The Diligent One Platform gives practitioners, the C-Suite and the board a consolidated view of their entire GRC practice so they can more effectively manage risk, build greater resilience and make better decisions, faster. At Diligent, we're building the future with people who think boldly and move fast. Whether you're designing systems that leverage large language models or part of a team reimaging workflows with AI, you'll help us unlock entirely new ways of working and thinking. Curiosity is in our DNA, we

Responsibilities

• Design and build scalable data platforms and distributed processing systems
• Develop modern ingestion, transformation, and retrieval pipelines for structured and unstructured data
• Build systems supporting AI/LLM applications, semantic search, RAG pipelines, vector search, and agentic workflows
• Work hands-on with engineering teams to implement production-grade solutions rather than producing slideware
• Evaluate and standardize frameworks, tooling, and infrastructure patterns across teams
• Improve performance, reliability, observability, and cost efficiency of data systems
• Partner with product and platform engineering teams to accelerate delivery of AI-native capabilities
• Drive pragmatic engineering decisions balancing speed, maintainability, and operational simplicity
• Mentor engineers technically through design reviews, architecture guidance, and pair debugging
• Help establish engineering standards around CI/CD, testing, data quality, monitoring, and operational excellence

Requirements

• Strong Hands-On Engineering Experience
• Candidates should have significant real-world experience building and operating production systems using many of the following:
• Data & Distributed Systems
• Airflow
• Elasticsearch / OpenSearch
• Vector databases and semantic retrieval systems
• MongoDB, PostgreSQL, DynamoDB, or similar platforms
• Cloud & Infrastructure
• AWS
• AWS CDK
• Serverless architectures
• Distributed observability and monitoring stacks
• AI / Search / Modern Data Applications
• LLM integration patterns
• RAG architectures
• Embeddings and vector search
• MCP servers and AI orchestration frameworks
• LangChain, LlamaIndex, DSPy, or similar ecosystems
• AI evaluation, tracing, and observability tooling
• Search relevance and ranking systems
• Backend Engineering
• Python strongly preferred
• Experience with Java, Go, or TypeScript is a plus
• API design and distributed service architectures
• Event-driven and asynchronous systems
• The Right Candidate
• Still enjoys building and debugging systems directly
• Has strong technical depth, not just architectural vocabulary
• Comfortable operating in ambiguity and fast-moving environments
• Understands how to simplify systems instead of endlessly abstracting them
• Has experience modernizing legacy platforms and evolving architectures incrementally
• Can distinguish between engineering fundamentals and hype cycles
• Values shipping working systems over theoretical perfection
• What Success Looks Like
• Engineering teams can move faster because the underlying platforms are reliable and scalable
• AI and data systems become production-grade rather than experimental prototypes
• Infrastructure costs and operational complexity are reduced through better architecture
• Search, ingestion, and retrieval systems improve significantly in performance and relevance
• Teams adopt consistent, maintainable technical patterns without excessive bureaucracy
• Experience with large-scale news, document, or regulatory data pipelines
• Experience with search relevancy tuning and semantic retrieval
• Exposure to governance, compliance, risk, or enterprise SaaS platforms
• Familiarity with modern AI engineering workflows and agentic systems
• Experience supporting both startup-speed execution and enterprise-scale operations
• Pay Range
• $180,000 - $210,000 CAD

Additional Information

Diligent is looking for a Technical Director of Data Engineering to help design and build the next generation of data and AI infrastructure powering our products and internal platforms. This is not a traditional management role. We are looking for an experienced builder; someone who still enjoys writing code, debugging distributed systems, evaluating frameworks, and getting hands-on with architecture and implementation. You will work across large-scale ingestion pipelines, search systems, AI/LLM infrastructure, event-driven architectures, vector databases, analytics platforms, and real-time data processing. You should be equally comfortable discussing high-level architecture with senior leadership and diving into a failing Kubernetes pod or optimizing a Spark job. The ideal candidate has strong opinions informed by real-world experience, understands tradeoffs deeply, and can move quickly without creating unnecessary complexity.

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at diligentcorporation? Share your experience