Applied Machine Learning & Data Engineering Senior Consultant II

About the role

This role supports high-impact AI and data automation initiatives focused on improving data ingestion, transformation, and mapping processes across the organization. This position is responsible for advancing AI-assisted data mapping capabilities to reduce manual effort, enhance accuracy, and deliver scalable, maintainable solutions for development teams. Initial responsibilities include building and optimizing intelligent import tools that leverage AI/ML techniques to parse, normalize, and validate structured and unstructured data from various sources. Core deliverables involve developing mapping logic, identifying data discrepancies, creating standardized ingestion templates, and producing load-ready outputs for downstream systems. The role also contributes to analytics and dashboarding efforts while supporting source-specific data rules and configurations. Over time, the scope will expand to broader AI, data-layer, and automation initiatives aligned with evolving business priorities, with a strong emphasis on delivering solutions that improve operational efficiency and overall data quality in high-visibility workflows. Key objectives: Reduce manual mapping effort, improve data accuracy and processing efficiency, deliver scalable, maintainable AI-driven pipelines

Responsibilities

• Design and build AI-powered data pipelines to process and transform structured and unstructured data (PDF, Excel, CSV, etc.)
• Develop and optimize LLM-based solutions (prompt engineering, mapping, extraction, and transformation)
• Translate complex data and technical outputs into actionable insights for business stakeholders
• Partner with cross-functional teams (product, engineering, business SMEs) to define and deliver solutions
• Validate and improve solution accuracy through testing, iteration, and stakeholder feedback
• Support integration of ML/AI solutions into enterprise systems and workflows (APIs, databases, cloud platforms)
• Document solutions to ensure scalability, maintainability, and long-term adoption
• How your day to day looks:
• Analyze and process carrier and CMS datasets
• Build AI/ML mapping logic using LLMs and prompts
• Develop data pipelines (Python/C# data processing logic, SQL, APIs, Azure)
• Validate outputs against expected results with business stakeholders
• Troubleshoot accuracy, performance, and edge cases
• Document pipelines and ensure production-readiness
• Support integration into enterprise systems (C#, APIs, Azure)
• Work Mix (exact mix will shift by project phase)
• 40% ML/prompt and mapping logic
• 35% data engineering / pipeline implementation
• 25% stakeholder collaboration and validation
• Required Qualifications
• 3+ years of related experience
• Strong proficiency in Python and SQL for data analysis and engineering
• Experience working with Generative AI / LLMs (prompt engineering, APIs, or similar)
• Hands-on experience with data transformation, file parsing, and pipeline development
• Knowledge of DevOps practices (CI/CD pipelines, version control, cloud deployment)
• Familiarity with cloud platforms (preferably Azure)
• Ability to communicate effectively with both technical and non-technical stakeholders
• Proven ability to work in ambiguous, fast-evolving environments

Requirements

• Experience with C# / .NET integration or enterprise application development
• Familiarity with LLM frameworks (LangChain, etc.) or document processing tools
• Knowledge of API design, JSON processing, and system integration patterns
• Background in insurance, healthcare, or other regulated industries

Benefits

Additional Information

National General is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers' evolving needs. We offer home, auto and accident and health insurance, as well as other specialty niche insurance products, through a large network of independent insurance agents, as well as directly to consumers. Job Description Business Title: Applied Machine Learning Engineer Senior Consultant II

Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at Allstate? Share your experience