Skip to main content
Back to jobs

AI Engineer

External
agencywithin logoAgencywithin · Long Island City, NY
$91K–$254K/yrFull-timeOn-site2mo ago
AWSAzureBigQueryCross-functional CollaborationdbtExcel
Cover LetterConnect

Prepare for this interview

Elite

AI-generated questions, company research, and talking points tailored to this role


Requirements

  • Strong programming skills in Python with experience building production systems.
  • Experience building applications powered by LLM APIs (OpenAI, Anthropic, Vertex AI, etc.).
  • Deep understanding of:
  • Context windows and token limits
  • Multi-turn conversation behavior
  • Tool/function calling
  • Structured outputs (JSON schema)
  • Prompt design and failure modes
  • Experience integrating external APIs into production systems.
  • Proficiency in SQL.
  • Experience working with cloud platforms (GCP, AWS, or Azure).
  • Strong problem-solving skills and ability to debug complex system behavior.
  • Excellent communication and cross-functional collaboration skills.
  • Strong Plus
  • Experience designing evaluation frameworks for LLM-powered systems.
  • Understanding of embeddings, vector search, and RAG architectures.
  • Experience building agent or multi-agent workflows.
  • Experience with stateful execution environments or sandboxed code execution.
  • Experience with document automation (Google Docs, Slides, PDFs).
  • Experience building internal developer tools or productivity platforms.
  • TypeScript.
  • Required knowledge of:
  • Python
  • SQL
  • Cloud Platforms (GCP, AWS, Azure)
  • LLMs / AI APIs
  • Git / GitHub
  • Data Warehouses (BigQuery, Snowflake, Redshift)
  • Data Transformation (dbt)
  • Semantic Layers (Cube, Looker, dbt Metrics)
  • TypeScript
  • Our interview process includes, but is not limited to the following:
  • Excel and Typing Test
  • We offer a competitive salary and benefits based on ability level, including:
  • Unlimited vacation policy
  • Monthly Phone Stipend
  • Comprehensive Medical, Dental, and Vision insurance options
  • 401(K) plan with matching
  • Dog friendly office
  • Hybrid work opportunity
  • Professional Development Program
  • Bonus Perk - Seamless allowance
  • Total compensation based on education, experience, and skills level ($90,900-$254,100)
  • Level 1 - Possesses essential capabilities
  • $90,900-$123,540
  • Level 2 - Possesses developing capabilities
  • $123,540-$156,180
  • Level 3 - Possesses notable capabilities.
  • $156,180-$188,820
  • Level 4 - Possesses strong capabilities.
  • $188,820-$221,460
  • Level 5 - Possesses advanced capabilities.
  • $221,460-$254,100
  • About WITHIN

Benefits

Dental insuranceVision insurance401(k)Paid time offPerformance bonus

Additional Information

About the Role We are seeking an exceptional AI Engineer to join our growing team and take a leading role in building and evolving our internal AI-powered agent platform used across the organization. In this role, you will design, build, and improve AI-powered applications that integrate large language models (LLMs) with internal tools, SQL databases, advertising APIs (Google Ads, Meta), and stateful code execution environments. You will play a critical role in improving reliability, context orchestration, evaluation maturity, and expanding Rai into document and workflow automation use cases. Your work will directly impact 200-500 employees and potentially clients, making AI a core productivity multiplier across the organization. Responsibilities include but are not limited to: LLM Application Development Design, build, and maintain AI-powered applications leveraging LLM APIs, tool calling, and structured outputs. Agent & Tool Orchestration Develop and improve agent architectures that integrate tools such as SQL execution, Python code execution, and marketing platform APIs through MCP-style integrations. Context & Memory Management Design and refine multi-turn context handling, session memory strategies, and prompt structures to improve response reliability and reduce hallucinations. Reliability & Guardrails Implement safeguards for tool execution, SQL queries, and code execution. Improve retry logic, structured validation, and hallucination mitigation strategies. Evaluation & Quality Assurance Contribute to building structured evaluation frameworks for LLM-powered systems, including multi-turn testing, regression testing, and task-based success metrics. Product Expansion Support expansion of Rai into document generation, tables, slide decks, and potential Google Workspace integrations. Collaboration Work closely with Machine Learning Engineers, Product Managers, and Full-Stack Engineers to ensure AI systems are robust, maintainable, and aligned with business goals. Continuous Innovation Stay current with advances in LLM systems, agent architectures, and applied AI tooling to continuously improve Rai's capabilities.


Your Match

How well this role fits your profile.

Company Intel

What employees say

Worked at agencywithin? Share your experience

Interested in this role?

Apply on the company's website.

Cover LetterConnect