Location: United States
Workplace: Remote
Employment Type: Full-time
Experience: Production LLM systems, end-to-end software development, and production system ownership
Core Areas: Software Engineering, Large Language Models (LLMs), AI Extraction Pipelines, Evaluation Infrastructure, Deterministic Rule Logic, Document Processing, Data Validation, Internal Review Tools
Compensation: Estimated base salary of $180,000-$230,000 annually, plus equity
About the Role
This Principal Software Engineer, AI position focuses on designing, building, and scaling production software systems that combine large language models (LLMs) with deterministic processing, structured validation, and human review. The engineer will help develop an AI-enabled audit workbench that transforms raw evidence into accurate, traceable, and reviewable workpapers through a pipeline-based architecture.
The role involves building document ingestion and extraction pipelines, recomputation and exception detection logic, workpaper generation, evaluation infrastructure, and frontend review interfaces. A central responsibility is determining which tasks require AI models and which must use deterministic code to ensure accuracy, reproducibility, and regulatory compliance. As an early engineering hire, this individual will help establish technical direction and scale the platform from approximately 100 clients to 10,000 while partnering with CPAs on audit requirements and review standards.
What You'll Do
- Design, develop, and scale an AI-enabled audit workbench that processes engagement evidence into structured, reviewable audit workpapers.
- Build and maintain document ingestion and information extraction pipelines for processing raw financial and compliance-related evidence.
- Develop deterministic recomputation and exception detection logic to validate extracted information and identify discrepancies.
- Implement workpaper generation capabilities that produce consistent, traceable outputs for professional review.
- Design and maintain evaluation infrastructure for production LLM systems, including golden datasets, regression testing, and measurable quality assessments.
- Develop frontend review interfaces that enable structured human validation, review, and approval of system-generated work.
- Establish reliable boundaries between stochastic AI model outputs and deterministic software logic, using exact code for calculations and regulatory rules.
- Apply language models to information extraction and narrative drafting while maintaining deterministic controls over numerical accuracy and compliance requirements.
- Design data schemas, validation mechanisms, and traceability processes that support auditable and reproducible system behavior.
- Evaluate AI extraction quality, identify regressions, and improve system reliability before changes reach production users.
- Take ownership of software systems throughout architecture, implementation, deployment, and ongoing production operation.
- Help establish engineering architecture, technical standards, and development practices as the platform scales to support a significantly larger client base.
- Partner directly with founding leadership and CPAs to translate audit requirements, regulatory logic, and review standards into reliable software functionality.
- Maintain strong engineering practices focused on correctness, maintainability, validation, and production reliability.
Qualifications
Required Qualifications
- Demonstrated experience developing and deploying production large language model (LLM) systems where output accuracy and correctness are critical.
- Hands-on experience with AI extraction pipelines, agentic workflows, document-understanding systems, or comparable production LLM applications.
- Strong software engineering capabilities, including the ability to design, implement, deploy, and operate complex systems independently.
- Experience developing evaluation frameworks for AI systems, maintaining golden datasets, and detecting regressions before they affect users.
- Ability to measure and assess model output quality through systematic evaluations rather than relying solely on subjective results.
- Sound technical judgment in selecting between probabilistic AI models and deterministic functions, particularly for financial calculations, compliance requirements, and other accuracy-sensitive operations.
- Understanding of reliable system design, data schemas, structured validation, and traceability in production software.
- Proven ability to build new software products and systems from initial concept through production deployment in a 0-to-1 engineering environment.
- Comfort working as an early engineering hire with substantial responsibility for technical architecture, system ownership, and development practices.
- Commitment to engineering fundamentals, maintainable code, testing, and dependable production operations.
Preferred Qualifications
- Experience building AI or software systems in high-stakes or regulated industries such as financial technology, legal services, healthcare, or regulatory technology.
- Familiarity with accounting, financial audits, the Employee Retirement Income Security Act (ERISA), 401(k) plan administration, or financial document processing. Previous audit experience is not required.
- Experience with layout-aware document extraction, optical character recognition (OCR), or related document-processing technologies.
- Familiarity with information retrieval systems and structured-output generation.
- Experience creating internal software tools, frontend review interfaces, or review-and-approval workflows.