Skip to main content
Overview: In this role you will design and optimize data pipelines powering AI-driven accounting agents, collaborating with cross-functional teams to scale data infrastructure. You will lead the development of high-performance PySpark ETL pipelines and manage data integrations from client sources and internal connectors. Your work supports reliable, scalable data and ML workflows, with governance and security baked into every stage. This is a hands-on leadership role at a company renowned for finance automation and cutting-edge AI. Compensation / Benefits Salary range provided Incentive programs Benefits and wellness plans Hybrid work with office in Pleasanton/New York/Los Angeles Professional development and inclusive culture
Responsibilities: Lead data pipeline development via PySpark ETL with a focus on quality and performance Manage integrations to client data sources using APIs and tools like FiveTran, Plaid, and BlackLine's internal connector ecosystem Ensure reliability through monitoring, automated testing, and data accuracy validation Optimize for scale with CDC, indexing strategies, and large datasets Collaborate with business stakeholders to refine data requirements and integrate AI and big data technologies
Key requirements: Extensive practical experience delivering business solutions affecting large parts of the organization Proficiency in Python, Java, or Scala Experience with ML frameworks (TensorFlow, PyTorch, scikit-learn) and orchestration tools (Airflow, Kubeflow, Vertex AI, MLflow) Production experience with ML/LLM pipelines across cloud providers (GCP, AWS, Azure) Familiarity with LangChain, LangGraph, ADK or similar agentic runtimes Strong CI/CD, IaC, and DevSecOps practices Observability stacks (Prometheus, Grafana, New Relic) for model/agent performance Containerization (Docker, Kubernetes) Automation scripting (Bash, Python) and workflow orchestration tools Cloud infrastructure management expertise Security, governance, and compliant AI/ML practices Leadership and mentorship Cross-functional collaboration Problem-solving and strategic thinking PySpark ETL APIs and integrations (FiveTran, Plaid) ML frameworks (TensorFlow, PyTorch, scikit-learn)

Also on the board Same function, level within a rung

Level

Lead

Location

Brooklyn, NY

Occupation

Software Developers

Industry

Computer Systems Design Services

Posted

today

Apply for this role (opens in a new tab)