CloudZero
Posted 5mo ago

Distinguished Architect, Data Platform

CloudZero
San Francisco, California, United States
$275k-$330k/yrRemoteFull Time
Responsibilities
  • Define architecture
  • Drive streaming
  • Mentor engineers
Requirements
  • 10+ years in data engineering with senior architecture experience
  • Large-scale streaming and open table formats
  • Strong SQL and data modeling skills
Technical tools mentioned
KafkaKinesisFlinkSparkIcebergDelta LakeHudiTrinoPrestoSpark SQLDuckDBSnowflakeBigQueryRedshift

Job description

About the Role

CloudZero is growing fast. Our customer base is expanding, the data challenges we're solving are getting more complex, and the platform is scaling to match. As a Distinguished Architect on the Data Engineering team, you'll own some of the hardest infrastructure problems at CloudZero: shaping the next-generation streaming data platform, the dimensional cost model underlying every attribution decision, the hot/cold storage architecture serving both real-time and historical queries, and the query engine that powers our entire product.

This is real platform architecture work at real scale, not a consulting role or a review-and-advise job. You'll define the roadmap, drive the foundational decisions, and be a force multiplier for a talented engineering team — evolving CloudZero from batch-oriented pipelines toward a streaming-first architecture where cost attribution reaches engineers within seconds of a resource being used, not the next morning.

This role is ideal for an architect who has built systems like this before, has the scars to prove it, and wants to see their decisions matter in direct and measurable ways for customers and for the business.

What You'll Do

Define the Data Platform Architecture

  • Lead end-to-end technical design for CloudZero's next-generation data platform, from event ingestion and stream processing through hot/cold storage and the query layer to the API surface

  • Document architectural decisions, tradeoffs, and migration strategies with the rigor of an RFC-driven process

  • Shape and drive every layer of the new architecture: event ingestion, stream processing and enrichment, real-time serving, analytical storage, query layer, and API

Drive Streaming Infrastructure to Production

  • Design and deliver CloudZero's real-time data pipeline from ingestion through enrichment to serving

  • Establish SLOs for throughput, latency, and correctness, and build the operational playbooks that make this system trustworthy enough to replace the batch pipelines our entire product currently depends on

  • Tackle real-time streaming at scale across thousands of customers simultaneously, with fault tolerance, backpressure awareness, and correctness as non-negotiables

Tackle the Dimension Cardinality Problem

  • Redesign CloudZero's dimensional cost model to support high-cardinality, multi-dimensional cost attribution without runaway materialization costs

  • Drive incremental, delta-based materialization strategies using modern open table formats, dramatically reducing expensive full-rebuild jobs and unlocking millions in annual infrastructure savings

Evolve the Query Layer

  • Assess CloudZero's current query infrastructure, drive in-flight migrations to completion, and lead the evolution of the query engine layer going forward

  • Own performance optimization across partition pruning, predicate pushdown, and query planning, and set the vision for how the query layer grows as data volumes scale 10x

Extend Cost Attribution to Real-Time

  • Evolve CloudZero's proprietary cost attribution engine from a batch-oriented model to one that assigns complex cost dimensions by team, feature, and customer within seconds of resource usage

  • Rethink enrichment, data lineage, and correctness guarantees in a streaming context

Shape the Data Engineering Roadmap

  • Partner with product, infrastructure, and analytics engineering to define a multi-year data platform roadmap

  • Build consensus across engineering leadership on foundational investments including table formats, streaming frameworks, query engines, and schema management

Elevate the Engineering Team

  • Participate in architecture reviews, contribute to design patterns and best practices, and mentor senior and staff engineers through code review, pairing, and structured feedback

  • Make everyone around you better, not by directing, but by raising the collective craft

What You Bring

Data Platform & Architecture

  • 10+ years in data engineering with a clear trajectory toward principal or staff-level architecture

  • Built and operated large-scale data platforms serving tens of millions of events per day in production

  • Deep experience with streaming systems such as Kafka, Kinesis, Flink, or Spark Streaming at real production throughput

  • Strong hands-on fluency with modern open table formats including Apache Iceberg, Delta Lake, and Hudi, including compaction, partitioning strategy, and time-travel queries

  • Designed hot/cold storage architectures with explicit latency SLOs per tier

  • Proven ability to drive a data platform end to end, not just a single layer

Data Modeling & Dimensional Design

  • Expert in dimensional data modeling including fact/dimension schema design, slowly changing dimensions, and cardinality management

  • Deep understanding of the materialization tradeoff space: full vs. incremental, push vs. pull, pre-aggregate vs. query-time

  • Experience with cost attribution, showback/chargeback, or multi-tenant data partitioning patterns

  • Strong SQL and query optimization background across predicate pushdown, partition pruning, and cost-based query planning

Query Engines & Compute

  • Hands-on with distributed query engines such as Trino, Presto, Spark SQL, or DuckDB including configuration, optimization, and production operations

  • Understands catalog and metadata management and how it couples to query engines

  • Comfortable with cloud data warehouses such as Snowflake, BigQuery, and Redshift and how they integrate with open table formats

  • Experience driving query engine migrations while maintaining production SLAs

Engineering Leadership

  • Track record as a technical anchor for a data platform or data engineering team

  • Writes clear ADRs, RFCs, and technical design docs that bring engineers along

  • Can drive multi-month, multi-team technical initiatives from inception to production without heavy process overhead

  • Communicates complex tradeoffs to non-technical stakeholders including product and business leadership

  • Comfortable in a high-autonomy environment: builds consensus, influences through expertise, and helps teams move forward

Bonus If You Have...

  • FinOps or cloud cost domain experience

  • Multi-cloud data ingestion across AWS, Azure, and GCP

  • Apache Flink at production scale

  • Lakehouse architecture patterns

  • Real-time feature engineering for ML

  • Data mesh or domain-oriented design patterns

  • Prior startup or high-growth SaaS experience

  • Open source contributions to the data ecosystem

About CloudZero

Cloud cost management is one of the biggest challenges organizations face today. As cloud adoption continues to accelerate, so do the complexities and costs associated with it, and macroeconomic conditions only increase pressure to prove cloud efficiency.

CloudZero is a SaaS platform at the intersection of next-generation cloud cost management and FinOps. We ingest billing and usage data from all cloud, SaaS, and PaaS providers, organize it in real time according to our customers' business structures, and empower organizations to make more informed business decisions.

Since our founding in 2016, our mission has been to make efficient innovation a reality for every cloud-driven organization. We believe every engineering decision is a buying decision, and we're applying proven reliability engineering principles to financial efficiency.

We believe the best AI empowers users with clear insights and confident decisions, transforming complex cloud cost data into actionable intelligence that drives meaningful business outcomes.

To date, we've raised over $56 million from leading venture capital firms. We're solving problems of massive scale, business importance, and complexity in a space that needs it more than ever.

Equal Opportunity Employer

CloudZero is an equal opportunity employer and values diversity. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status or disability status. All job offers are contingent upon the candidate passing background and reference checks.

Please note: CloudZero is unable to sponsor employment visas. Candidates must have permanent authorization to work in the United States without the need for current or future sponsorship.

About CloudZero

Private SaaS platform helping finance, IT, and engineering teams optimize cloud and AI costs.

Year founded
2016
Employees
159
Organization type
Private
Latest investment
Raised $56.00M Series C (2025) — led by BlueCrest Capital Management, Innovius Capital
Headquarters
US

Similar jobs

Architect roles near San Francisco, California
3d
Save
Mark Applied
Hide
Principal Architect, Product Engineering
San Francisco, California, United States
$275k-$310k/yr HybridFull Time
Niantic Spatial
Niantic Spatial: Independent American geospatial AI building real-world foundation models for robots, AI agents, people, and public-sector operations.
15+ YOE15+ years of software engineering experience, large-scale systems architecture, distributed systems, cloud-native services, APIs, production coding, technical strategy, and cross-functional influence.
Google Cloud Platform, Amazon Web Services, REST APIs, TypeScript, Node.js, C++, Swift, Kotlin, PostgreSQL, Spanner, Identity and Access Management
4d
Save
Mark Applied
Hide
Architect – AI / Agentic Automation
San Jose, California, United States
$187k-$268k/yr HybridFull Time
Cisco
CiscoNASDAQ: CSCO: Global leader in networking, cybersecurity, and cloud-native technology solutions.
8+ YOEBachelor’s degree in STEM and 8+ years of software engineering experience designing complex backend, distributed, automation, or platform systems, with AI, cloud-native, Kubernetes, and multi-agent architecture expertise.
Python, Go, Rust, TypeScript, React, Node.js, .NET, PostgreSQL, Redis, Kafka, Kubernetes, Docker, CI/CD, MCP, A2A, LLMs
1w
Save
Mark Applied
Hide
Senior Architect, Agentic AI for Marketing
Santa Clara or California
$224k-$357k/yr HybridFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Computing platform for AI and accelerated graphics.
12+ YOEBS, MS, or PhD in a technical field or equivalent experience; 12+ years in software, AI/ML, architecture, data platforms, or production systems; Python, LLM, RAG, distributed systems, cloud, and agentic AI expertise.
Python, APIs, Linux, MCP, Kubernetes, Docker, CI/CD, NVIDIA NeMo Agent Toolkit, LangGraph, LlamaIndex, LangChain, CrewAI, Semantic Kernel, OpenAI Agents SDK, Google ADK, NIM, NeMo, NeMo Retriever, NeMo Guardrails, Nemotron, Triton, NVIDIA AI Enterprise
1w
Save
Mark Applied
Hide
Senior Architect, Agentic AI for Marketing
Santa Clara or California
$224k-$357k/yr HybridFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Computing platform for AI and accelerated graphics.
12+ YOEBS, MS, PhD, or equivalent experience; 12+ years in software, AI/ML, architecture, data platforms, or production systems; LLM, agentic AI, distributed systems, Python, and cloud infrastructure experience.
Python, APIs, Linux, RAG, MCP, Kubernetes, Docker, CI/CD, NVIDIA NeMo Agent Toolkit, LangGraph, LlamaIndex, LangChain, CrewAI, Semantic Kernel, OpenAI Agents SDK, Google ADK, NIM, NeMo, NeMo Retriever, NeMo Guardrails, Nemotron, Triton, NVIDIA AI Enterprise, GPU-enabled Kubernetes
1w
Save
Mark Applied
Hide
Senior Architect, Agentic AI for Marketing
Santa Clara or United States
$224k-$357k/yr OnsiteFull Time
NVIDIA
NVIDIANASDAQ: NVDA: Computing platform for AI and accelerated graphics.
12+ YOEBachelor's, master's, PhD, or equivalent experience; 12+ years in software, AI/ML, architecture, data platforms, or production systems; LLM, generative AI, RAG, agentic AI, Python, APIs, and distributed infrastructure experience.
Python, APIs, Linux, MCP, Kubernetes, Docker, CI/CD, NVIDIA NeMo Agent Toolkit, LangGraph, LlamaIndex, LangChain, CrewAI, Semantic Kernel, OpenAI Agents SDK, Google ADK, NIM, NeMo, NeMo Retriever, NeMo Guardrails, Nemotron, Triton, NVIDIA AI Enterprise
2w
Save
Mark Applied
Hide
Principal Architect — Security, Privacy and Governance in Large-scale Enterprise Agentic-AI Solutions
San Jose, California, United States
$190k-$361k/yr HybridFull Time
Adobe
AdobeNASDAQ: ADBE: Empowering everyone to create through innovative digital experiences.
20+ YOERequires 20+ years in software engineering or security architecture, expertise in distributed systems, access control, agentic AI, identity management, and authorization. Advanced technical degree preferred.
OAuth, OIDC, SAML, OPA, Rego, Cedar, Kafka, Spark
2w
Save
Mark Applied
Hide
Senior Architect
Pune or Hyderabad or Mumbai or Bangalore or Prague or Bucharest or San Jose or Warsaw or Mexico City or Buenos Aires or São Paulo
RemoteFull Time
Fortrea
FortreaNASDAQ: FTRE: Global contract research organization providing clinical development solutions.
10+ YOEBachelor's degree or equivalent experience, 10 years in enterprise architecture, Azure data solutions, data governance, MDM, metadata, cloud platforms, integrations, and IaC tools.
Microsoft Azure, AWS, Google Cloud, Terraform, Ansible, Puppet
2w
Save
Mark Applied
Hide
Architect: Building, Site & Sustainability Engineering Team
Menlo Park or United States
$170k-$234k/yr HybridFull Time
Meta
MetaNASDAQ: META: Builds technologies that help people connect, find communities, and grow businesses.
12+ YOERegistered Architect with 12+ years in mission-critical design, construction, or operations; large-scale project, data center lifecycle, project management, AI tooling, building systems, codes, and English communication experience required.
Google Suite, Revit/BIM, Bluebeam, Autodesk, Autodesk Construction Cloud (ACC), Unifier, Visio, Oracle P6, IBC, ASCE, SLiM