BuildFlow Framework

BuildFlow is an open source framework that lets you build your entire backend system in minutes using Python. No configuration required.

 1  
 2  from typing import Any, Dict
 3  
 4  from buildflow import Flow
 5  from buildflow.io.portable import AnalysisTable, Topic
 6  
 7  app = Flow()
 8  
 9  @app.pipeline(
10    source=Topic(topic_id="input-topic"),
11    sink=AnalysisTable("output-table"))
12  def process(element: Dict[str, Any]):
13      return element
14

Open Source Meta-Framework

BuildFlow brings the best open source frameworks together into a single API.

The Runtime is built on top of Ray Core
The Infrastructure is managed with Pulumi
The Metrics module is compatible with Prometheus and Grafana.

See Overview

Autoscaling by Default

Automatically scale to any size workload. Ensure your pipeline can handle high usage during peak hours, and is cost effective during low usage periods.

Learn About Autoscaling

Effecient Parallel Processing

BuildFlow runs in a highly-concurrent, async runtime allowing you to easily scale out parallel tasks with Ray or any other async framework.

Learn About Parallelism

Infrastructure from Code

BuildFlow helps eliminate operational work by including an (optional) resource creation / management module. For most use cases, this can eliminate the need for a separate deployment tool like Terraform or from visit the cloud console.

Learn About Infrastructure from Code