TinyBird 101

Notes on Tinybird platform.

#Tinybird Data platform for building real-time data products. Focus on data freshness (process and consumption) + low latency.

  • Based on ClickHouse --> One of the best OS analytical databases. Can be queried using SQL. It is based on columns

Main concepts

  • Workspace: Unit of work
  • Datasoruce: Basically, a table - Tools on top on a table to make it a Datasource
  • Pipe: SQL queries that can be composed on different parts - Handles the logic of the queries. There are many types.
    • A type is endpoint --> Create a SQL query and expose result as an endpoint
    • Materialized Views Pipes, Copy pipes (easily move data to query to datasource) and Sink (export data to an S3 bucket) pipes are another type of pipes
  • Token: Gives permissions
  • Playground: Play without concern

UI

On pipes, there is a build tool on the top for adding different parameters. The language for building those might be a bit complex, use the UI helper to ease the task.

example

Also on the top right one can create directly an api endpoint plus an auth token for accessing the queried data.

We can add a default value on WHERE ORDER BY etc clauses so that it always default, even if we don't pass any value at all.

TinyBird CLI

tb init --> Init tinybird workspace locally. It will create some folders locally:

  • datasources: tb pull --> Will pull data that we have on our workspace account into previously created folders from tb init. Things like pipes or datasources with their fixtures will be downloaded tb push --> Push changes (ex: new pipes) to remote repo

Resources

https://www.tinybird.co/docs/guides/publish/advanced-dynamic-endpoints-functions