Skip to content

Under the hood

Let's review the processes and protocols involved in each step of deploying and executing a serverless function with Taubyte. This overview will help clarify the interactions between different components and the significance of each protocol in the overall architecture.

Project Creation

The creation of a project is predominantly a client-side operation, facilitated by either the Web Console or the tau-cli. This process involves direct communication with GitHub to establish the necessary repositories. Only towards the end does the auth protocol come into play, tasked with registering these repositories. This registration process includes storing critical secrets such as the deployment key, essential for cloning code during CI/CD processes, and the webhook secret, used to authenticate events coming from GitHub.

The sequence diagram below illustrates these interactions in detail:

    participant Client as Web Console/tau-cli
    participant GitHub
    participant Auth as auth

    Note over Client,GitHub: Project creation initiated on client side
    Client->>GitHub: Create repositories
    Note over GitHub: Repositories and necessary metadata created

    GitHub-->>Client: Confirm repository creation
    Note over Client,Auth: Only at the end does auth play a role

    Client->>Auth: Register repositories
    Note over Client,Auth: Includes storing deployment key and webhook secret

    Auth-->>Client: Confirm registration and storage of secrets

Pushing Changes to GitHub

When changes are pushed to GitHub, a sequence of events is triggered, initiating a process that involves multiple protocols to manage and execute CI/CD jobs. The patrick protocol plays a pivotal role in verifying the legitimacy of events and creating jobs, while the monkey protocol is responsible for the execution of these jobs. Subsequently, the tns stores and replicates the compiled configuration, and the hoarder protocol ensures the replication of updated assets.

The sequence diagram below visualizes this workflow:

    participant GitHub as GitHub
    participant Patrick as Patrick
    participant Monkey as Monkey
    participant TNS as TNS Protocol
    participant Hoarder as Hoarder

    Note over GitHub,Patrick: git push

    GitHub->>Patrick: Github Web Hook
    Patrick->>Patrick: Verify events are legit
    Patrick->>Patrick: Create jobs for events

    Patrick-->>Monkey: Broadcast jobs
    Note over Monkey: Bid for the job(s)

    Monkey->>Monkey: Job Execution
    Note over TNS: Assuming successful job execution

    Monkey->>TNS: Publish configuration

    Monkey->>Monkey: File chunked, structured in DAG

    Monkey->>TNS: Update Asset CIDs

    Monkey-->>Hoarder: Notify of new CID
    Hoarder->>Hoarder: Replicate

    Note over Monkey,Hoarder: Replicated by at least one node

It's important to note that because we are using Dreamland, a simulated environment, there's no direct way for GitHub to communicate with your nodes. This limitation is addressed by using the fixture called push-all, which emulates the git events that would typically trigger the CI/CD process in a production environment.

Hitting the Endpoint

The process for handling a request at an endpoint varies depending on the network's configuration. In a simple setup without gateways and HTTPS, the request's host name is resolved by the seer protocol before the request itself is processed by a substrate node. However, in networks with gateways or HTTPS requests, additional steps, like retreiving x509 certificate from auth, are involved.

The diagrams below detail these processes for different configurations:

  • Without Gateway and HTTPS:

        participant Client
        participant Seer as Seer
        participant Substrate
        participant TNS as TNS
        participant Peers as Other Peers
        Client->>Seer: Resolve Host
        Seer-->>Client: Nearest Substrate IP
        Client->>Substrate: HTTP request
        Substrate->>TNS: Lookup function
        TNS-->>Substrate: Returns config
        Substrate->>Peers: Request WebAssembly module
        Peers-->>Substrate: Download WebAssembly module
        Substrate->>Substrate: Execute handler
        Substrate-->>Client: Relay response

  • With Gateway:

        participant Client
        participant Seer as Seer DNS
        participant Gateway
        participant Substrate
        Client->>Seer: DNS query for endpoint
        Seer-->>Client: Resolve to nearest Gateway IP
        Client->>Gateway: HTTP request
        Gateway->>Substrate: Establish p2p connection to Substrates
        Gateway->>Substrate: Tunnel HTTP to best Substrate
        Note over Gateway,Substrate: Handling by Substrate is similar to before

  • With HTTPS:

        participant Client
        participant Gateway_Substrate as Gateway/Substrate
        participant Auth as Auth Protocol
        Client->>Gateway_Substrate: HTTPS request
        Gateway_Substrate->>Auth: Request certificate
        Auth-->>Gateway_Substrate: Return certificate
        Gateway_Substrate-->>Client: Return certificate to Client
        Note over Gateway_Substrate,Client: Secure connection established

    Note: Taubyte supports the Automatic Certificate Management Environment (ACME) protocol for auto-generating certificates using Let’s Encrypt.

These diagrams demonstrate the seamless and autonomous nature of the Taubyte platform, which requires no manual configuration and ensures a smooth deployment and execution process for serverless functions.