Under the hood

Let's review the processes and protocols involved in each step of deploying and executing a serverless function with Taubyte. This overview will help clarify the interactions between different components and the significance of each protocol in the overall architecture.

Project Creation

The creation of a project is predominantly a client-side operation, facilitated by either the Web Console or the tau-cli. This process involves direct communication with GitHub to establish the necessary repositories. Only towards the end does the auth protocol come into play, tasked with registering these repositories. This registration process includes storing critical secrets such as the deployment key, essential for cloning code during CI/CD processes, and the webhook secret, used to authenticate events coming from GitHub.

The sequence diagram below illustrates these interactions in detail:

sequenceDiagram
    participant Client as Web Console/tau-cli
    participant GitHub
    participant Auth as auth

    Note over Client,GitHub: Project creation initiated on client side
    Client->>GitHub: Create repositories
    Note over GitHub: Repositories and necessary metadata created

    GitHub-->>Client: Confirm repository creation
    Note over Client,Auth: Only at the end does auth play a role

    Client->>Auth: Register repositories
    Note over Client,Auth: Includes storing deployment key and webhook secret

    Auth-->>Client: Confirm registration and storage of secrets

Pushing Changes to GitHub

When changes are pushed to GitHub, a sequence of events is triggered, initiating a process that involves multiple protocols to manage and execute CI/CD jobs. The patrick protocol plays a pivotal role in verifying the legitimacy of events and creating jobs, while the monkey protocol is responsible for the execution of these jobs. Subsequently, the tns stores and replicates the compiled configuration, and the hoarder protocol ensures the replication of updated assets.

The sequence diagram below visualizes this workflow:

sequenceDiagram
    participant GitHub as GitHub
    participant Patrick as Patrick
    participant Monkey as Monkey
    participant TNS as TNS Protocol
    participant Hoarder as Hoarder

    Note over GitHub,Patrick: git push

    GitHub->>Patrick: Github Web Hook
    Patrick->>Patrick: Verify events are legit
    Patrick->>Patrick: Create jobs for events

    Patrick-->>Monkey: Broadcast jobs
    Note over Monkey: Bid for the job(s)

    Monkey->>Monkey: Job Execution
    Note over TNS: Assuming successful job execution

    Monkey->>TNS: Publish configuration

    Monkey->>Monkey: File chunked, structured in DAG

    Monkey->>TNS: Update Asset CIDs

    Monkey-->>Hoarder: Notify of new CID
    Hoarder->>Hoarder: Replicate

    Note over Monkey,Hoarder: Replicated by at least one node

It's important to note that because we are using Dreamland, a simulated environment, there's no direct way for GitHub to communicate with your nodes. This limitation is addressed by using the fixture called push-all, which emulates the git events that would typically trigger the CI/CD process in a production environment.

Hitting the Endpoint

The process for handling a request at an endpoint varies depending on the network's configuration. In a simple setup without gateways and HTTPS, the request's host name is resolved by the seer protocol before the request itself is processed by a substrate node. However, in networks with gateways or HTTPS requests, additional steps, like retreiving x509 certificate from auth, are involved.

The diagrams below detail these processes for different configurations:

• Without Gateway and HTTPS:

  sequenceDiagram
      participant Client
      participant Seer as Seer
      participant Substrate
      participant TNS as TNS
      participant Peers as Other Peers
  
      Client->>Seer: Resolve Host
      Seer-->>Client: Nearest Substrate IP
      Client->>Substrate: HTTP request
  
      Substrate->>TNS: Lookup function
      TNS-->>Substrate: Returns config
  
      Substrate->>Peers: Request WebAssembly module
      Peers-->>Substrate: Download WebAssembly module
  
      Substrate->>Substrate: Execute handler
      Substrate-->>Client: Relay response

• With Gateway:

  sequenceDiagram
      participant Client
      participant Seer as Seer DNS
      participant Gateway
      participant Substrate
  
      Client->>Seer: DNS query for endpoint
      Seer-->>Client: Resolve to nearest Gateway IP
      Client->>Gateway: HTTP request
  
      Gateway->>Substrate: Establish p2p connection to Substrates
      Gateway->>Substrate: Tunnel HTTP to best Substrate
      Note over Gateway,Substrate: Handling by Substrate is similar to before

• With HTTPS:

  sequenceDiagram
      participant Client
      participant Gateway_Substrate as Gateway/Substrate
      participant Auth as Auth Protocol
  
      Client->>Gateway_Substrate: HTTPS request
      Gateway_Substrate->>Auth: Request certificate
      Auth-->>Gateway_Substrate: Return certificate
      Gateway_Substrate-->>Client: Return certificate to Client
  
      Note over Gateway_Substrate,Client: Secure connection established

  Note: Taubyte supports the Automatic Certificate Management Environment (ACME) protocol for auto-generating certificates using Let’s Encrypt.

These diagrams demonstrate the seamless and autonomous nature of the Taubyte platform, which requires no manual configuration and ensures a smooth deployment and execution process for serverless functions.