Architecture > Advanced Setup

Running 3forge on a Distributed Kubernetes Cluster

This guide provides instructions for deploying a distributed AMI application on a Kubernetes cluster. Please read the main Kubernetes page first.

Overview

AMI's modular architecture enables easy distributed deployment in a Kubernetes cluster.

For this guide, each AMI component (Center, Web, and Relay) is deployed in its own Kubernetes pod, each running in its own StatefulSet.

This setup uses:

• StatefulSets for each component to provide stable, unique network identifiers (hostnames)
• Headless Services for stable, internal DNS discovery (e.g., ami-center-0.ami-center-headless-service)
• Cluster IP Service on the Center for internal access
• NodePort Service for external access to the Web component

Requirements

• A working Kubernetes cluster
• kubectl and kustomize are installed and configured

• Each AMI component as Docker images (ami-center, ami-relay, ami-web) hosted in some repository, with correctly configured local.properties files for each component:

  Center

  # Center
  ami.components=center
  ami.center.port=3270
  ami.db.jdbc.port=3280
  ami.port=3289
  ami.db.console.port=3290
  f1.console.port=3285
  

  Web

  # Web
  ami.components=web
  ami.center.port=3270
  ami.center.host=ami-center-service
  f1.console.port=3285
  http.port=33332
  

  Relay

  # Relay:
  ami.components=relay
  ami.center.port=3270
  ami.center.host=ami-center-service
  

Setup

Create a new directory named distributed_k8s and place the following YAML files inside it.

kustomization.yamlweb.yamlcenter.yamlrelay.yaml

This bundles K8s resources.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - center.yaml
  - relay.yaml
  - web.yaml

This defines the Web component and exposes a static port for external access.

# Headless Service for the Web StatefulSet
apiVersion: v1
kind: Service
metadata:
  name: ami-web-headless-service
spec:
  clusterIP: None
  selector:
    app: web
---
# Web StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ami-web
spec:
  serviceName: "ami-web-headless-service"
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      # imagePullSecrets:
      # - name: regcred # Uncomment if using private registry
      containers:
      - name: web
        image: ami-image # or whatever your Docker image for AMI is called
        ports:
          - name: f1-console-port
            containerPort: 3285
          - name: http-port
            containerPort: 33332
          - name: center-port
            containerPort: 3270
---
# Web Service using NodePort
apiVersion: v1
kind: Service
metadata:
  name: ami-web-service
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - name: http
      port: 80 # Port inside the cluster
      targetPort: 33332 # Port on the pod
      nodePort: 30080 # Port to access from outside the cluster

This defines the Center component.

# Headless Service for the Center StatefulSet
apiVersion: v1
kind: Service
metadata:
  name: ami-center-headless-service
spec:
  clusterIP: None
  selector:
    app: center
---
# Center StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ami-center
spec:
  serviceName: "ami-center-headless-service"
  replicas: 1
  selector:
    matchLabels:
      app: center
  template:
    metadata:
      labels:
        app: center
    spec:
      # imagePullSecrets:
      #   - name: regcred # Uncomment if using private registry
      containers:
      - name: center
        image: franc1sd/ami-center:stable
        ports:
          - name: center-port
            containerPort: 3270
          - name: db-jdbc-port
            containerPort: 3280
          - name: port
            containerPort: 3289
          - name: db-console-port
            containerPort: 3290
          - name: f1-console-port
            containerPort: 3285
---
# Center Service
apiVersion: v1
kind: Service
metadata:
name: ami-center-service
spec:
  type: ClusterIP
  selector:
    app: center
  ports:
    - name: center-port
      port: 3270 # Service listening port
      targetPort: 3270 # Port on pod for traffic
    - name: db-jdbc-port
      port: 3280
      targetPort: 3280
    - name: port
      port: 3289
      targetPort: 3289
    - name: db-console-port
      port: 3290
      targetPort: 3290
    - name: f1-console-port
      port: 3285
      targetPort: 3285

This defines the Relay component.

# Headless Service for the Relay StatefulSet
apiVersion: v1
kind: Service
metadata:
  name: ami-relay-headless-service
spec:
  clusterIP: None
  selector:
    app: relay
---
# Relay StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: ami-relay
spec:
  serviceName: "ami-relay-headless-service"
  replicas: 1
  selector:
    matchLabels:
      app: relay
  template:
    metadata:
      labels:
        app: relay
    spec:
      # imagePullSecrets:
      #   - name: regcred # Uncomment if using private registry
      containers:
      - name: relay
        image: franc1sd/ami-relay:stable
        ports:
          - name: center-port
            containerPort: 3270
---
# Relay Service
apiVersion: v1
kind: Service
metadata:
  name: ami-relay-service
spec:
  type: LoadBalancer 
  selector:
    app: relay
  ports:
    - name: center-port
      port: 3270
      targetPort: 3270

Workflow

1. Apply the Manifests. From within the distributed_k8s directory, run the following kustomize command to deploy all resources:

kubectl apply -k .

1. Verify deployment and check the status of your pods and services by running the following command:

kubectl get all

1. Access the pod and the AMI interface at http://<YOUR_NODE_IP>:30080.

2. To clean up and delete all the resources created by this guide, run the following command from the distributed_k8s directory:

kubectl delete -k .