Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
Share
STRATEGY & INSIGHTS
17 min read
Share
The reference implementation can be used as a collection of individual reusable libraries and templates, or as a unified application framework.The reference implementation is organized into two sections:
Tier | Layer | Reference Architecture | Reference Implementation |
---|---|---|---|
Platform | Platform Services | Lightweight Pub/Sub Broker Protocol Bridge Event Streaming Broker Model OTA Service Model Registry Device Registry Training DataStore Container Registry Container Orchestration Engine Container Workflow Engine Edge Native Storage | Embedded Go MQTT Broker MQTT-Kafka Protocol Bridge Kafka/Strimzi Model OTA Server Model Registry μService Device Registry μService Training Datastore μService Docker Registry Service K3S Argo Workflows Longhorn |
Platform | MLOps | MLOps CD MLOps UI Control and Data Events Training Pipelines Ingest Pipelines MLOps DAGs | Argo CD Argo Dashboard Control and Data Topics Argo Workflows Training Pipeline Data Ingest μService Argo Demo DAG |
Tier | Layer | Reference Architecture | Reference Implementation |
---|---|---|---|
Inference | Cognition | Alerts Compressed ML Model Context Specific Inferencing Streaming Data Orchestration Agent | Motor Condition Alerts Quantized Model TF Lite PyCoral Logistic Regression Module Kafka K3S Agent |
Things | Perception | Protocol Gateway Sensor Data Acquisition Pre Processing Filter FOTA ML Model Actuator Control Closed Loop Inferencing Aggregation | OpenMQTTGateway Sensor module FFT DSP Module TF Lite Model Download Servo Controller Module TFLM Module Aggregation Module |
Infrastructure Tier | Device | AI Accelerator | Compute | Memory | OS/Kernel |
---|---|---|---|---|---|
Platform | Jetson Nano DevKit | GPU - 128-core NVIDIA Maxwell™ | CPU – Quad-core ARM® A57 @ 1.43 GHz | 2 GB 64-bit LPDDR4 | Ubuntu 18.04.6 LTS 4.9.253-tegra |
Platform | Raspberry Pi 4 | None | Quad Cortex-A72 @ 1.5GHz | 4GB LPDDR4 | Debian GNU/Linux 10 (buster) 5.10.63-v8+ |
Inference | Coral Dev Board | GPU - Vivante GC7000Lite TPU - Edge TPU VPU - 4Kp60 HEVC/H.265 | Quad Cortex-A53 @ 1.5 GHz | 1 GB LPDDR4 | Mendel GNU/Linux 5 (Eagle) 4.14.98-imx |
Inference | ESP32 SoC | None | MCU - Dual Core Xtensa® 32-bit LX6 @ 40Mhz | 448 KB ROM 520 KB SRAM | ESP-IDF FreeRTOS |
Things | ESP32 SoC | None | MCU - Dual Core Xtensa® 32-bit LX6 @ 40Mhz | 448 KB ROM 520 KB SRAM | ESP-IDF FreeRTOS |
bash brew install android-platform-tools
screen /dev/tty.SLAB_USBtoUART 115200
dpkg --print-architecture
sudo apt-get update
sudo apt-get upgrade
add cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1
dpkg -l | grep -i docker
sudo apt-get purge -y docker-engine docker docker.io docker-ce docker-ce-cli
sudo apt-get autoremove -y --purge docker-engine docker docker.io docker-ce
sudo rm -rf /var/lib/docker /etc/docker
sudo rm /etc/apparmor.d/docker
sudo groupdel docker
sudo rm -rf /var/run/docker.sock
sudo rm -rf ~/.docker
#replace the <IP Address> with the IP Address of the device or VM
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig ~/.kube/config --write-kubeconfig-mode 666 --tls-san <IP Address> --node-external-ip=<IP Address>" sh -
crictl info
cat /var/lib/rancher/k3s/server/token
#replace the <IP Address> with the IP Address of the K3S server node
#replace the <TOKEN> with the token from the server node
curl -sfL https://get.k3s.io | K3S_URL=https://<IP Address>:6443 K3S_TOKEN=<TOKEN> sh -
crictl info
With each successful agent node setup, you should be able to see the entire cluster by running this command on the K3S server node
kubectl get nodes -o wide -w
This is what I see on my cluster kubectl create ns architectsguide2aiot
kubectl label nodes agentnode-raspi1 controlnode=active
apiVersion: v1
kind: ConfigMap
metadata:
name: longhorn-default-setting
namespace: longhorn-system
data:
default-setting.yaml: |-
backup-target:
backup-target-credential-secret:
system-managed-components-node-selector:"controlnode: active"
.
.
.
# add this for each of the the following CRDs
# DaemonSet/longhorn-manager
# Service/longhorn-ui
# Deploymentlonghorn-driver-deployer
nodeSelector:
controlnode: active
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: longhorn-ingress
namespace: longhorn-system
annotations:
# type of authentication
nginx.ingress.kubernetes.io/auth-type: basic
# prevent the controller from redirecting (308) to HTTPS
nginx.ingress.kubernetes.io/ssl-redirect: "false"
# name of the secret that contains the user/password definitions
nginx.ingress.kubernetes.io/auth-secret: basic-auth
# message to display with an appropriate context why the authentication is required
nginx.ingress.kubernetes.io/auth-realm: "Authentication Required "
spec:
rules:
- http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: longhorn-frontend
port:
number: 80
- Replica Node Level Soft Anti-Affinity : true
- Replica Zone Level Soft Anti-Affinity : true
- System Managed Components Node Selector : controlnode: active
kubectl label nodes agentnode-raspi2 controlnode=active
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
longhorn-csi-plugin-rw5qv 2/2 Running 4 (18h ago) 10d 10.42.5.50 agentnode-raspi2 <none> <none>
longhorn-manager-dtbp5 1/1 Running 2 (18h ago) 10d 10.42.5.48 agentnode-raspi2 <none> <none>
instance-manager-e-f74eeb54 1/1 Running 0 172m 10.42.5.53 agentnode-raspi2 <none> <none>
engine-image-ei-4dbdb778-jbw5g 1/1 Running 2 (18h ago) 10d 10.42.5.52 agentnode-raspi2 <none> <none>
instance-manager-r-9f692f5b 1/1 Running 0 171m 10.42.5.54 agentnode-raspi2 <none> <none>
Name : artifacts-registry-volm
Size: 1 Gi
Replicas: 1
Frontend : Block Device
kubectl get pv,pvc -n architectsguide2aiot
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/artifacts-registry-volm 1Gi RWO Retain Bound architectsguide2aiot/artifacts-registry-volm longhorn-static 12d
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/artifacts-registry-volm Bound artifacts-registry-volm 1Gi RWO longhorn-static 12d
sudo apt-get update
sudo apt-get remove docker docker-engine docker.io
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
-d -p 5000:5000 --restart=always --name registry registry:2
{
"insecure-registries": ["localhost:5000"]
}
systemctl restart docker.service
#replace the <IP Address> with the IP Address of the node hosting the docker registry service
mirrors:
docker.<IP Address>.nip.io:5000:
endpoint:
- "http://docker.<IP Address>.nip.io:5000"
#replace the <IP Address> with the IP Address of the node hosting the docker registry service
[plugins.cri.registry]
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins.cri.registry.mirrors."docker.<IP Address>.nip.io:5000"]
endpoint = ["http://docker.<IP Address>.nip.io:5000"]
systemctl restart k3s-agent.service
crictl info
docker buildx
docker buildx create --name mybuilder
kubectl create ns architectsguide2aiot
kubectl apply -n architectsguide2aiot -f https://github.com/argoproj/argo-workflows/releases/download/v3.1.11/install.yaml
kubectl patch configmap/workflow-controller-configmap \
-n architectsguide2aiot \
--type merge \
-p '{"data":{"containerRuntimeExecutor":"k8sapi"}}'
kubectl -n architectsguide2aiot port-forward svc/argo-server 2746:2746
kubectl -n architectsguide2aiot exec argo-server-<pod name> -- argo auth token
kubectl create ns architectsguide2aiot
kubectl create -f 'https://strimzi.io/install/latest?namespace=architectsguide2aiot' -n architectsguide2aiot
kubectl apply -f 'https://strimzi.io/examples/latest/kafka/kafka-persistent-single.yaml' -n architectsguide2aiot
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: architectsguide2aiot-aiotops-cluster
spec:
kafka:
version: 2.8.0
replicas: 1
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
- name: external
port: 9094
type: nodeport
tls: false
configuration:
bootstrap:
nodePort: 32199
brokers:
- broker: 0
nodePort: 32000
- broker: 1
nodePort: 32001
- broker: 2
nodePort: 32002
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
template:
pod:
tolerations:
- key: "dedicated"
operator: "Equal"
value: "Kafka"
effect: "NoSchedule"
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: dedicated
operator: In
values:
- Kafka
kubectl apply -f 'https://strimzi.io/examples/latest/kafka/kafka-persistent-single.yaml' -n architectsguide2aiot
kubectl wait kafka/my-cluster --for=condition=Ready --timeout=300s -n architectsguide2aiot
kubectl label nodes agentnode-coral-tpu1 tpuAccelerator=true
kubectl label nodes agentnode-coral-tpu2 tpuAccelerator=true
kubectl label nodes agentnode-coral-tpu3 tpuAccelerator=true
kubectl label nodes agentnode-nvidia-jetson gpuAccelerator=true
In order to prevent strimzi from scheduling workloads on the devices in the inference tier use the following taints:
kubectl taint nodes agentnode-coral-tpu1 dedicated=Kafka:NoSchedule
kubectl taint nodes agentnode-coral-tpu2 dedicated=Kafka:NoSchedule
kubectl taint nodes agentnode-coral-tpu3 dedicated=Kafka:NoSchedule
Get emerging insights on emerging technology straight to your inbox.
Outshift is leading the way in building an open, interoperable, agent-first, quantum-safe infrastructure for the future of artificial intelligence.
* No email required
The Shift is Outshift’s exclusive newsletter.
Get the latest news and updates on generative AI, quantum computing, and other groundbreaking innovations shaping the future of technology.