We have checked out the shifu/docs/design and found something to update.

1.deciceshifu.ConfigFilePath has been given a constant when it is passed in. Can omit the step of judging whether it is empty
2.Namespace is obtained from the environment variable. Is it necessary to add a step to determine whether it is empty

Please Kaiwen's wechat contact id to "Donoteattoomuchla"

Describe the bug

Use Shifu to create a MQTT deviceShifu will fail using v0.0.6.

To Reproduce

git clone https://github.com/Edgenesis/shifu.git
cd shifu/examples/mqttDeviceShifu/mqtt_deploy

Modify spec.address and protocolSettings.MQTTSetting.MQTTTopic to my MQTT broker's IP address and topic.

$ sudo kubectl apply -f .

$ sudo kubectl get pods -A
NAMESPACE            NAME                                            READY   STATUS    RESTARTS     AGE
deviceshifu          deviceshifu-mqtt-deployment-74bfc6d877-4bxhn    0/1     Error     1 (9s ago)   14s

$ sudo kubectl describe pod deviceshifu-mqtt-deployment-74bfc6d877-4bxhn -n deviceshifu
Name:             deviceshifu-mqtt-deployment-74bfc6d877-4bxhn
Namespace:        deviceshifu
Priority:         0
Service Account:  edgedevice-sa
Node:             kind-control-plane/172.18.0.2
Start Time:       Thu, 01 Sep 2022 14:21:07 +0800
Labels:           app=deviceshifu-mqtt-deployment
                  pod-template-hash=74bfc6d877
Annotations:      <none>
Status:           Running
IP:               10.244.0.8
IPs:
  IP:           10.244.0.8
Controlled By:  ReplicaSet/deviceshifu-mqtt-deployment-74bfc6d877
Containers:
  deviceshifu-http:
    Container ID:   containerd://c595e4a2e1eae64ae0c097f679d30a713b2df07039f16c64bfe40d658f7aaa4d
    Image:          edgehub/deviceshifu-http-mqtt:v0.0.6
    Image ID:       docker.io/edgehub/deviceshifu-http-mqtt@sha256:329ad1beaa74d6fa8c586ec8055e1e8db584a0595039e50d9917f496b3f0bfbd
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Thu, 01 Sep 2022 14:22:05 +0800
      Finished:     Thu, 01 Sep 2022 14:22:10 +0800
    Ready:          False
    Restart Count:  3
    Environment:
      EDGEDEVICE_NAME:       edgedevice-mqtt
      EDGEDEVICE_NAMESPACE:  devices
    Mounts:
      /etc/edgedevice/config from deviceshifu-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-x27b9 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  deviceshifu-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      mqtt-configmap-0.0.1
    Optional:  false
  kube-api-access-x27b9:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  87s                default-scheduler  Successfully assigned deviceshifu/deviceshifu-mqtt-deployment-74bfc6d877-4bxhn to kind-control-plane
  Normal   Pulled     29s (x4 over 87s)  kubelet            Container image "edgehub/deviceshifu-http-mqtt:v0.0.6" already present on machine
  Normal   Created    29s (x4 over 87s)  kubelet            Created container deviceshifu-http
  Normal   Started    29s (x4 over 87s)  kubelet            Started container deviceshifu-http
  Warning  BackOff    11s (x5 over 75s)  kubelet            Back-off restarting failed container

$ sudo kubectl logs deviceshifu-mqtt-deployment-74bfc6d877-4bxhn -n deviceshifu
Password:
2022/09/01 06:27:26 MQTT Server Address is empty, use address instead
2022/09/01 06:27:26 Connected
2022/09/01 06:27:26 Subscribed to topic: topic0
2022/09/01 06:27:26 updating device edgedevice-mqtt status to: Pending
2022/09/01 06:27:26 deviceShifu edgedevice-mqtt started
2022/09/01 06:27:26 Wait 5 seconds before updating status
deviceShifu edgedevice-mqtt's http server started
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xfa4e4f]

goroutine 11 [running]:
github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.getTelemetryCollectionServiceMap(0xc000098cf0)
	/shifu/deviceshifu/pkg/deviceshifubase/telemetry_service.go:49 +0xaf
github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.(*DeviceShifuBase).StartTelemetryCollection(0xc000098cf0, 0x0?)
	/shifu/deviceshifu/pkg/deviceshifubase/deviceshifubase.go:222 +0x6f
created by github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.(*DeviceShifuBase).Start
	/shifu/deviceshifu/pkg/deviceshifubase/deviceshifubase.go:258 +0x119

Expected behavior

I use deviceshifu-http-mqtt:v0.0.1 and it works!

$ sudo kubectl get pods -A -w
deviceshifu          deviceshifu-mqtt-deployment-65fd55674b-ptcq7    1/1     Running       0          7s
$ sudo kubectl exec -it nginx -- bash
root@nginx:/# curl deviceshifu-mqtt.deviceshifu.svc.cluster.local/mqtt_data
{"mqtt_message":"哈哈哈","mqtt_receive_timestamp":"2022-09-01 06:55:15.60403948 +0000 UTC m=+47.354876899"}

Server (please complete the following information):

• Shifu Version: v0.0.6
• kind 0.14.0 on macOS 12.5.1
• CPU Architecture x86-64

OPC UA's deviceshifu will always reconnect to the device many times when device is reboot, until delete the deviceshifu

如题，方便直接添入github账号的开发者添加我们的微信，加入我们的微信群。可以留下郑凯文和郭琦的微信号引导添加，郑凯文微信号：zhengkaiwen196649 郭琦wechat：if7369

When trying to apply https://github.com/Edgenesis/shifu/blob/main/k8s/crd/install/shifu_install.yml through Kustomization the following error occurs:

Kustomization.yaml:

Where shifu.yaml is the above shifu_install.yaml file.

It seems the EdgeDevice CRD is duplicated in shifu_install.yaml:

Removing either of these sections allows applying the Kustomization to succeed.

Currently, shifu's response for trying to use an instruction that doesn't exist is 404 page not found.

Please change this to something like Error: Device instruction does not exist as it's currently not too clear it's shifu giving back the 404, especially when using a GO driver with an HTTP server.
It would also be convenient if this were configurable, but mainly having a different default would solve it.

Some of the device return not only a single message, but multiple messages for a single command
e.g.

Calibrate
Calibrate started
(10 seconds later)
Calibrate finished

Currently, resources associated with Shifu are named crd-controller-manager, crd-manager-config, etc.
It would be helpful for us if Shifu associated resources could have Shifu in their name for easier identification. Thanks!

Currently after issuing a reboot request and socket is closed from device end, the socket does not get reopened, and subsequent requests has error.

Shifu's feature roadmap is off. Please update it to give the community correct information.

Now this project uses godoc to generate documentation for developers. However, we also need to generate documentation for users.

Now API references for users are hosted on https://shifu.run/docs/references/api/. One disadvantage of this is that we should manually generate these API references, which is a time-wasting.

We can use the struct tag feature in Golang to make the reference information in source codes. Just as the following example:

package main

import (
	"fmt"
	"github.com/fatih/structs"
)

type User struct {
	id   int    `description-en:"user's ID" description-zh:"用户的ID"`
	name string `description-en:"user's name" description-zh:"用户的名字"`
	age  int    `description-en:"user's age" description-zh:"用户的年龄"`
	date `description-en:"account's date" description-zh:"用户相关的日期"`
}

type date struct {
	createdAt string `description-en:"created time of account" description-zh:"账户创建的日期"`
	updatedAt string `description-en:"updated time of account" description-zh:"账户更新的时间"`
}

func main() {
	user := &User{}
	fields := structs.Fields(user)
	getInfoOf(fields, "")
	// id: [user's ID, 用户的ID]
	// name: [user's name, 用户的名字]
	// age: [user's age, 用户的年龄]
	// date: [account's date, 用户相关的日期]
	//     createdAt: [created time of account, 账户创建的日期]
	//     updatedAt: [updated time of account, 账户更新的时间]
}

func getInfoOf(fields []*structs.Field, prefix string) {
	for _, field := range fields {
		tag_name := field.Name()
		tag_en := field.Tag("description-en")
		tag_zh := field.Tag("description-zh")
		if field.IsEmbedded() {
			fmt.Printf("%s: [%s, %s]\n", tag_name, tag_en, tag_zh)
			prefix_new := fmt.Sprintf("    %s", prefix)
			getInfoOf(field.Fields(), prefix_new)
		} else {
			fmt.Printf("%s%s: [%s, %s]\n", prefix, tag_name, tag_en, tag_zh)
		}
	}
}

In this project, for example, the API reference of DeviceShifuDriverProperties is in https://shifu.run/docs/references/api/deviceshifu-configmap#deviceshifudriverproperties. We can add struct tag in go codes (pkg/deviceshifu/deviceshifubase_config.go) as follows:

// DeviceShifuDriverProperties properties of deviceshifuDriver
type DeviceShifuDriverProperties struct {
	DriverSku       string `yaml:"driverSku" usage-en:"hardware models supported by the driver, such as Hikvision Camera." usage-zh:"表示驱动所适用的硬件型号，如 Hikvision Camera。"`
	DriverImage     string `yaml:"driverImage" usage-en:"(string) container image name of the driver, such as driver/hikvision-camera:v1.2.3." usage-zh:"表示驱动的容器镜像名称，如 driver/hikvision-camera:v1.2.3。"`
	DriverExecution string `yaml:"driverExecution,omitempty" usage-en:"execution path of the drive. For command line driver, relative/absolute path of the driver execution file needs to be filled in, such as python driver.py or C:\\driver.exe." uages-zh:"表示驱动的执行路径。针对于命令行的驱动，这里需要填写驱动的执行文件的相对/绝对路径，如 python driver.py 或 C:\\driver.exe。"`
}

Then we can use a script to generate according markdown strings to put on https://shifu.run/docs/references/api/.

/test/scripts/shifu-application-demo-env-setup.sh Script error reporting in Linux Environment

#Solve the problem by setting the following command via VIM：

:set ff=unix
:wq

Describe the bug
deviceShifu is posting to empty endpoints when no telemetry services are specified

To Reproduce

1. kubectl apply -f examples/deviceshifu/demo_device/edgedevice-agv/
2. kubectl logs ...
3. The following log will appear in the deviceshifu log:

I0919 14:14:02.064861       1 deviceshifubase.go:208] Status is: true
I0919 14:14:02.064881       1 deviceshifubase.go:139] updating device edgedevice-agv status to: Running
I0919 14:14:05.088750       1 telemetry_service.go:34] pushing &{0xc0000c50c0 {0 0} false <nil> 0x6c3160 0x6c3260} to
E0919 14:14:05.088910       1 telemetry_service.go:38] HTTP POST error for telemetry service , error: Post "": unsupported protocol scheme ""

Expected behavior
deviceShifu should not send out telemetries when no endpoints are specified

Log Output

If applicable, add logs to help explain your problem.

Server (please complete the following information):

• Shifu Version: v0.1.1
• Kubernetes distro: Kubernetes running in kind
• Version v1.24.0
• CPU Architecture amd64

Additional context

Add any other context about the problem here.

In examples/rtspDeviceShifu/camera-deployment/deviceshifu-camera-configmap.yaml:

    instructionSettings:
      defaultTimeoutSeconds: 0

should be added or the camera will stop streaming after the default 4 seconds.

Supplement:

data:
  driverProperties: |
    driverSku: HikVision
    driverImage: edgenesis/camera-python:v0.0.1-1
  instructions: |
    instructionSettings:
      defaultTimeoutSeconds: 0
    instructions:
      capture:
      info:
      stream:
      move/up:
      move/down:
      move/left:
      move/right:

By default, The SINUMERIK OPCUA server deactivates anonymous authentication in some CNC versions like 04.07 and 04.08.
Besides, Anonymous access can be a security risk. Anonymous access should be only for commissioning.

Should this be a picture

When try to add a PLC device, 4 yaml flies are needed and serval files should be edited accordingly.
But currently, the PLC configuration of different protocols are written in different yaml files.
Such as Siemens-S7 IP is written in plc-deviceshifu-deployment.yaml, for OPCUA IP is written in opcua_edgedevice.yaml, they are different type files.

I think users or developers may feel confused when facing this situation.
So, suggest to optimize the file organization, to unify the PLC configuration in the same type file, establish a unified specification, thus whenever add a PLC, we always know to change the same yaml file.

it seems that the problem of large data volume has not been considered?

Many devices directly use a TCP/IP socket to communicate. Please add TCP/IP as a protocol that supports pass through data with basic encoding.
We need the following configurable parameters:

• encoding (ex: UTF-8, ASCII, raw bytes, etc)
• terminating character for sending a packet (ex: \r\n, \n, none)
• terminating character for receiving a packet (ex: \r, \n, none)

OS:MacOS Monterey 12.0.1

1.When I build deviceShifu binaries directly execute this command, system report an error

CGO_ENABLED=0 GOOS=$(go env GOOS) GOARCH=$(go env GOARCH) go build -a -o /output/deviceshifu-http-socket deviceshifu/cmd/cmdSocket/main.go
go build command-line-arguments: mkdir /output/: read-only file system

MacOS does not allow users to create folders in the root directory

2.When build deviceShifu docker images execute this command, It Seems that $(go env GOOS) And $(go env GOARCH) are invalid

make buildx-load-image-deviceshifu
docker buildx build --platform= / -f /Users/xxx/Desktop/Project/Go/shifu/deviceshifu/Dockerfile.deviceshifu --build-arg PROJECT_ROOT="/Users/xxx/Desktop/Project/Go/shifu" /Users/xxx/Desktop/Project/Go/shifu -t edgehub/deviceshifu-http-http:v0.0.1 --load

Use $(shell go env GOOS)And $(shell go env GOARCH) replace them)

3. After Question2, when executing the dockerfile,appear another error

make buildx-load-image-deviceshifu
...
=> ERROR exporting to image

exporting to image:

error: failed to solve: operating system is not supported
make: *** [buildx-load-image-deviceshifu] Error 1

It Seems That Darwin is not supported in this images edgehub/distroless-static:nonroot

On macOS GOOS should be linux instead of darwin

4.guide-on-writing-an-application-for-deviceShifu.md documentation error

for example

./test/scripts/shifu-application-demo-env-setup.sh apply applicationDemo

the shifu-application-demo-env-setup.sh not exists. This markdown should be rewrite

This document no longer exists:
https://github.com/Edgenesis/shifu/blob/main/README.md?plain=1#L38
https://github.com/Edgenesis/shifu/blob/main/README-zh.md?plain=1#L37

should be updated to the latest one in PR #69

The useless protocol need to be deleted.

Can I support serial device data acquisition? For example, weighing machine equipment and light curtain equipment

When sending command using /cmd route, we need to add configuration that the device uses non-default \n separator for a message

For OPC UA Server of Siemens PLCs, the NodeID always contain "", such as ns=3;s="TEST", this is default in Siemens PLC and cannot be modified.
When add NodeID of Siemens PLC in the yaml file like this, Pod deviceshifu-opcua-deployment-xxxx will CrashLoopBackOff.

Please fix this issue for Siemens PLCs.

One more things, it would be better to design a solution for multiple read, once HTTP curl get all tagvalues, which is important for industrial application.

Accepting disclaimer at https://shifu.run/disclaimer/ redirects to https://shifu.run/disclaimer/demo which is 404 not found

current dev container https://github.com/deis/docker-go-dev in Shifu is linux/x86 only. Need to add arm64 support

Describe the bug

A clear and concise description of what the bug is.
deviceshifu crash when using socket for it's telemtrySettings is omit

To Reproduce

Steps to reproduce the behavior:

1. ... install shifu crd
2. ...apply socket_config.yaml
3. ...
4. ...

  telemetries: |
    telemetries:

Expected behavior
Running deviceshifu succeed
A clear and concise description of what you expected to happen.

Log Output

If applicable, add logs to help explain your problem.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xaf0404]

goroutine 23 [running]:
github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.getTelemetryCollectionServiceMap(0x4000127470)
        /shifu/deviceshifu/pkg/deviceshifubase/telemetry_service.go:49 +0x74
github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.(*DeviceShifuBase).StartTelemetryCollection(0x4000127470, 0x0?)
        /shifu/deviceshifu/pkg/deviceshifubase/deviceshifubase.go:221 +0x68
created by github.com/edgenesis/shifu/deviceshifu/pkg/deviceshifubase.(*DeviceShifuBase).Start
        /shifu/deviceshifu/pkg/deviceshifubase/deviceshifubase.go:257 +0x11c

Server (please complete the following information):

• Shifu Version: [e.g. v0.0.5] v0.0.5
• Kubernetes distro [e.g. K8s, K3s] kind
• Version [e.g. v1.24.0] v0.14.0
• CPU Architecture [e.g. amd64, arm64w] arm64

Additional context

Add any other context about the problem here.

Currently we have 3 versions of deviceshifu:

deviceshifu-http:v0.0.1
deviceshifu-http-multi:v0.0.1
deviceshifu-http-http:v0.0.1

Need to change all occurrences to deviceshifu-http-http:v0.0.1

In our project, there are some concepts such as devices, controllers, deviceshifu, etc.
But the names of these Kubernetes resources are confusing for users, such as:

1. the controller name is crd-controller-manager in crd-system namespace.
2. the device pod name is a pure device name and in devices namespace.
3. the deviceshifu pod name start with edgedevice- and is in the default namespace.

In order to make users understand the shifu resources easily and make the components' relationship clear, I suggest renaming these resources such as:

1. make 2 namespaces, one named shifu-system which manages all the CRs, and another one named Shifu which manages the device's related resources including edge device and deviceShifu.
2. the device pod/service/ConfigMap, etc. start with "device-" in the namespace "shifu"
3. the deviceShifu pod/service/ConfigMap, etc. starts with "deviceshifu-" or "shifu-" in the namespace "shifu".
4. the controller-related resource start with "shifu-controller-" in the namespace "shifu-system".

Multi-tenant consideration:
For the resources in namespace "shifu", we also can consider just letting users decide the namespace name because the users can have permission to do resource isolation. Please consider the multi-tenant management use cases.

Describe the bug

Pipeline failed to build deviceshifu-http-plc4x
https://dev.azure.com/Edgenesis/shifu/_build/results?buildId=2303&view=results

To Reproduce

Steps to reproduce the behavior:

1. run make buildx-push-image-deviceshifu-http-plc4x

Expected behavior

should build all plc4x images

Log Output
#0 2.867 gcc: error: unrecognized command-line option '-m64'

If applicable, add logs to help explain your problem.

Server (please complete the following information):

• Shifu Version: [v0.2.1]
• Kubernetes distro [n/a]
• Version [n/a]
• CPU Architecture [arm64]

Additional context

Add any other context about the problem here.

Need to update github.com/gogo/protobuf

Issue here:
https://github.com/Edgenesis/shifu/security/dependabot/1

Describe the bug

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:

1. ...
2. ...
3. ...
4. ...

Expected behavior

A clear and concise description of what you expected to happen.

Log Output

If applicable, add logs to help explain your problem.

Server (please complete the following information):

• Shifu Version: [e.g. v0.0.5]
• Kubernetes distro [e.g. K8s, K3s]
• Version [e.g. v1.24.0]
• CPU Architecture [e.g. amd64, arm64w]

Additional context

Add any other context about the problem here.

Need to read and write some registers on Siemens plc, such as DB, PE series of s7

https://shifu.run/zh-Hans/docs/references/api/edgedevice/

We plan to move to using a standard embedded system across a number of devices. The main controller will likely be a single-board computer running a RTOS. It would save a lot of time and effort to have a set of libraries we can use to directly support and utilize Shifu features in the device.
Some examples include:

• A Shifu-aware state machine
• A way to export device-level commands to Shifu automatically
• Auto device discovery when connected / disconnected from a network
• Integration of healthcheck with hardware-level watchdog
• A standard way to query device state data and better synchronization of the digital twin and the real device

This issue is not fully developed and likely needs to be revised / updated.

Add support for getting stateful info about a device from Shifu. Shifu should internally store this stateful data and expose it through a standard setter / getter interface.

A direct example use case could be storing whether a device has been primed and allowing applications to query this.

I saw that in some Go projects, an icon called go documentation is set in README.

If this project uses godoc, this icon could also be added.

Currently, running kubectl apply does not correctly update edgedevice resources.
For example: kubectl apply -f <working_dir>/helloworld-device/configuration does not actually apply changes to deviceshifu-helloworld-configmap.yaml.

Please add a note in the docs explaining that as a workaround one must first use kubectl delete followed by kubectl apply.

For each device we want to integrate, we often have to write a lot of device-specific code.
Currently we can either create a layer on top of Shifu (between Shifu and our services), or create an image and have Shifu deploy / manage it. These approaches both have some downsides.

• With a layer on top of Shifu, Shifu can't understand the data coming from the device and can't do things like health checks, error handling, state tracking or anything that needs parsing of the data returned
• For both methods, each device we want to integrate requires writing a lot of boilerplate code such as health checks, communication interfaces, API registration, etc
• We already have a system in place to deploy and manage Docker containers and communicate through these with physical devices. Going from service to Shifu resource to our image to the physical device adds another layer, additional configuration, and still requires us to develop a device-specific image each time. Although we'll always have to write device-specific code somewhere, the current feature set is not very different from our direct Docker image deployment method despite the additional overhead.

We hope we can instead offload some features we have to write for each device into Shifu, and find a way to better manage the device-level code required for each device.

This issue is not fully developed and likely needs to be revised / updated.