cloud-barista / cm-beetle Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 5.0 3.01 MB

License: Apache License 2.0

Dockerfile 0.94% Makefile 0.39% Go 94.57% Python 3.31% Shell 0.54% HTML 0.25%

cm-beetle's Introduction

Cloud-Barista

The Cloud-Barista is a Multi-Cloud Service Platform SW.

Cloud-Barista consists of multiple frameworks (sub-systems) to accommodate microservice-like architecture.

Please take a look Cloud-Barista Website for a detail decription.

Note for developing and using Cloud-Barista

Development stage of Cloud-Barista

Cloud-Barista is currently under development. (not v1.0 yet)
We welcome any new suggestions, issues, opinions, and controbutors !
Please note that the functionalities of Cloud-Barista are not stable and secure yet.
Becareful if you plan to use the current release in production.
If you have any difficulties in using Cloud-Barista, please let us know.
(Open an issue or Join the Cloud-Barista Slack)

This repository is an integrated archive for repository of major frameworks. These repositories are included and listed in the root directory. This repo reflects the latest release only.

The main frameworks or tools are as follows (the release version of each repository may vary),

CB-Spider (connects all clouds in a single interface)
- Upstream repo: https://github.com/cloud-barista/cb-spider
CB-Tumblebug (manages multi-cloud infrastructures)
- Upstream repo: https://github.com/cloud-barista/cb-tumblebug
CB-Dragonfly (monitors multi-cloud services)
- Upstream repo: https://github.com/cloud-barista/cb-dragonfly
- Note: not updated since v0.8.0, possible to be deprecated
CB-Bridge/cb-operator (operation tool for Cloud-Barista system runtime)
- Upstream repo: https://github.com/cloud-barista/cb-operator
- Note: As the components of Cloud-Barista are currently simplified, using cb-operator might require additional effort for users. It is recommended to use cb-tumblebug directly.
CB-Bridge/cb-store (provides an unified DB interface for meta info of Cloud-Barista)
- Upstream repo: https://github.com/cloud-barista/cb-store
CB-Bridge/cb-log (provides log library to Cloud-Barista system)
- Upstream repo: https://github.com/cloud-barista/cb-log

[Note] CB-Larva is a special repository that incubates (research and develop) new Multi-Cloud technologies. CB-Larva explores interesting ideas and shows the possibility of those (i.e., Proof of Concept (POC)). That's why we encourage you to take a look and contribute to the special repository. Please note that the source code of CB-Larva would not be released and archived in this repository for the time being.

CB-Larva/cb-cladnet (POC for the cloud adaptive network)
- Upstream repo: https://github.com/cloud-barista/cb-larva/tree/main/poc-cb-net

[실행 환경]

Linux (추천: Ubuntu v22.04)

[설치 및 실행]

Cloud-Barista 플랫폼 통합 실행 (Docker 이미지 기반)
- cb-operator 를 통해 Cloud-Barista 전체 FW를 통합 실행할 수 있음
  - 참고: cloud-barista/cb-operator
  - As the components of Cloud-Barista are currently simplified, using cb-operator might require additional effort for users. It is recommended to use cb-tumblebug directly.
Cloud-Barista 플랫폼 개별 FW 소스 다운로드 및 설치
- CB-Spider 설치 및 실행
  - cloud-barista/cb-spider README를 참고하여 설정, 설치
  - CB-Spider 실행
- CB-Tumblebug 설치 및 실행
  - cloud-barista/cb-tumblebug README를 참고하여 설정, 설치
    - CB-Spider API 서버 주소를 conf/setup.env 에 설정
    - CB-Dragonfly API 서버 주소를 conf/setup.env 에 설정
  - CB-Tumblebug 실행
- CB-Dragonfly 설치 및 실행
  - cloud-barista/cb-dragonfly README를 참고하여 설정, 설치
  - CB-Dragonfly 실행

[사용 방법 및 예시]

주요 서비스: 멀티 클라우드 인프라 서비스 (MCIS)

멀티 클라우드 인프라 서비스 환경 구성
- CB-Tumblebug 설정 및 실행
  - CB-Spider (필수)
  - CB-Dragonfly (MCIS 모니터링, CB-Tumblebug 자동 제어 기능에 필요)
멀티 클라우드 인프라 서비스 사용 방법
멀티 클라우드 인프라 유스케이스

[API 및 문서]

[특이 사항]

개발 단계: 기능 개발 우선 단계 (상용 활용시 안정화 및 보완 필요)
CSP 연동 검증 상태
- CB-Spider 기준 테스트 완료된 CSP: 링크 1 및 링크 2 참고
- CB-Tumblebug 기준 테스트 완료된 CSP: 링크 참고
- 현재는 개발 단계이므로 기능 안정성은 낮을 수 있음 (버그 리포트 기여 환영합니다!)

cm-beetle's People

Contributors

Stargazers

Watchers

Forkers

yunkon-kim seokho-son sykim-etri fossabot ish-hcc

cm-beetle's Issues

`cm-beetle:edge` is available

From now on, cm-beetle:edge image is available (see the details of type=edge in docker/metadata-action)

Container image registries:

GitHub Container Registry (GHCR): https://github.com/cloud-barista/cm-beetle/pkgs/container/cm-beetle
Docker Hub: https://hub.docker.com/r/cloudbaristaorg/cm-beetle/tags

When a PR is merged, the CD workflow will build and publish the edge image.

Note - About GitHub Packages (see the details in About billing for GitHub Packages

...
GitHub Packages is not available for private repositories owned by accounts using legacy per-repository plans.
....
GitHub Packages usage is free for public packages.
For private packages, each account on GitHub.com receives a certain amount of free storage and data transfer, depending on the account's plan. Any usage beyond the included amounts is controlled by spending limits.
...

...

Introduction to System Information Extractor

클라우드 마이그레이션을 수행함에 있어, 기존 컴퓨팅 인프라의 세부 형상 정보를 파악하는 것은 매우 중요한 작업입니다. 어느 수준까지 세부 형상을 파악하는지에 따라 난이도가 상당히 높을 수 있습니다.

그래서 탐색 개발로써, 물리/가상머신의 세부 형상 정보를 파악해 보는 정도의 프로그램이 구현되었습니다. (마이그레이션의 다양한 요소 중, 인프라 측면을 간단히 파악)

자세한 사항 내용은 아래 설명(README) 및 코드를 참고하시기 바랍니다.

System Information Extractor (special thanks @seokho-son @Leegyeongjin)

Manifests for deploying CM-Beetle on Kubernetes cluster

Simple README about `config` package

An error message about Config is appeared when running the program

ERROR 2023/12/19 20:42:44 get config file: Config File "config" Not Found in "[]"

There is no problem running the program. This can be resolved later.

Visualizing a codebase

Ref.: https://githubnext.com/projects/repo-visualization/

Method to synchronize source or destination model

What would you like to be enhanced
: Method to synchronize source or destination model(i.e., struct)

Why is this needed
: To create Swagger document (swag init is possible only when the struct is in the project)
: For efficient model (i.e. structure) synchronization and version/synchronization time recording.

Proposed solution
: Currently, a script is used.
: There is no clear way to improve it 😅

There is an issue checking `member_association` in the `auto_merge` workflow

https://github.com/cloud-barista/cm-beetle/actions/runs/6857359814/job/18646303981

Separate Zerolog for Echo Web Framework into middlewares package

Auto merge may not work in some cases

What happened
: Auto merge may not work in some cases

Enhance the Makefile

First of all, thank you to the CM-Honeybee maintainers 🙏

Ref. Makefile in the CM-Honeybee repo

What would you like to be enhanced
: Makefile

Why is this needed
: The Makefile in CM-Honeybee includes useful tools for contributing, such as lint, race, dependency, and so on.
: Also, the mechanisms and descriptions are great!

Proposed solution
: Review the CM-Honeybee's Makefile and select some of the tools needed.
: Apply the selected tools to CM-Beetle's Makefile.

Enhance `logger` to support multiple logger instances

Ref. #29

What would you like to be enhanced
: Enhance logger package to support multiple logger instances

Why is this needed
: This is necessary to set a different log level for each package.

Add or improve CM-Beetle deployment configurations and templates

What would you like to be enhanced
: IaaS, PaaS, system and container orchestration deployment configurations and templates (docker-compose, kubernetes/helm, mesos, terraform, bosh)

Why is this needed
: It is needed to provide a basic CM-beetle deployment method.

Proposed solution
: Add or improve deployment configurations and templates in /deployments

Candidates

Kubernetes manifests (To be updated)
Docker compose (To be added)
not limited.

It seems that external structs are not recognized when executing `swag init`.

It seems that external structs are not recognized when executing swag init.

What I've done

Get CM-Honeybee's infra package

go get -u github.com/cloud-barista/cm-honeybee/model/infra

go: downloading github.com/cloud-barista/cm-honeybee v0.0.0-20231208055314-9d736a30aa55
go: downloading github.com/libvirt/libvirt-go-xml v7.4.0+incompatible
go: downloading github.com/NeowayLabs/drm v0.0.0-20190824133025-4939fc0ad345
go: downloading github.com/jollaman999/utils v1.0.10
go: downloading golang.org/x/net v0.19.0
go: downloading golang.org/x/tools v0.16.0
go: downloading github.com/google/uuid v1.4.0
go: downloading github.com/labstack/echo/v4 v4.11.3
go: downloading github.com/labstack/gommon v0.4.1
go: downloading github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec
go: downloading golang.org/x/crypto v0.16.0
go: downloading github.com/dustin/go-humanize v1.0.1
go: downloading modernc.org/libc v1.37.0
go: downloading modernc.org/mathutil v1.6.0
go: downloading modernc.org/memory v1.7.2
go: downloading modernc.org/sqlite v1.27.0
go: added github.com/NeowayLabs/drm v0.0.0-20190824133025-4939fc0ad345
go: added github.com/cloud-barista/cm-honeybee v0.0.0-20231208055314-9d736a30aa55
go: added github.com/jollaman999/utils v1.0.10
go: upgraded github.com/labstack/echo/v4 v4.11.2 => v4.11.3
go: upgraded github.com/labstack/gommon v0.4.0 => v0.4.1
go: added github.com/libvirt/libvirt-go-xml v7.4.0+incompatible
go: upgraded golang.org/x/crypto v0.14.0 => v0.16.0
go: upgraded golang.org/x/net v0.17.0 => v0.19.0
go: upgraded golang.org/x/tools v0.7.0 => v0.16.0

Import a package

package controller

import (
	"fmt"
	"net/http"

	"github.com/cloud-barista/cm-honeybee/model/infra"
	"github.com/labstack/echo/v4"
)

Embedding Infra struct to RecommendInfraRequest (i.e, request body)

type RecommendInfraRequest struct {
	infra.Infra
}

Click to see full source code

// Package common is to handle REST API for common funcitonalities
package controller

import (
	"fmt"
	"net/http"

	"github.com/cloud-barista/cm-honeybee/model/infra"
	"github.com/labstack/echo/v4"
)

type Infrastructure struct {
	Network        string
	Disk           string
	Compute        string
	SecurityGroup  string
	VirtualMachine string
}

type RecommendInfraRequest struct {
	infra.Infra
}

type RecommendInfraResponse struct {
	Infrastructure
}

// RecommendInfra godoc
// @Summary Recommend an appropriate infrastructure for cloud migration
// @Description It recommends a cloud infrastructure most similar to the input. Infrastructure includes network, storage, compute, and so on.
// @Tags [Recommendation] Infrastructure
// @Accept  json
// @Produce  json
// @Param UserInfrastructure body RecommendInfraRequest true "Specify network, disk, compute, security group, virtual machine, etc."
// @Success 200 {object} RecommendInfraResponse "Successfully recommended an appropriate infrastructure for cloud migration"
// @Failure 404 {object} common.SimpleMsg
// @Failure 500 {object} common.SimpleMsg
// @Router /recommendation/infra [post]
func RecommendInfra(c echo.Context) error {

	// Input
	req := &RecommendInfraRequest{}
	if err := c.Bind(req); err != nil {
		return err
	}

	fmt.Print(req)
	fmt.Print(req.Compute)
	fmt.Print(req.Network)
	fmt.Print(req.GPU)

	res := &RecommendInfraResponse{}
	// Process

	// Ouput

	// if err != nil {
	// 	common.CBLog.Error(err)
	// 	mapA := map[string]string{"message": err.Error()}
	// 	return c.JSON(http.StatusInternalServerError, &mapA)
	// }

	return c.JSON(http.StatusOK, res)

}

Execute swag init

I encountered the below message.

make swag

2023/12/10 23:07:05 Error parsing type definition 'controller.RecommendInfraRequest': : cannot find type definition: infra.Infra
2023/12/10 23:07:05 Skipping 'controller.RecommendInfraRequest', recursion detected.

Click to see full message

make swag
cd pkg/ && make swag
make[1]: Entering directory '/mnt/d/dev/cloud-barista/cm-beetle/pkg'
~/go/bin/swag i -g ./api/rest/server/server.go -o ./api/rest/docs
2023/12/10 23:07:05 Generate swagger docs....
2023/12/10 23:07:05 Generate general API Info, search dir:./
2023/12/10 23:07:05 warning: failed to get package name in dir: ./, error: execute go list command, exit status 1, stdout:, stderr:no Go files in /mnt/d/dev/cloud-barista/cm-beetle/pkg
2023/12/10 23:07:05 Generating common.ConfigInfo
2023/12/10 23:07:05 Generating common.SimpleMsg
2023/12/10 23:07:05 Generating common.RestGetAllConfigResponse
2023/12/10 23:07:05 Generating common.ConfigReq
2023/12/10 23:07:05 Generating common.JSONResult
2023/12/10 23:07:05 Generating common.RestGetAllNsResponse
2023/12/10 23:07:05 Generating common.NsInfo
2023/12/10 23:07:05 Generating common.IdList
2023/12/10 23:07:05 Generating common.NsReq
2023/12/10 23:07:05 Generating controller.MigrateInfraRequest
2023/12/10 23:07:05 Generating model.TbMcisDynamicReq
2023/12/10 23:07:05 Generating model.TbVmDynamicReq
2023/12/10 23:07:05 Generating controller.MigrateInfraResponse
2023/12/10 23:07:05 Generating model.TbMcisInfo
2023/12/10 23:07:05 Generating model.StatusCountInfo
2023/12/10 23:07:05 Generating model.TbVmInfo
2023/12/10 23:07:05 Generating common.GeoLocation
2023/12/10 23:07:05 Generating model.RegionInfo
2023/12/10 23:07:05 Generating common.ConnConfig
2023/12/10 23:07:05 Generating model.SpiderVMInfo
2023/12/10 23:07:05 Generating common.IID
2023/12/10 23:07:05 Generating common.KeyValue
2023/12/10 23:07:05 Generating controller.MigrateNetworkRequest
2023/12/10 23:07:05 Generating model.DummyNetwork
2023/12/10 23:07:05 Generating model.Network
2023/12/10 23:07:05 Generating model.Subnet
2023/12/10 23:07:05 Generating controller.MigrateNetworkResponse
2023/12/10 23:07:05 Generating controller.MigrateStorageRequest
2023/12/10 23:07:05 Generating model.DummyStorage
2023/12/10 23:07:05 Generating model.Storage
2023/12/10 23:07:05 Generating controller.MigrateStorageResponse
2023/12/10 23:07:05 Generating controller.MigrateInstanceRequest
2023/12/10 23:07:05 Generating model.DummyInstance
2023/12/10 23:07:05 Generating model.Instance
2023/12/10 23:07:05 Generating controller.MigrateInstanceResponse
2023/12/10 23:07:05 Generating controller.RecommendInfraRequest
2023/12/10 23:07:05 Error parsing type definition 'controller.RecommendInfraRequest': : cannot find type definition: infra.Infra
2023/12/10 23:07:05 Skipping 'controller.RecommendInfraRequest', recursion detected.
2023/12/10 23:07:05 Generating controller.RecommendInfraResponse
2023/12/10 23:07:05 Generating controller.Infrastructure
2023/12/10 23:07:05 Generating controller.GetUsersResponse
2023/12/10 23:07:05 Generating model.MyUser
2023/12/10 23:07:05 Generating controller.GetUserResponse
2023/12/10 23:07:05 Generating controller.CreateUserRequest
2023/12/10 23:07:05 Generating controller.UpdateUserRequest
2023/12/10 23:07:05 Generating controller.UpdateUserResponse
2023/12/10 23:07:05 Generating controller.PatchUserRequest
2023/12/10 23:07:05 Generating controller.PatchUserResponse
2023/12/10 23:07:05 create docs.go at api/rest/docs/docs.go
2023/12/10 23:07:05 create swagger.json at api/rest/docs/swagger.json
2023/12/10 23:07:05 create swagger.yaml at api/rest/docs/swagger.yaml
make[1]: Leaving directory '/mnt/d/dev/cloud-barista/cm-beetle/pkg'

Error: fetch is not set. Please pass a fetch implementation as new Octokit({ request: { fetch }}). Learn more at https://github.com/octokit/octokit.js/#fetch-missing

Anything else we need to know?
: This is also issued on https://github.com/pascalgn/automerge-action, and the related PR is opened.