Giter Club home page Giter Club logo

cloud-provision's Introduction

cloud-provision

Get cloud instances with your favourite software pre-loaded

This Golang package can be used to provision cloud hosts using a simple CRUD-style API along with a cloud-init user-data script. It could be used to automate anything from k3s clusters, to blogs, or CI runners. We use it to create the cheapest possible hosts in the cloud with a public IP address.

provision.go

type Provisioner interface {
	Provision(BasicHost) (*ProvisionedHost, error)
	Status(id string) (*ProvisionedHost, error)
	Delete(HostDeleteRequest) error
}

Where is this package used?

Feel free to send a PR to add your project

This package is used by:

  • inletsctl - Go CLI to create/delete exit-servers and inlets/-pro tunnels
  • inlets-operator - Kubernetes operator to automate exit-servers and inlets/-pro tunnels via CRDs and Service definitions

Try an example program

The tester app takes in a cloud-config file and provisions a host with Nginx - polling until it is ready for access.

#cloud-config
packages:
  - nginx
runcmd:
  - systemctl enable nginx
  - systemctl start nginx

See the example here: Tester app

Rules for adding a new provisioner

The first rule about the provision package is that we don't do SSH. Key management and statefulness are out of scope. Cheap servers should be treated like cattle, not pets. ssh may well be enabled by default, but is out of scope for management. For instance, with DigitalOcean, you can get a root password if you need to log in. Configure as much as you can via cloud-init / user-data.

  • Use the Ubuntu 16.04 LTS image
  • Select the cheapest plan and update the README with the estimated monthly cost
  • You need to open all ports on any firewall rules since the inlets client advertises its ports at runtime
  • This API is event-driven and is expected to use polling from the Kubernetes Operator or inletsctl CLI, not callbacks or waits
  • Do not use any wait or blocking calls, all API calls should return ideally within < 1s
  • Document how you chose any image or configuration, so that the code can be maintained, so that means links and // comments
  • All provisioning code should detect the correct "status" for the provider and set the standard known value
  • Always show your testing in PRs.

Finally please add an example to the documentation for your provider in the inlets/docs repo.

If you would like to add a provider please propose it with an Issue, to make sure that the community are happy to accept the change, and to maintain the code on an ongoing basis.

Maintainers for each provider

cloud-provision's People

Contributors

alexellis avatar frezbo avatar jsiebens avatar rgee0 avatar welteki avatar zechen0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

cloud-provision's Issues

Equinix Metal Provisiner Errors

Hi,

i get following errors, when using the Equinix Metal Provisioner:

  1. Default Region am1 does not exist
Using provider: equinix-metal
Requesting host: reverent-khayyam0 in ams1, from equinix-metal
2021/07/27 06:31:29 [DEBUG] POST https://api.equinix.com/metal/v1/projects/xx/devices
POST https://api.equinix.com/metal/v1/projects/xxx/devices: 422 ams1 is not a valid facility 
  1. You can not create an Equinix Metal Server without an SSH Key anymore:
Using provider: equinix-metal
Requesting host: magical-johnson8 in fr2, from equinix-metal
2021/07/27 06:36:30 [DEBUG] POST https://api.equinix.com/metal/v1/projects/xxx/devices
POST https://api.equinix.com/metal/v1/projects/xxx/devices: 422 must have at least one SSH key or explicitly send no_ssh_keys option 

See -> https://feedback.equinixmetal.com/platform/p/allow-to-deploy-servers-without-ssh-key
And new API for the no_ssh_keys option -> https://metal.equinix.com/developers/api/devices/#devices-createdevice

For issue 2 we need to update the Go lib -> 0.17.0

GCE: the iam.serviceAccountUser role is unused

The GCE provisioner requires a GCP service account with two roles:

  • roles/compute.admin
  • roles/iam.serviceAccountUser

The first role makes sense, but the second one was a surprise to me. Looking at gce.go, I discovered that the VM is created with a "service account VM" (a service account that is mounted into the VM and allows for GCP API calls):

ServiceAccounts: []*compute.ServiceAccount{
{
Email: "default",
Scopes: []string{
compute.ComputeScope,
},
},
},

Should we drop this mounted service account and update the documentation (see sections inlets-operator and inletsctl) to only require roles/compute.admin?

[Suggestion] Remove inlets specific references from the provisioners

At this moment, many provisioners have some specific, hard-coded references to inlets. Like tags or firewall rules.
Referring to this repository: https://github.com/inlets/provision-cloudinit-go, removing those references and making the provisioners more generic, they can be used as a library by others.

Expected Behaviour

Most provisioners create cloud host with hard-coded inlets references, like tags.

Current Behaviour

The provisioners can be used by others without creating inlets tags.

Possible Solution

Pass the inlets references to the provisioners as arguments. I'm aware this can be a tedious task, as there are many provisioners and they all have a target cloud specific implementation.

Steps to Reproduce (for bugs)

N/A

Context

Playing around with the provisioners as a library, I've noticed that the cloud hosts are tagged with "inlets"., while I was creating other, non-inlets relared hosts.

Your Environment

  • inlets version inlets --version
    N/A
  • Docker/Kubernetes version docker version / kubectl version:
    N/A
  • Operating System and version (e.g. Linux, Windows, MacOS):
    N/A
  • Link to your project or a code example to reproduce issue:
    https://github.com/inlets/provision-cloudinit-go

Linode: Remove Startupscripts

When deleting an exit sever on linode, we should remove the startupscript too.

If a sever can not be created, for whatever reasons we should also delete any orphan startupscripts.
image

Create and assign a static IP when using GCE

Thanks to @richardcase for reporting this - I don't know if it's new behaviour or if we never noticed it before, but GCE VMs now get an ephemeral IP instead of a stable one.

I've updated the docs with instructions on what to do to work around this, but it'd be a far better user experience, if the provisioning library did this for customers.

The IPs cost 2.88USD / mo, unless they are orphaned in which case the cost increases, so it'd be important to track it and remove it when the VM was removed.

I also did some research into whether a stable DNS entry was created (like you get with EC2 VMs on AWS, but there isn't one)

https://docs.inlets.dev/reference/inletsctl/#example-usage-with-aws-ec2-temporary-credentials

Scaleway STARDUST1-S Instance Type Support

Scaleway now has a STARDUST1-S Instance Type "STARDUST1-S"
1 vCPU, 1 GB of RAM, 1 IPv4 address, 10GB storage, up-to 100Mbps Bandwidth. โ‚ฌ0.0025/hour.

Having support for this would be by far the lowest cost option to use with inlets for European customers. It would be nice to add support for Amsterdam regions as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.