Giter Club home page Giter Club logo

ocp4's Introduction

ironicbadger/ocp4

This repo contains code to deploy Openshift 4 for my homelab. It focuses on UPI with vSphere 6.7u3, a full write up is available on openshift.com.

May 2021 - The code here is working against 4.7.

Pre-reqs

On a Mac you will need to install a few packages via brew.

brew install jq watch gsed

Usage

Code for each OCP release lives on a numbered branch. The master branch represents the latest stable iteration and will likely be behind branches. In otherwords, check the number branches first before looking at master.

  • This repo requires Terraform 0.13 or newer
  • Install oc tools with ./install-oc-tools.sh --latest 4.6
  • This code use yamldecode - details here
  1. Create ~/.config/ocp/vsphere.yaml for yamldecode use, sample content:
alex@mooncake ~ % cat .config/ocp/vsphere.yaml
vsphere-user: [email protected]
vsphere-password: "123!"
vsphere-server: 192.168.1.240
vsphere-dc: ktzdc
vsphere-cluster: ktzcluster
  1. Configure DNS - https://blog.ktz.me/configure-unbound-dns-for-openshift-4/ - if using CoreDNS this is optional.
  2. Create install-config.yaml and ensure cluster_slug matches metadata: name: below.
apiVersion: v1
baseDomain: openshift.lab.int
compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0
controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 3
metadata:
  name: ocp4
platform:
  vsphere:
    vcenter: 192.168.1.240
    username: [email protected]
    password: supersecretpassword
    datacenter: ktzdc
    defaultDatastore: nvme
fips: false 
pullSecret: 'YOUR_PULL_SECRET'
sshKey: 'YOUR_SSH_PUBKEY'
  1. Customize clusters/lab/terraform.tfvars with any relevant configuration.

  2. Run make tfinit to initialise Terraform modules

  3. Run make lab to create the VMs and generate/install ignition configs

  4. Monitor install progress with make wait-for-bootstrap

  5. Check and approve pending CSRs with make get-csr and make approve-csr

  6. Run make bootstrap-complete to destroy the bootstrap VM

  7. Run make wait-for-install and wait for the cluster install to complete

  8. Enjoy!

ocp4's People

Contributors

cptmorgan-rh avatar ironicbadger avatar marcno avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

ocp4's Issues

Terraform Plan - error: invalid configuration:

Error: invalid configuration:
error at $.contents: invalid unit content: unexpected newline encountered while parsing option name

on ....\modules\ignition_coredns\main.tf line 6, in data "ignition_systemd_unit" "coredns":
6: data "ignition_systemd_unit" "coredns" {

Error: invalid configuration:
error at $.contents: invalid unit content: unexpected newline encountered while parsing option name

on ....\modules\ignition_haproxy\main.tf line 1, in data "ignition_systemd_unit" "haproxy":
1: data "ignition_systemd_unit" "haproxy" {

Not able to resolve DNS using coredns - even from coredns server directly

[core@ocp46-coredns ~]$ dig lb.ocp46.openshift.ako.test @10.206.43.254

; <<>> DiG 9.11.13-RedHat-9.11.13-6.el8_2.1 <<>> lb.ocp46.openshift.ako.test @10.206.43.254
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 42931
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;lb.ocp46.openshift.ako.test. IN A

;; AUTHORITY SECTION:
. 86385 IN SOA a.root-servers.net. nstld.verisign-grs.com. 2021051300 1800 900 604800 86400

;; Query time: 40 msec
;; SERVER: 10.206.43.254#53(10.206.43.254)
;; WHEN: Thu May 13 13:58:40 UTC 2021
;; MSG SIZE rcvd: 131

Terraform Apply invalid configuration

When launching the Terraform apply command, the following error is generated:

Error: invalid configuration:
error at $.contents: invalid unit content: unexpected newline encountered while parsing option name

on ../../modules/ignition_coredns/main.tf line 6, in data "ignition_systemd_unit" "coredns":
6: data "ignition_systemd_unit" "coredns" {

Error: invalid configuration:
error at $.contents: invalid unit content: unexpected newline encountered while parsing option name

on ../../modules/ignition_haproxy/main.tf line 1, in data "ignition_systemd_unit" "haproxy":
1: data "ignition_systemd_unit" "haproxy" {

Static IP setting cannot be set

IP cannot change using Afterburn, Do i need a OPNsense for OCP 4.6.1?

My vCenter is behind my physical router, and the router have DHCP function, it provided 172.29.1.XX subnet.

I used almost all the same config of the resource, the only config changes are the vCenter stuffs

after the terraform apply, all node still use the ip 172.29.1.XX, as so, the OCP cluster cannot start up.

  • Using previously-installed community-terraform-providers/ignition v2.1.1
  • Using previously-installed terraform-providers/ignition v1.2.1
  • Using previously-installed hashicorp/vsphere v1.24.2

==============install-config.yaml============================
'''
apiVersion: v1
baseDomain: openshift.lab.int
compute:

  • hyperthreading: Enabled
    name: worker
    replicas: 0
    controlPlane:
    hyperthreading: Enabled
    name: master
    replicas: 3
    metadata:
    name: ocp46
    platform:
    vsphere:
    vcenter: 172.29.1.206
    username: [email protected]
    password: ''
    datacenter: myDatacenter
    defaultDatastore: myDatastore
    fips: false
    pullSecret: '{"auths":{
    sshKey: 'ssh-rsa

==============4.6/terraform.tfvars==============================

Node IPs

loadbalancer_ip = "192.168.5.160"
coredns_ip = "192.168.5.169"
bootstrap_ip = "192.168.5.168"
master_ips = ["192.168.5.161", "192.168.5.162", "192.168.5.163"]
worker_ips = ["192.168.5.164", "192.168.5.165"]
#worker_ips = ["192.168.5.164", "192.168.5.165", "192.168.5.166"]

Cluster configuration

rhcos_template = "rhcos-4.6.1"
cluster_slug = "ocp46"
cluster_domain = "openshift.lab.int"
machine_cidr = "192.168.5.0/16"
netmask ="255.255.0.0"

DNS

local_dns = "192.168.5.169" # probably the same as coredns_ip
#public_dns = "192.168.1.254" # e.g. 1.1.1.1
#gateway = "192.168.1.254"
#public_dns = "192.168.1.1" # e.g. 1.1.1.1
#gateway = "192.168.1.1"
public_dns = "172.29.1.1" # e.g. 1.1.1.1
gateway = "172.29.1.1"

Ignition paths

Expects openshift-install create ignition-configs to have been run

probably via generate-configs.sh

bootstrap_ignition_path = "../../openshift/bootstrap.ign"
master_ignition_path = "../../openshift/master.ign"
worker_ignition_path = "../../openshift/worker.ign"
'''
===========4.6/variables.tf====================
'''
###########################

OCP Cluster Vars

variable "cluster_slug" {
type = string
}

variable "bootstrap_complete" {
type = string
default = "false"
}
################

VMware vars - unlikely to need to change between releases of OCP

variable "rhcos_template" {
type = string
}

provider "vsphere" {
user = yamldecode(file("/.config/ocp/vsphere.yaml"))["vsphere-user"]
password = yamldecode(file("
/.config/ocp/vsphere.yaml"))["vsphere-password"]
vsphere_server = yamldecode(file("~/.config/ocp/vsphere.yaml"))["vsphere-server"]

If you have a self-signed cert

allow_unverified_ssl = true
}

data "vsphere_datacenter" "dc" {
name = yamldecode(file("~/.config/ocp/vsphere.yaml"))["vsphere-dc"]
}

data "vsphere_compute_cluster" "cluster" {
name = yamldecode(file("~/.config/ocp/vsphere.yaml"))["vsphere-cluster"]
datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_network" "network" {
name = "VM Network"
datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_datastore" "nvme500" {
...
..
.
'''
=================ocp4/modules/rhcos-static/main.tf===============================
'''
locals {
ignition_encoded = "data:text/plain;charset=utf-8;base64,${base64encode(var.ignition)}"
}

data "ignition_file" "hostname" {

path = "/etc/hostname"
mode = "775"

content {
content = var.name
}
}

data "ignition_config" "vm" {

merge {
source = local.ignition_encoded
}
files = [
data.ignition_file.hostname.rendered
]
}

resource "vsphere_virtual_machine" "vm" {

name = var.name
resource_pool_id = var.resource_pool_id
datastore_id = var.datastore
num_cpus = var.num_cpu
memory = var.memory
memory_reservation = var.memory
guest_id = var.guest_id
folder = var.folder
enable_disk_uuid = "true"

wait_for_guest_net_timeout = "0"
wait_for_guest_net_routable = "false"

network_interface {
network_id = var.network
adapter_type = var.adapter_type
}

disk {
label = "disk0"
size = var.disk_size
thin_provisioned = var.thin_provisioned
}

clone {
template_uuid = var.template
}

extra_config = {
"guestinfo.ignition.config.data" = base64encode(data.ignition_config.vm.rendered)
"guestinfo.ignition.config.data.encoding" = "base64"

# configures the static IP
# https://www.man7.org/linux/man-pages/man7/dracut.cmdline.7.html
"guestinfo.afterburn.initrd.network-kargs" = "ip=${var.ipv4_address}::${var.gateway}:${var.netmask}:${var.name}:ens192:off:${var.dns_address}"

}
}
'''
=======================make 46 output=======================================
'''
[root@localhost ocp4]# make 46
./generate-configs.sh
INFO Consuming Install Config from target directory
WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings
INFO Manifests created in: manifests and openshift
INFO Consuming Master Machines from target directory
INFO Consuming OpenShift Install (Manifests) from target directory
INFO Consuming Worker Machines from target directory
INFO Consuming Openshift Manifests from target directory
INFO Consuming Common Manifests from target directory
INFO Ignition-Configs created in: . and auth
cd clusters/4.6; terraform apply -auto-approve
module.bootstrap[0].data.ignition_file.hostname: Refreshing state...
module.dns_vm[0].data.ignition_file.hostname: Refreshing state...
module.coredns.data.ignition_file.openshift_lab_db: Refreshing state...
module.worker[1].data.ignition_file.hostname: Refreshing state...
module.lb.data.ignition_systemd_unit.haproxy: Refreshing state...
module.lb.data.ignition_user.core: Refreshing state...
module.lb_vm[0].data.ignition_file.hostname: Refreshing state...
module.lb.data.ignition_file.haproxy: Refreshing state...
module.master[2].data.ignition_file.hostname: Refreshing state...
module.coredns.data.ignition_file.corefile: Refreshing state...
module.coredns.data.ignition_user.core: Refreshing state...
module.coredns.data.ignition_systemd_unit.coredns: Refreshing state...
module.master[0].data.ignition_file.hostname: Refreshing state...
module.worker[0].data.ignition_file.hostname: Refreshing state...
module.master[1].data.ignition_file.hostname: Refreshing state...
module.lb.data.ignition_config.lb: Refreshing state...
module.bootstrap[0].data.ignition_config.vm: Refreshing state...
module.worker[1].data.ignition_config.vm: Refreshing state...
module.worker[0].data.ignition_config.vm: Refreshing state...
module.coredns.data.ignition_config.coredns: Refreshing state...
module.master[2].data.ignition_config.vm: Refreshing state...
module.master[0].data.ignition_config.vm: Refreshing state...
module.master[1].data.ignition_config.vm: Refreshing state...
module.lb_vm[0].data.ignition_config.vm: Refreshing state...
module.dns_vm[0].data.ignition_config.vm: Refreshing state...
data.vsphere_datacenter.dc: Refreshing state...
data.vsphere_datastore.nvme500: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_datastore.spc500: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_datastore.mx1tb: Refreshing state...
data.vsphere_compute_cluster.cluster: Refreshing state...
module.master[1].vsphere_virtual_machine.vm: Creating...
module.master[0].vsphere_virtual_machine.vm: Creating...
module.worker[0].vsphere_virtual_machine.vm: Creating...
module.lb_vm[0].vsphere_virtual_machine.vm: Creating...
module.master[2].vsphere_virtual_machine.vm: Creating...
module.dns_vm[0].vsphere_virtual_machine.vm: Creating...
module.worker[1].vsphere_virtual_machine.vm: Creating...
module.bootstrap[0].vsphere_virtual_machine.vm: Creating...
module.master[1].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [10s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [20s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [30s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [40s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [50s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [1m0s elapsed]
module.master[1].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [1m10s elapsed]
module.worker[0].vsphere_virtual_machine.vm: Creation complete after 1m16s [id=42196866-972c-a82d-c19e-c090af927089]
module.master[1].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.master[0].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.lb_vm[0].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.master[2].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.worker[1].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.bootstrap[0].vsphere_virtual_machine.vm: Still creating... [1m20s elapsed]
module.dns_vm[0].vsphere_virtual_machine.vm: Creation complete after 1m20s [id=4219f36d-de4e-68b2-170a-90d7144c1cc4]
module.master[2].vsphere_virtual_machine.vm: Creation complete after 1m20s [id=4219bc8a-4d3a-3ef6-a80d-3fa2b3a376d8]
module.master[1].vsphere_virtual_machine.vm: Creation complete after 1m21s [id=4219bfc8-57d3-abed-201d-f5f53a0a5e58]
module.lb_vm[0].vsphere_virtual_machine.vm: Creation complete after 1m20s [id=421941d6-6d38-4fff-3f89-20e8048eff6a]
module.master[0].vsphere_virtual_machine.vm: Creation complete after 1m23s [id=42199085-d992-47b9-5f48-3ec52cbe2004]
module.worker[1].vsphere_virtual_machine.vm: Creation complete after 1m22s [id=421997b6-0ea2-78c6-baec-85e6085e59d0]
module.bootstrap[0].vsphere_virtual_machine.vm: Creation complete after 1m23s [id=4219c703-c669-4604-e0b6-c3be5da0fca0]

Warning: Interpolation-only expressions are deprecated

on main.tf line 10, in module "master":
10: folder = "${var.cluster_slug}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 5 more similar warnings elsewhere)

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.
'''

make tfinit doesn't work

tfinit:
        cd clusters/4.5; terraform init
        cd clusters/4.5-staticIPs; terraform init
        cd clusters/4.6; terraform init

while terraform init works in the clusters/4.6 folder it doesn't work for clusters/4.5 as it gives error

Initializing modules...
- lb in

Error: Unreadable module directory

Unable to evaluate directory symlink: lstat ../../modules/lb: no such file or
directory

There is no folder called lb in the modules directory

Where are machines getting cloned from? No permission to clone from cli

Part of the logs. Not sure what kind of permission is needed. Or am I missing some parameters

2020-11-16T11:07:05.183-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Cloning virtual machine "/CSPLAB/vm/Sandbox/zarin.ohiba/zo-ocp2/zo-ocp2-worker2"
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Resource pool found: resgroup-1411
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ValidateHost: no host supplied, nothing to do
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating full disk relocate spec list
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskRefreshOperation: Found controller "controller-1000" for device "disk-1000-0"
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices located: disk-1000-0
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices order after sort: disk-1000-0
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Current resource set: (key 0 at <new device>)
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Resource set order after sort: (key 0 at <new device>)
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating relocators for source disks
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator list: 
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator generation complete
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ExpandVirtualMachineCloneSpec: Clone spec prep complete
2020-11-16T11:07:05.186-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Cloning virtual machine "/CSPLAB/vm/Sandbox/zarin.ohiba/zo-ocp2/zo-ocp2-master2"
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Resource pool found: resgroup-1411
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ValidateHost: no host supplied, nothing to do
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating full disk relocate spec list
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskRefreshOperation: Found controller "controller-1000" for device "disk-1000-0"
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices located: disk-1000-0
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices order after sort: disk-1000-0
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Current resource set: (key 0 at <new device>)
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Resource set order after sort: (key 0 at <new device>)
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating relocators for source disks
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator list: 
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator generation complete
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ExpandVirtualMachineCloneSpec: Clone spec prep complete
2020-11-16T11:07:05.190-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Cloning virtual machine "/CSPLAB/vm/Sandbox/zarin.ohiba/zo-ocp2/zo-ocp2-bootstrap"
2020/11/16 11:07:05 [DEBUG] module.lb_vm[0].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.lb_vm[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.lb_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.lb_vm[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.lb_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.lb_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] vertex "module.lb_vm[0].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "module.lb_vm (close)" errored, so skipping
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Resource pool found: resgroup-1411
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ValidateHost: no host supplied, nothing to do
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating full disk relocate spec list
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskRefreshOperation: Found controller "controller-1000" for device "disk-1000-0"
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices located: disk-1000-0
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk devices order after sort: disk-1000-0
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Current resource set: (key 0 at <new device>)
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Resource set order after sort: (key 0 at <new device>)
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Generating relocators for source disks
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator list: 
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] DiskCloneRelocateOperation: Disk relocator generation complete
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] ExpandVirtualMachineCloneSpec: Clone spec prep complete
2020-11-16T11:07:05.197-0800 [DEBUG] plugin.terraform-provider-vsphere_v1.24.2_x4: 2020/11/16 11:07:05 [DEBUG] Cloning virtual machine "/CSPLAB/vm/Sandbox/zarin.ohiba/zo-ocp2/zo-ocp2-coredns"
2020/11/16 11:07:05 [DEBUG] module.master[0].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.master[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.master[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.master[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.master[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.master[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [DEBUG] module.worker[1].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] vertex "module.master[0].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.worker[1].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.worker[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.worker[1].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.worker[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.worker[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] vertex "module.worker[1].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "module.worker (close)" errored, so skipping
2020/11/16 11:07:05 [DEBUG] module.master[1].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.master[1].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.master[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.master[1].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.master[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.master[1].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] vertex "module.master[1].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "module.master (close)" errored, so skipping
2020/11/16 11:07:05 [DEBUG] module.bootstrap[0].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [DEBUG] module.dns_vm[0].vsphere_virtual_machine.vm: apply errored, but we're indicating that via the Error pointer rather than returning it: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.bootstrap[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.dns_vm[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.bootstrap[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.dns_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyProvisioners
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalApplyProvisioners: vsphere_virtual_machine.vm has no state, so skipping provisioners
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalMaybeTainted
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.bootstrap[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalMaybeTainted: module.dns_vm[0].vsphere_virtual_machine.vm encountered an error during creation, so it is now marked as tainted
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteState
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.bootstrap[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] EvalWriteState: removing state object for module.dns_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalIf
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalWriteDiff
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.bootstrap[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] vertex "module.bootstrap[0].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] eval: *terraform.EvalApplyPost
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "module.bootstrap (close)" errored, so skipping
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalApplyPost, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [ERROR] eval: *terraform.EvalSequence, err: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.
2020/11/16 11:07:05 [TRACE] [walkApply] Exiting eval tree: module.dns_vm[0].vsphere_virtual_machine.vm
2020/11/16 11:07:05 [TRACE] vertex "module.dns_vm[0].vsphere_virtual_machine.vm": visit complete
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "module.dns_vm (close)" errored, so skipping
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/hashicorp/vsphere\"] (close)" errored, so skipping
2020/11/16 11:07:05 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020/11/16 11:07:05 [TRACE] statemgr.Filesystem: not making a backup, because the new snapshot is identical to the old
2020/11/16 11:07:05 [TRACE] statemgr.Filesystem: no state changes since last snapshot
2020/11/16 11:07:05 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate

Error: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.

  on ../../modules/rhcos-static/main.tf line 25, in resource "vsphere_virtual_machine" "vm":
  25: resource "vsphere_virtual_machine" "vm" {



Error: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.

  on ../../modules/rhcos-static/main.tf line 25, in resource "vsphere_virtual_machine" "vm":
  25: resource "vsphere_virtual_machine" "vm" {



Error: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.

  on ../../modules/rhcos-static/main.tf line 25, in resource "vsphere_virtual_machine" "vm":
  25: resource "vsphere_virtual_machine" "vm" {



Error: error cloning virtual machine: ServerFaultCode: Permission to perform this operation was denied.

  on ../../modules/rhcos-static/main.tf line 25, in resource "vsphere_virtual_machine" "vm":
  25: resource "vsphere_virtual_machine" "vm" {


Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'

Regardless of which terraform version is used, the following error is generated during apply:

Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for
device '0'.
on ../../modules/rhcos-static/main.tf line 25, in resource "vsphere_virtual_machine" "vm":
25: resource "vsphere_virtual_machine" "vm" {

I'm trying to install OCP 4.7 on vSphere 7.0.3.
Terraform init output:
Initializing provider plugins...

  • Finding latest version of hashicorp/vsphere...
  • Finding latest version of terraform-providers/ignition...
  • Finding latest version of community-terraform-providers/ignition...
  • Installing hashicorp/vsphere v2.1.1...
  • Installed hashicorp/vsphere v2.1.1 (signed by HashiCorp)
  • Installing terraform-providers/ignition v1.2.1...
  • Installed terraform-providers/ignition v1.2.1 (signed by HashiCorp)
  • Installing community-terraform-providers/ignition v2.1.2...
  • Installed community-terraform-providers/ignition v2.1.2 (self-signed, key ID ...)

Terraform Plan failed with error message

"Error: invalid configuration:
error at $.contents: invalid unit content: unexpected newline encountered while parsing option name"

I have encounter this error, how should I address this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.