hystax / optscale Goto Github PK
View Code? Open in Web Editor NEWFinOps and cloud cost optimization tool. Supports AWS, Azure, GCP, Alibaba Cloud and Kubernetes.
Home Page: https://hystax.com
License: Apache License 2.0
FinOps and cloud cost optimization tool. Supports AWS, Azure, GCP, Alibaba Cloud and Kubernetes.
Home Page: https://hystax.com
License: Apache License 2.0
@sd-hystax May I ask how the bill data is processed, what is the logic of importing, cleaning, and aggregation, and how to operate repetitive data? Is there any relevant documentation? Taking AWS as an example, the first import of bills into mongo raw_expenses, the first How to update and insert in the second time? How to clean and aggregate the original data into the mongo resources and clickhouse expenses tables
Is there any detailed documentation on this system? The installation instructions are very minimal. And I have not yet found any instructions on how to use it after its installed. Or even how to log in.
I have created a single EC2 instance that meets the requirements stated.
I SSHed into the instance and started running the install commands.
This command fails:
ansible-playbook -e "ansible_ssh_user=ubuntu" -k -K -i "10.70.2.60," ansible/k8s-master.yaml
I am using ubuntu which has no password.
I am running this on the instance itself so why does it need to SSH.
From the documentation, I had assumed all the commands listed were to be directly on the instance, do I have that wrong?
Do I need to create a specific users/pass combination on the target server and then run this command from a separate server?
Please let me know.
secrets:
cluster: fc83d31-461d-44c5-b4d5-41a32d6c36a1
agent: 355d14dd-73b1-4834-aa0e-ccf10849c496
service_credentials:
aws:
access_key_id:
secret_access_key:
azure:
client_id:
tenant:
secret:
subscription_id:
alibaba:
access_key_id:
secret_access_key:
encryption_salt: myencypt10ns@lt
smtp:
server:
email:
port:
password:
slacker:
slack_signing_secret:
slack_client_id:
slack_client_secret:
google_calendar_service:
access_key:
type: service_account
project_id: optscale
private_key_id: eeee000
private_key: |
-----BEGIN PRIVATE KEY-----
-----END PRIVATE KEY-----
client_email: [email protected]
client_id: ""
auth_uri: https://accounts.google.com/o/oauth2/auth
token_uri: https://oauth2.googleapis.com/token
auth_provider_x509_cert_url: https://www.googleapis.com/oauth2/v1/certs
client_x509_cert_url: https://www.googleapis.com/robot/v1/metadata/x509/[email protected]
encryption_key: fffffxdddeadb33f
auth:
google_oauth_client_id: ""
microsoft_oauth_client_id: ""
accounts.google.comaccounts.google.com
Sign in - Google Accounts
'+.P(.N(
accounts.google.comaccounts.google.com
Google Cloud Platform
Google Cloud Platform lets you build, deploy, and scale applications, websites, and services on the same infrastructure as Google.
The instruction say to modify overlay/user_template.yml
I want to set this up for AWS only.
What AWS permissions does optscale need?
What would go in here?
pricing_data:
dataset_name: pricing_dataset
table_name: cloud_pricing_export
project_id: yourprojectid
How much, if any of these items can be configured after the install?
prune_node_modules.sh
.env file
Hello,
We are using OVH as secondary cloud provider.
Optscale do not take in charge this Cloud provider.
So:
If it's not the case, do you have any advise / starting point to integrate a new provider ?
It would help to have commit/branch example of a cloud provider integration ? Is-it available somewhere ?
Thanks,
Thomas.
TASK [common : Install docker-ce] ********************************************************************************************************************************************************************
fatal: [172.31.29.83]: FAILED! => {"cache_update_time": 1677632144, "cache_updated": true, "changed": false, "msg": "'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install 'docker-ce=5:19.03.'' failed: E: Version '5:19.03.' for 'docker-ce' was not found\n", "rc": 100, "stderr": "E: Version '5:19.03.' for 'docker-ce' was not found\n", "stderr_lines": ["E: Version '5:19.03.' for 'docker-ce' was not found"], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nPackage docker-ce is not available, but is referred to by another package.\nThis may mean that the package is missing, has been obsoleted, or\nis only available from another source\nHowever the following packages replace it:\n docker-ce-cli\n\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Package docker-ce is not available, but is referred to by another package.", "This may mean that the package is missing, has been obsoleted, or", "is only available from another source", "However the following packages replace it:", " docker-ce-cli", ""]}
PLAY RECAP *******************************************************************************************************************************************************************************************
172.31.29.83 : ok=10 changed=5 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
I deployed optscale on my server using it's git repository in this process it send unauthorized request to other anonymous servers. I tried two times both time i get IT regulation mail regarding security breach.
How can i solve this. And can i get a demo of how to attach aws root user as it doesn't take root acoount.
Hi, I want to connect GCP data source follow your docs: https://hystax.com/documentation/optscale/e2e_guides/e2e_gcp.html
I have some question:
I'm having issues deploying optscale on a VM in Oracle Cloud. Can you help?
`TASK [k8s-configure : Install nginx ingress controller] *******************************************************************************************************************************
fatal: [10.0.0.7]: FAILED! => {"changed": true, "cmd": "helm upgrade --install ngingress stable/nginx-ingress --set rbac.create=true --set controller.hostNetwork=true --set controller.extraArgs.default-ssl-certificate=default/defaultcert --set controller.kind=DaemonSet", "delta": "0:00:02.113905", "end": "2023-04-17 16:09:18.639070", "msg": "non-zero return code", "rc": 1, "start": "2023-04-17 16:09:16.525165", "stderr": "WARNING: This chart is deprecated\nError: UPGRADE FAILED: Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/configmaps?labelSelector=NAME%!D(MISSING)ngingress%!C(MISSING)OWNER%!D(MISSING)TILLER%!C(MISSING)STATUS%!D(MISSING)DEPLOYED\": dial tcp 10.96.0.1:443: connect: no route to host", "stderr_lines": ["WARNING: This chart is deprecated", "Error: UPGRADE FAILED: Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/configmaps?labelSelector=NAME%!D(MISSING)ngingress%!C(MISSING)OWNER%!D(MISSING)TILLER%!C(MISSING)STATUS%!D(MISSING)DEPLOYED": dial tcp 10.96.0.1:443: connect: no route to host"], "stdout": "UPGRADE FAILED\nError: Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/configmaps?labelSelector=NAME%!D(MISSING)ngingress%!C(MISSING)OWNER%!D(MISSING)TILLER%!C(MISSING)STATUS%!D(MISSING)DEPLOYED\": dial tcp 10.96.0.1:443: connect: no route to host", "stdout_lines": ["UPGRADE FAILED", "Error: Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/configmaps?labelSelector=NAME%!D(MISSING)ngingress%!C(MISSING)OWNER%!D(MISSING)TILLER%!C(MISSING)STATUS%!D(MISSING)DEPLOYED": dial tcp 10.96.0.1:443: connect: no route to host"]}
NO MORE HOSTS LEFT ********************************************************************************************************************************************************************
PLAY RECAP ****************************************************************************************************************************************************************************
10.0.0.7 : ok=68 changed=59 unreachable=0 failed=1 skipped=20 rescued=0 ignored=0`
@maxb-hystax Does optscale support Huawei Cloud?
Hello,
I've deployed OptScale and now want to add a data source, but my company has a security policy to not allow creation of static IAM users. Is there a possibility to add the data source in any other way, like for example using EC2 IAM role?
There is a document describing the manual import of CUR: https://hystax.com/documentation/optscale/e2e_guides/e2e_aws_root_manual_import.html
which is a possible solution, but it also needs a data source to be created with AWS credentials first.
Thanks in advance,
Paweł
I can't find where to setup the SSO with Microsoft . Is there any updated documentation? Does this function works for free accounts?
Hello everyone!
I try to find some documentation on "how to install", but I didn't find it. I want to test in my environment (EC2 Instance for example).
Someone can help me in this case?
Thanks
The line in runkube.py
that creates a Docker client expects the daemon to be exposed on a local TCP port:
cl = DockerClient(base_url='tcp://{0}:{1}'.format(node, self.dport))
However, the default on most operating systems is a socket, and exposing the TCP port is advanced. This can be addressed by simply referencing the socket address instead:
cl = DockerClient(base_url='unix://var/run/docker.sock')
Ideally, it would check for both and use the first one it finds.
I am using Ubuntu 20.04 and k8s version 1.17. Is this version f k8s still supported?
TASK [k8s-configure : Install nginx ingress controller] ****************************************************************************************************************fatal: [10.0.0.4]: FAILED! => {"changed": true, "cmd": "helm upgrade --install ngingress stable/nginx-ingress --set rbac.create=true --set controller.hostNetwork=true --set controller.extraArgs.default-ssl-certificate=default/defaultcert --set controller.kind=DaemonSet", "delta": "0:10:02.374654", "end": "2023-04-12 18:00:04.699181", "msg": "non-zero return code", "rc": 1, "start": "2023-04-12 17:50:02.324527", "stderr": "E0412 17:50:04.692378 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:04 socat[138996] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:05.699383 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:05 socat[138999] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:07.152980 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:07 socat[139035] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:09.921417 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:09 socat[139045] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:13.758916 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:13 socat[139068] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:21.097142 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:21 socat[139115] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:30.276485 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:30 socat[139186] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:50:48.744004 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:48 socat[139290] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:51:10.279392 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:51:10 socat[139402] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:51:44.926914 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:51:44 socat[139577] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:52:57.223321 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:52:57 socat[139981] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:54:30.179969 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:54:30 socat[140481] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nWARNING: This chart is deprecated\nE0412 17:55:04.693832 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:04 socat[140662] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:05.700376 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:05 socat[140664] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:07.626284 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:07 socat[140680] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:10.145650 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:10 socat[140700] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:14.077114 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:14 socat[140719] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:21.212406 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:21 socat[140756] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:31.452311 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:31 socat[140818] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:55:50.435016 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:50 socat[140919] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:56:12.910805 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:56:12 socat[141045] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:56:52.309720 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:56:52 socat[141258] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:58:06.678200 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:58:06 socat[141648] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nE0412 17:59:41.379685 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:59:41 socat[142173] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused\nError: UPGRADE FAILED: context deadline exceeded", "stderr_lines": ["E0412 17:50:04.692378 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:04 socat[138996] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:05.699383 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:05 socat[138999] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:07.152980 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:07 socat[139035] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:09.921417 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:09 socat[139045] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:13.758916 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:13 socat[139068] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:21.097142 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:21 socat[139115] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:30.276485 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:30 socat[139186] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:50:48.744004 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:50:48 socat[139290] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:51:10.279392 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:51:10 socat[139402] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:51:44.926914 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:51:44 socat[139577] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:52:57.223321 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:52:57 socat[139981] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:54:30.179969 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:54:30 socat[140481] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "WARNING: This chart is deprecated", "E0412 17:55:04.693832 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:04 socat[140662] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:05.700376 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:05 socat[140664] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:07.626284 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:07 socat[140680] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:10.145650 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:10 socat[140700] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:14.077114 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:14 socat[140719] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:21.212406 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:21 socat[140756] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:31.452311 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:31 socat[140818] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:55:50.435016 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:55:50 socat[140919] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:56:12.910805 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:56:12 socat[141045] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:56:52.309720 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:56:52 socat[141258] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:58:06.678200 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:58:06 socat[141648] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "E0412 17:59:41.379685 138988 portforward.go:400] an error occurred forwarding 42479 -> 44134: error forwarding port 44134 to pod 58b922e7943c6e9494360d5780fa390aa7ea142a4401c0e7af213f3178ebd009, uid : exit status 1: 2023/04/12 17:59:41 socat[142173] E connect(5, AF=2 10.0.0.4:44134, 16): Connection refused", "Error: UPGRADE FAILED: context deadline exceeded"], "stdout": "UPGRADE FAILED\nError: context deadline exceeded", "stdout_lines": ["UPGRADE FAILED", "Error: context deadline exceeded"]}
Hello,
There is a default account included by default:
user_root = User('root', type_root, '[email protected]', 'p@ssw0rd',
I tried & check through the API/UX.
This account is active, for security raison, that should definitively not be the case...
Thomas.
I spun up an Ubuntu 20.04.6 LTS machine in AWS and followed setup documentation found here. The cluster comes up without issue.
Following the documentation found here, however, after setting up credentials, I receive:
Object name : GCP Organization
Object type : report_import
Description : Billing data import for cloud account GCP Organization (#####################) failed: 'NoneType' object has no attribute 'lower'
I compared the costs of services obtained from AWS Cost Explorer and Cost Explorer from OptsCale. And the value obtained by Cost Explorer from OptsCale was $1200 higher compared to the value obtained from AWS Cost Explorer. I tried to see what could have happened (values in different services, a bug, error in counting, etc) but I didn't find any answer.
Attached is an Excel file showing the daily comparison of the values obtained and the total for each platform. The biggest difference is in the comparison of the RDS service, which is higher in the OptsCal Cost Explorer analysis of more than $800 compared to the AWS Cost Explorer.
analyze cost optscale aws cost explorer.xlsx
Hello,
I'm running a previous version of OPTSCALE.
I'm interested about the last changes (specifically the BI module !), but the number of changes look very high.
So, I have two possibilities:
What do you suggest ?
Are you planning new massive update in the coming month ? ( It may be interesting to wait little bit in this case)
Anyway, good work !
Thomas.
Can you please tell me, in which container are all the images saved? Cause when i am trying to do some changes in my repository, its not reflecting.
I am unable to edit the files and build docker image due to missing files.
In NGUI, .env, scripts/prune_node_modules.sh are missing
In rest-api, to pull pip libraries http://pypi.dts.loc:8080/pip/ is not working, rest_api_server/recommendation_cleanup_scripts file is missing, live_demo.json is missing
and many more...
If I change the currency, the symbol changes but the value remains remain. Any idea whats happening here?
Hi @maxb-hystax @sd-hystax @nexusriot @tguisep ,
I have configured Postfix SMTP on Ubuntu Linux OS and added SMTP configuration details in user-template.yaml. But It is not working. Anyone here can help me to solve this issue ASAP.
If I try to modify in the files for RnD purpose, it's not reflecting in the browser. And I don't know how to restart the application, can you help me with it?
The solution is currently being loaded. This has been the status for the past 5 hours, with nothing shown on the UI. How do I check if any kind of processing is happening. The pods show as Evicted, Running, Completed, Pending, ErrImageNeverPull, but is there a way to check the logs/database to see if anything has been dumped/processed?
I am unable to run the test cases. Its giving me auth_server or cloud_adapter module error, even though they are folders.
Is there a way to conduct local testing? Previously, it was very inconvenient to directly update the mirror image for verification.
Optscale can't read data billing from subscription Azure Enterprise Agreement. I'm suspicious that the Azure Enterprise Agreement subscription api is another API.
Hello,
I'm using the API to process statistics.
But on one of the algorithm, I have a gap (10%) between my results and that I should have.
Through the API, I try to retrieves the cost of each EC2 instance:
Eg:
Cost of march for EC2 from aws billing us-east-1 : 20k$
Cost of march for EC2 from the Optscale UI us-east-1 : 20k$ (all good)
My algorithm:
===> Get all the EC2 for the region:
cloud_resources = /restapi/v2/organizations/{self.organization_id}/cloud_resources
params: region: us-east-1
=====> for each **instance** in **cloud_resources**:
resource_cost = /restapi/v2/resources/{resource_id}/raw_expenses
params: start_date=1677628800, end_date=1680307199 (timestamp retrieved from UX call to be sure to have same timestamps for march)
total_cost.append(resource_cost['total_cost'])
=== Finally:
sum(total_cost) ==> 18k$
So, where are the 2$k differences ?
I discovered than some EC2 are missing in cloud_resources.
/restapi/v2/organizations/{self.organization_id}/cloud_resources
does-it return the current inventory only, or also all the previous resources (it may explain the gap) ? .
Thanks.
Hello,
By default, the GoogleMap api key is included in the docker image ( AIzaSyAVsTq9KVpPWuIIDzhhvwOFvpnqJeldmoQ ).
You should remove-it, allowing to show the google map, in development purpose, to avoid loading error.
Thomas.
How to get the root access in optscale?
If I try to modify in the files for RnD purpose, it's not reflecting in the browser. And I don't know how to restart the application, can you help me with it?
Hello,
I'm looking for the user/password of the nginx proxy for the elk stack and grafana. (and maybe after the kibana/grafana password)
I couldn't find it (source code, secrets, configmap...).
service/elkservice service/grafana
Thanks,
Thomas.
Hello,
Aws prices are stored in mongodb/restapi/aws_prices
Is-there a way to force a update of this collection ? (API call...).
I have some issues on it, it would help to diag.
Thanks.
Hi, Can you please tell me how to launch it, cause I followed all the steps in the deployment, next what needs to be done to launch the project/repository?
Hi @nexusriot @maxb-hystax @sd-hystax @tguisep @hx-nick
I am facing this issue. I have attached the error. Kindly help!
I can't find where to setup the SSO with Google. Is there any updated documentation? Does this function works for free accounts?
Hello,
I am trying to deploy an instance of optscale using the instructions provided here. I have noticed the following issues when ansible-playbook
command is run:
TASK [common : Copy tools requirements file] ************************************************************************************************************************************************************************************************
fatal: [10.0.15.214]: FAILED! => {"changed": false, "checksum": "ecae9ee210af06e1df285c88c933f53a583203df", "msg": "Destination /optscale/tools not writable"}
I overcame this by running chmod -R 0777 /optscale
TASK [k8s-configure : Create .kube] *********************************************************************************************************************************************************************************************************
fatal: [10.0.15.214]: FAILED! => {"changed": false, "gid": 1000, "group": "ubuntu", "mode": "0775", "msg": "chown failed: [Errno 1] Operation not permitted: b'/home/ubuntu/.kube'", "owner": "ubuntu", "path": "/home/ubuntu/.kube", "size": 4096, "state": "directory", "uid": 1000}
I have followed the instructions as per the page. What could be going wrong here? Anything that I have missed (that's not part of the deployment guide)?
It looks like the code doesn't have the dependencies (versions) correctly setup and hence I couldn't get it to run locally as is. However, running the deployment scripts work since the images are downloaded from the private docker repository of optscale. Is that by design or am I wrong in my assessment?
Hi @maxb-hystax
Do you have support to integrate optscal with keycloak , okta etc. ?
PLease let me know the config files for that.
Thanks
Mousumi
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.