klothoplatform / klotho Goto Github PK

The id contains the path#id so our validation fails for gateways

We provided a way to create CDN for Gateway and Static unit and have them bundled. However if they are bundled they will always default to the defaultCacheBehavior. We likely want to add a Gateways routes onto the origin in the compiler and create createGatewayOrderedCacheBehaviors the parameter of type aws.types.input.cloudfront.DistributionOrderedCacheBehavior so we can bundle multiple things throughout the CDN.

If we can also solve this for static units we should, but currently we dont have routes for that so initial reaction is only allow one bucket in a distro and have it be the catch all default (/*) behavior

We don't actually use the kind value at the moment for secrets so I set it to secrets_manager in the doc since that made more sense. https://github.com/klothoplatform/docs/pull/145 is the docs change so we should make them line up

The Python expose plugin is substituting the capability in place of the FastAPI app variable name. This causes the dispatcher to attempt to invoke API calls against the wrong variable and causes requests to fail.

We currently create the payloads bucket in the CloudCCLib constructor (synchronous) and its name includes an accountId prefix coming from a promise/output, which is supplied asynchronously. This seems to work for the actual bucket name in s3, but results in a blank pulumi resource name (URN).

Ideally, we shouldn't be creating resources or even invoking async functions in the constructor. Moving this functionality into an async init() method should resolve this (seems to work for me now when tested locally).

today we only support read/write. We should probably support delete object, etc like we do in node

as we sanitize resource names for IaC it will be a lot simpler if we have restrictions on the appname since we are going to use it in every resource name

If someone wants to nest dockerfiles to fit their project structure, we should allow it. This is for dockerfile overrides though and not our autogenned dockerfiles. Our autogenned should still be put at root path since thats where we understand the context.

Longer term, users should be able to create any number of subnets of various types, for now, we create public/private and the goal is to pick between these 2.

If it's in public, we need to assign it a public IP

One of the things I hear a lot from practitioners is the requirement to have "ownership" of cloud resources. Some are using IaC tags, Some runtime tags, and Some innovative ones are using CI logs to trace back the owner.

Ownership is required when you want to know, "who owns the resource that costs so much/ throws errors / has security misconfigs?". Baking tags into the resources helps in those pains.

At bridgecrew we used this util named yor to bake in "ownership" from git log data into IaC.

Just an idea on a tagging feature

Our topology uses the cloudResource.Type() which is never synced with the type in config. This is done for exec units by explicitly setting the type in code, but we should probably have a better solution than hardcoding it

Language: Python
Provider: AWS
Version: v0.5.19-darwin-amd64

I created a simple webhook and used the expose + persist capabilities - but Klotho is saying it didn't create any resources for the expose:

Source

as we sanitize resource names for IaC it will be a lot simpler if we have restrictions on the appname since we are going to use it in every resource name

same as #31

I noticed that the README doesn't have any directions to install Klotho on Windows.
I also noticed that there is a Windows binary for Klotho, in the Releases section on GitHub.
Most developer tools on Windows are packaged up for distribution with the Scoop package manager.

Could you guys add a Scoop package manifest file, to install Klotho via Scoop CLI?
It's basically like MacOS Homebrew, but for Windows.

https://github.com/ScoopInstaller/Scoop

Would be interested in trying this out with my team if it supported Django. We are on AWS using a fargate ECS cluster to host our Django instance(s) with auto-scaling. Django connects to our RDS database instance with credentials pulled from secrets manager.

We have all of the IAC code completed to launch a single exec backed Websocket API Gateway, but i wasnt quite sure how to make them useable without parsing source code for middleware or other libraries. I dont want to throw away the code so its in pr, but this would be to make it to the point where we can surface it via config.

Today we can use type Alb to support websockets rather than api gateway

Today, --version print out:

• the big green KLOTHO header to stdout
• something like Version: v0.0.0-darwin-arm64 to stderr

I think a better output would be:

• klotho v0.0.0-darwin-arm64 to stdout
  • or a variant like klotho v0.0.0 darwin/arm64
  • no big green KLOTHO
• nothing to stderr

This would make it easier to get the version in bash scripts, etc.

For comparison:

$ python3 --version
Python 3.10.8

$ pulumi version
v3.50.2

$ go version
go version go1.19.2 darwin/arm64

$ bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin22)
Copyright (C) 2007 Free Software Foundation, Inc.

$ jq --version
jq-1.6

$ git --version
git version 2.37.1 (Apple Git-137.1)

$ brew --version
Homebrew 3.6.17
Homebrew/homebrew-core (git revision 751c1148c06; last commit 2023-01-03)
Homebrew/homebrew-cask (git revision a7b94350a7; last commit 2023-01-04)

$ vim --version
VIM - Vi IMproved 9.0 (2022 Jun 28, compiled Sep 15 2022 18:56:17)
macOS version - arm64
Included patches: 1-472
Compiled by Homebrew
Huge version with MacVim GUI.  Features included (+) or not (-):
+acl               +find_in_path      +mouse_xterm       +tcl
+arabic            +float             +multi_byte        +termguicolors
... (lots of other stuff. boy this one's noisy!)

$ emacs --version
GNU Emacs 26.3
Copyright (C) 2019 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

All of these print only to stdout, except that pulumi prints the version to stdout but also a warning about needing to do brew upgrade to stderr. And emacs, but that's to be expected given my version of it.

same as https://github.com/klothoplatform/klotho-history/issues/578 and #30, just for static units

we completely removed these tests but need them back in

Since we include certain yaml/dockerfile/other files as source files we may sometimes create an exec unit with no executable. This will cause us to fail on pulumi up time because we cant create a docker lambda

Keep the payload bucket name for users so they wont have to migrate data. Migrate just the internal Klotho data.

We werent properly outputting FileRefs if we arent running on . (klotho . --app blah). We made a change for search the deck to include the config path, which breaks FileRefs if they want to reference something not in the klotho project.

Gordon brought up the example: ie, a third-party plugin could do something like unit.Add(&core.FileRef{FPath: "/etc/passwd"})

We should be able to support this for other plugins

helm can support taking in a list of values files and concatenating the values. Today we only take a single file in to render templates, but we should be able to support what helm supports

We want to support routing to the same pod for the same client (sticky sessions) in the K8s world. For now this is just Pulumi pass-through settings, eventually we'll handle this with k8s manifest/helm

Need to investigate why.

today we only pass back environment variables for lambda. Assuming that we can do this uniform for all exec unit types we should add this to generic config (not pulumi params) and then write these env vars as core.EnvironmentVariables so its reusable across platforms and IaCs

use filepath.ToSlash similar to how we do in #30. Theres some other cleanup in terms of the functions in this pr that we can translate over also

support ACM and the ability to import cert data into ACM to use cnames with cloudfront

This is a meta-flag which turns on or off specific flags that are more aligned with CI usage.

There's a few that aren't properly namespace'd. Do a sweep through all of the resources, but some I know at time of writing:

• API Gateway
• Log groups

We should also consider having a santize method purely for the length of the resource to ensure we will be able to create it.

we should be able to use the topology to see if these are needed. The recent ALB support cant support fargate until we do so, so that is also a definition of done (Testing ALB + Fargate ECS)

Persist + Type (dynamo/redis)

Support ASP.NET web API running in AWS Lambda.

Static Unit creates a bucket for each site using the following bucket naming format: static-website-<UNIT_ID>. Since S3 bucket names are globally unique, deployment may fail if a bucket of the same name already exists.

A more appropriately namespaced alternative format is the following:
<ACCOUNT_ID>-<APP_NAME>-<UNIT_ID>

https://catalog.workshops.aws/eks-immersionday/en-US/kubecost/install-kubecost