Comments (10)
I've managed to run this on Lambda with a minimal set of changes without issues, so closing this.
Would you mind share the steps with us so we can add them to the README?
from turborepo-remote-cache.
Hi. Lambda is something that we didn't explore yet. It would be nice to provide a guide, though.
from turborepo-remote-cache.
I've managed to run this on Lambda with a minimal set of changes without issues, so closing this.
from turborepo-remote-cache.
It's probably nicer to just run this locally on each developer's machine and proxy to S3. That way you don't have to worry as much about security vulnerabilities in the server code and compute costs.
from turborepo-remote-cache.
It's probably nicer to just run this locally on each developer's machine and proxy to S3. That way you don't have to worry as much about security vulnerabilities in the server code and compute costs.
Yes, but its hard to do this across teams – difficult to enforce a long-running process as a prerequisite for most dev environments, especially when most of them prefer their own systems/OSes/platforms/workflows.
from turborepo-remote-cache.
Ah, we have a docker-compose that runs services that people need for dev (mongodb, redis, and this), so it works for us anyway.
from turborepo-remote-cache.
I got it to run on lambda/apigw and I can hit it the api just fine but for some reason the turbo client doesn't love it. I don't get any errors but the turbo client gives me cache misses despite being able to curl and get the results. It does however push the resources to s3 🤷♂️
More investigation is needed
Here's the handler code
import awsLambdaFastify from '@fastify/aws-lambda'
import { createApp } from "turborepo-remote-cache/build/app";
const app = createApp({
trustProxy: true,
})
const proxy = awsLambdaFastify(app)
export const handler = proxy
For now I just wrote a script that starts the server and executes turbo locally
#!/usr/bin/env node
const { spawn } = require('child_process');
const { generateBinPath } = require('turbo/node-platform');
const AWS = require('aws-sdk');
async function setConfig() {
process.env.TURBO_TOKEN = 'SUPER_SECRET_VARIABLE';
process.env.TURBO_API = 'http://localhost:3000';
process.env.TURBO_TEAMID = 'team_ramrod;
process.env.STORAGE_PROVIDER = 's3';
process.env.STORAGE_PATH = 'fluent-turborepo-cache';
}
async function checkAwsConnection() {
var sts = new AWS.STS();
await sts.getCallerIdentity({}).promise();
}
async function exec() {
let useRemoteCache = false;
try {
await checkAwsConnection();
console.log('AWS Connection Successful using remote cache');
useRemoteCache = true;
await setConfig();
} catch (e) {
console.log('AWS connection could not be established using local cache');
}
if (useRemoteCache) {
const { createApp } = require('turborepo-remote-cache/build/app');
const turboCacheServer = createApp({ trustProxy: true, logger: undefined });
try {
await turboCacheServer.listen(3000, '127.0.0.1');
} catch (err) {
turboCacheServer.log.error(err);
process.exit(1);
}
}
const turbo = spawn(generateBinPath(), process.argv.slice(2), { stdio: 'inherit' });
turbo.on('close', (code) => {
process.exit(code);
});
}
exec();
from turborepo-remote-cache.
If you pass the option -vv
to turbo when you run it, you can get some extra debug logs that might help.
from turborepo-remote-cache.
@dobesv Yeah, -vv
and -vvv
show the GET and PUT operations working but it ultimately ends up in a cache miss 🤷♂️ .
I can see the items in s3. I can hit the lambda endpoint and GET them... but the turbo cache client always determines it a cache miss...
Same s3 bucket with a local server works no problem
from turborepo-remote-cache.
Hmm strange, must be something about the response that it is getting that isn't right. Maybe you could run wireshark or use a logging HTTP proxy and take a look at the actual HTTP responses and see if there's something weird going on there. Maybe your lambda is not returning things in the format you think.
from turborepo-remote-cache.
Related Issues (20)
- ERR_STREAM_PREMATURE_CLOSE on PUT Artifact with Azure Blob Storage implementation HOT 3
- Current state of main branch is not deployable to vercel.
- Lambda/API Gateway integration not working HOT 15
- apiUrl is not working HOT 2
- Dynamic authorization tokens HOT 3
- Help request - configured with s3, issue in the server log HOT 7
- Remote caching disabled since turbo 1.9.4 HOT 5
- Free Hosted Remote Cache discontinued
- Incompatible with Yarn/npm, or Node <18 HOT 1
- [email protected] is no longer supported
- fastify-warning is renamed to process-warning
- querystring API is considered Legacy
- Upgrade uuid to version 7 or higher HOT 1
- request has been deprecated
- azure-storage has been deprecated HOT 1
- Option to log to file HOT 1
- Support read-only mode to prevent unwanted cache update HOT 9
- Add an option to increase bodyLimit. HOT 1
- Digitalocean spaces support HOT 4
- Verify that this matches the official spec of Turbo's API HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from turborepo-remote-cache.