aragon / buidler-aragon Goto Github PK

To avoid publishing too many files (e.g. old source maps, bundles, or even artifacts), it would be nice if the publish command checked the stat of dist/ before going further in the build process.

For more context, see #121 (comment).

🚀 Feature

aragon run should be able to take a run-time configuration to deploy additional apps to support a template's dependencies.

The configuration should be smart enough to handle a user specifying their current directory's app, as well as apps that are pre-installed in aragen (with optional configuration to force these apps to be re-deployed).

It should also work when the current directory only includes a template.

Motivation

Currently, aragon run is meant only for spinning up a single app.

Lots of developers are building "suites" of apps, where they need to install more than one of their apps during development.

This issue summarizes the progress/status before v0.2.0 stable release

Goals

• Release v0.2.0 (Tentative date Wednesday)
• Communicate the release to the community (Tentative date Friday)

Required items

• Update react-boilerpalte to have the new publish task @0xGabi aragon/aragon-react-boilerplate#105 (waiting to release)
• Manually test new features @0xGabi
• Write comms for the 0.2.0 release @0xGabi (First draft)
• Support repos without an app dir #108
• Support publishing on different networks using a local key file and Frame @dapplion #99
• Sanity check IPFS API before doing a POST request on uploadReleaseToIpfs #101 @dapplion
• Review and decided to merge #82
• Review and merge #91 (we decided not to modify the arfiact.json)
• On publish, show URL with the API's own gateway instead of another gateway
• Debug mnemonic file loading

Nice to have items

• Clean start-task code
• Support repos without smart contract code
• Support script hooks and dao creation on other netowrks
• Improve deploy contract logic @dapplion
• Move chainId hardcoded (5555) to an environment or constants file with a comment

Previous Goals

This issue summarizes the progress/status towards a 0.2 stable release

• Fully functional publish command: must be able to release existing apps and community apps on localhost
• Offer primitive support for running secondary apps with a front end in development

Required items

• Review and merge #36
• Review and merge #79
• Read IPFS ignore patterns from files (.ipfsignore) and do not expose an option #92
• In publish, default managerAddres to account[0] #90
• Review environment, appName, appId deprecation and requirements with #91
• Send the publish tx by default with an option to configure it to be a dry run, printing the tx Data #90
• In publish, it would be nice to have these three cases covered #89:
  • Repo doesn't exist; deploys new contract and repo
  • Repo exists, do a minor or patch with just content changing
  • Repo exists, do a major with new contract and content

Out of scope

• Any new features on aragen / Snapshots
• Not using ganache server and using buidlerevm
• Interactive UI

:)

Brief aside, but we should be able to have prettier add ending commas in cases like this, to decrease the line diffs.

Originally posted by @sohkai in #90 (comment)

Add

"trailingComma": "es5"

To the .prettierrc file

Then run

node_modules/.bin/prettier --write "**/*.{ts,js}"

Warning: will generate a massive diff, do in a period of low development

🚀 Feature

🚀 Feature

• Support repos without smart contract code, only a buidler script. It could be a great way to create organization templates.
• Support script hooks and dao creation on other networks

🚀 Feature

It would be useful to be able to just generate the transaction data and output so the user can sign transactions using multi-sig wallets or other signing providers.

Have you read the Contributing Guidelines on issues?

yes

Motivation

We are testing a case where a gnosis multi-sig has permission to execute actions on behalf of an agent, the best way to do this would be to use the dao act command.

It also sometimes useful for troubleshooting issues with frame to be able to simply use a different signing provider.

Pitch

This should be a fairly trivial change that would be quite useful.

One way of doing it would be during CI using a docker image.

The app doesn't show until both builds are complete, and sometimes, if it takes long, the browser doesn't refresh after the builds are done. A manual refresh does the trick. When this happens, it feels a bit flaky.

Review non private functions and make sure their params and return params are all documented.

Brett's comments

The main pros I see in making it separate:

• Dedicated user interface
• Ability to move quickly (e.g. add features like remote-IPFS pinning, etherscan verification) without having to consider the other areas of development tooling
• Smaller codebases and surface areas
• Faster install times, as we can prompt the user to download the separate tooling when they're ready to publish (we already do this for the CLI in most A1-developed repos)

The cons I see are relatively minor:

• Another package in a monorepo somewhere (maintenance)
• May be annoying / unexpected

I would favor offering Infura or an Aragon open IPFS node to starting users and never starting a local node

The way I've been thinking about this is to either prompt the user (like we do in the current publishing experience of aragonCLI) and annoy them with a giant warning like "TO KEEP YOUR APP AVAILABLE, YOU MUST UPLOAD YOUR CONTENT TO A PINNING SERVER SOMEWHERE", or just warn the user but make this transparent for them at the beginning.

A1 can run an open pinning service just for publishes to aragonPM repos, but we'd have to design some sort of heurstic to avoid abuse as best as possible (e.g. limit to ~5mb, check content is actually on given repo version, pin only last X versions, etc.). There are some open threads about doing this type of light authentication but I think it's very reasonable for us to provide this for most beginning developers.

Executing transactions directly from a builder plugin run doesn't seem secure to me. I would prefer to output a prefilled link or transaction data that can be reviewed, verified and executed latter in a more transparent manner

Yes, absolutely. Having the ability to choose different "outputs" for the transaction is incredibly important and something we've been sorely needing in the CLI (e.g. "dry-run": aragon/aragon-cli#760).

Having these kinds of outputs will allow developers to choose how / when they want to publish the new version and also verify the the tx data against more hardened sources.

We have also been thinking about potential integrations with tools like Superblock's deployment pipeline, to move the deployment to a CI/CD pipeline as much as possible, and this can be seen as another "output".

(The way Superblocks works is it expects you to run a build on your own CI service, and then pipe "output" via a truffle provider / web3 provider to their service, where you can choose how you'd like to deploy.)

I wonder, do Aragon devs have the private keys with access to repos such as finance.aragonpm.eth locally in a laptop for the aragon-cli to have access to? Or what is the current procedure for sensitive apps such as this?

Haha, absolutely not. I am honestly surprised whenever this comes up; we would not be on mainnet if we had those private keys exposed in a laptop. Currently, devs publishing "officially endorsed" packages are required to at least secure their keys via a hardware wallet and use Frame to publish (see deployment permissions).

What we currently do is not ideal either, but plans to move it into a better structure have stalled because of problems with intent pathing for the publish task in the CLI (aragon/aragon-cli#1150).

This would be my ideal end state in the next 1-2 months as we make progress on the publishing process:

1. For each "publisher", create Aragon organizations with Agent apps
2. Migrate permissions associated with hardware wallets to the Agent
3. Publishing tool propagates IPFS content to a pinning service ahead of transaction publishing, either locally or on CI
   • This allows for zero-downtime deployments; we currently suffer at least 5min of downtime each time we publish, as we propagate IPFS content after the publish tx is confirmed
4. Developer broadcasts publish intent (locally, Superblocks, exposed key in CI/CD service, etc)
5. Using the organization, members vote and actually execute the aragonPM repo publish

For the developers, this flow should be as close to what is currently available; we'd just tell them they need to confirm the vote (if they have to) at a specific URL.

Migrate apm publishing from CLI to Buidler plugin.

Don't delete support in CLI yet. We want to let it sit in the plugin for a while before retracting the feature from the CLI.

The flatten code at code.sol includes all contracts in Import.sol where most are unrelated to the actual target Contract.

References

• buidler's glob to capture local contracts: https://github.com/nomiclabs/buidler/blob/64809c65da9da5eb9d8946458ec8fa8c2a7aa20c/packages/buidler-core/src/builtin-tasks/compile.ts#L73

🚀 Feature

It would be nice to have an easy way to run an app in development against a live organization that has it installed.

To be clear, this would mean that aragon run would not deploy an organization, but simply set up the client with the correct settings to connect to the correct environment and locally-served app.

A different command name could work as well, but this seems like a natural progress of aragon run:

1. Developer develops app locally
2. Developer publishes app
3. Users install app onto org
4. Developer continues to develop app, but will at times find it useful to connect to a live instance to debug / test

Depending on the implementation, changes could be made to the Aragon client to accept more configuration options to make this smoother, or perhaps not even requiring the configuration to be at build time (e.g. taking in query parameters to set configuration).

Motivation

Make it easier for developers to set up a development environment that has the Aragon client and their app configured correctly.

(E.g. compare with currently required steps for developing on aragon-apps)

🚀 Feature

Its useful to have contracts verified on etherscan but it can be a bit of a pain to go through that process so it often doesn't happen. When the CLI is used to deploy contracts we have all the necessary information to also verify them, so we should.

It may require a new configuration option for users to add an api key for etherscan.

Have you read the Contributing Guidelines on issues?

yes

Motivation

Verified contracts are much easier to inspect, and having more of aragon related contracts verified on etherscan would improve developer and user experiences.

Pitch

Depends on NomicFoundation/hardhat#502

These files need to be watched, but on change we only issue a warning to the user, asking to reset the start task.

Similar to the arapp.json, we should also check that an app has a correct manifest.json.

This should be an optional check, so the check should only be triggered if:

• You have a manifest.json
• You have an arapp.json

We're currently still defining some pieces of the manifest.json for the upcoming App Center, but a proper schema will be provided soon. Some of the items will be optional, and perhaps we could log warnings out if we detect those aren't available.

It depends on buidler releasing this feature.

Fix our current hack: https://github.com/aragon/buidler-aragon/blob/develop/src/tasks/start/backend/bases/deploy-bases.ts#L23

Right now, the plugin requires that we start aragen manually.

See #10 (comment)

Tasks

• Dynamically deploy subdomains when needed, so that users can specify ens names like ..aragonpm.eth
• Make sure that we still do some validation tho.
• Remove other environments from the arapp.json file, to keep things minimal.

It would be nice to have a way to run only the client and the app's frontend against a real chain, in case already-released apps want to test against real organizations and application state.

🚀 Feature

• Improve deploy contract code on publish-task
• Move chainId hardcoded (5555) to an environment or constants file
• Clean start-task code
• Double-check app scripts (serve, watch, sync-assets) if they exist and warn if they don't rather than make it a hard requirement.
• Add app scripts (serve, watch, sync-assets) as Aragon buidler cofiguration options.

Nice to have items that were found during the v0.2.0 review but didn't make it on time for the release:

• Misspelled stipIpfsPrefix #128 (comment)
• joinIpfsLocation return example type #128 (comment)
• Rename functionsOrSourceCode to include flatCode or similar for consistency #122 (comment)
• Add trailing coma to minimize diffs going forward #123
• Use utils/accounts getRootAccount consistently instead of calling web3 directly

If hooks are defined like this:

task ARAGON_GET_DEPLOYMENT_ARGS = return await config.aragon.hooks.getDeploymentArgs()

Not only hooks are available, but their entry points are extensible by devs developing other plugins on top of the Aragon plugin.

🐛 Bug Report

For example during aragon/aragon publication if we don't use the --only-content flag we recieve this cryptic error that is not very useful to understand what's happening:

The "path" argument must be of type string. Received type undefined

When we have an Aragon Client already in place, aragon-cli skips its initialization with the following message:

⚠ Server already listening at port 3000, skipped starting Aragon

This is not the case for Aragon buidler:

Error in plugin @aragon/buidler-aragon: Cannot start client. Port 3000 is in use.

Buidler allows to define internal tasks that devs are able to extend/tweak. Atm, the plugin's tasks use helper functions. Some of these could be converted into internal tasks, making them extensible. We'd need to think about which could be extended and how. We don't want to make everything extensible.

This issue summarizes the progress/status before v0.2.0 stable release

Goals

• Release v0.2.0 (Tentative date Wednesday)

• Communicate the release to the community (Tentative date Friday)

Required items

• Manually test new features @0xGabi

• Support publishing on different networks using a local key file and Frame @dapplion

• Write comms for the 0.2.0 release @0xGabi

• Sanity check IPFS API before doing a POST request on uploadReleaseToIpfs @dapplion

• Review and merge #91

Nice to have items

• Clean start-task code
• Improve deploy contract logic @dapplion
• Move chainId hardcoded (5555) to an environment or constants file with a comment

Currently ganache runs on port 8545 and there is no way to change it.

See also: aragon/aragon-cli#1219

It seems that the problem is that the mapping can resolve stuff at the first level, such as ~/typechain or ~/params but not at higher levels like ~/src/utils/arappUtils.

Weirdly enough, it works with ts-node in tests but not in the plugin sources.

💥 Proposal

Building up on Buidler plugin branch

The current watchers for the start command are:

• contracts/ -> re-deploy implementation contract
• app/ -> re-build front-end and refresh app server

I propose to add

• contracts/ -> re-build artifacts.json (add actions in the current watcher instance)
• arapp.json -> re-build artifacts.json
• manifest.json -> re-build / copy manifest.json

Instead of building the artifacts just once as part of the startFrontend() function

🚀 Feature

In addition to fetching and publishing with only the local IPFS node (we should always do this), provide optional integrations with cloud-based IPFS pinning services (e.g. Pinata, Temporal).

Publishing

Usually this just means taking in API keys via configuration variables, and pushing the files through an HTTP endpoint.

Pushing an entire folder may be a bit of a pain though, given that some of these cloud providers (e.g. Pinata) do not expose an IPFS-client compatible API. We may have to resort to adding to a pin queue.

Fetching

We could experiment with a time-based fallback mechanism:

1. Query local gateway for CID for 5-10s
2. If not found within timeout, query "preferred" gateways (e.g. http://ipfs.eth.aragon.network) for 10-30s
3. If not found within timeout, query global gateways (e.g. https://cloudflare-ipfs.com/ipfs/)

Motivation

Resolving files through IPFS is a tricky beast, and its UX is confusing for beginners. Even if a CID eventually resolves, it often takes many minutes and multiple retries.

Keeping files available and pinned is an every harder ask for devs who don't want to run their own IPFS infrastructure.

Fetching and publishing content through non-local, cloud-based providers not only increases the number of nodes hosting the data, but also increases the potential for a CID to be resolved without fuss.

Pitch

Cloudflare recently announced that they were integrating with Pinata, connecting directly to their nodes from Cloudfllare's gateways.

This sounds like a great option to decrease both load and reliance on Aragon-maintained infrastructure for resolving IPFS files. Aragon-maintained infrastructure may have firewalling / other issues (due to security, etc.) that prevent them from providing optimal propagation for pinned CIDs. They are also fairly permissioned, with most of the pinned files resulting from direct authorization from an organization member.

This check to ensure that the app is built while running npm run serve is very dangerous.

This check will stall the process for any app that is not using parcel. Also, even parcel can change this log format in a patch release and cause a major issue to our users without us even noticing.

Not sure, but code hardcodes x.aragonpm.eth and arapps use x.open.aragonpm.eth. Anything BUT the x might be ignored.