Summary

Need to add indexing of any owner/admin updates to pot config, as found in README and codebase.

Simplest solution for now might be to just call get_config() on pot whenever one of these methods is called, and save the updated data in DB. Can use fastnear for this, e.g.:

requests.get(f"{settings.FASTNEAR_RPC_URL}/account/{pot_id}/view/get_config")

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

Need to find a way to track receipts for transaction & find the associated transaction hash for a given receipt, and store this on the relevant models (should contain both tx_hash and receipt_id)

Additionally, receipt ID should not be stored in tx_hash field.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

Since no Stellar event streaming service exists, the simplest way to index Stellar events is to poll getEvents RPC API endpoint. Relevant contract addresses &/or topics can be added if desired.

I'd suggest implementing this as a celery-beat periodic task that runs very frequently, e.g. every 5 seconds. Check implementation of existing celery beat tasks (e.g. fetch_usd_prices()) and follow a similar pattern. NB: You shouldn't need any additional systemd services on the EC2 instance as the existing celery-beat-worker-{env} and celery-beat-{env} services should suffice for these additional tasks.

You'll want to pay particular attention to:

1. Deduplication - since you are polling for events within a period and making sure you don't miss any events, it's possible that there would be some overlap and you might receive the same event twice. You should consider how you want to handle this situation.
2. Reindexing - your solution for deduplication may restrict your ability to reindex/spot-index. Make sure you have a plan for reindexing.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Describe the bug

Cooldown end time is not being accurately updated in indexer. E.g. ai.v1.potfactory.potlock.near:

Postgres (incorrect):

On-chain (correct):

NB: don't worry about cooldown_period_ms for now as this isn't present in pot config and is not required for UI currently.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

• Add remaining endpoints
• Add thunderclient for testing
• Add rate limiting
• Add request caching

Summary

Would be very useful to see the block timestamp of the latest indexed block (BlockHeight.objects.first()) as a datetime, to display on the admin dashboard or on other status monitoring applications.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

Need a GET endpoint to retrieve all PotFactories. Response should include FKs and lazy load associated records (e.g. include a Account record for each admin & whitelisted deployer etc)

Add related OpenAPI schemas & ensure they are correct

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

• Run periodic job to update totals on Account model
• Run periodic job to fetch exchange rates from coingecko for donations that have no usd amounts

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

Currently, we only fetch and save historical token prices for NEAR native token. We need to expand this to include FTs.

To-do:

• Add nullable coingecko_id field to Token model
• When a new Token is created (e.g. in save() override when it's a new record - see this example), call Coingecko /coins/list API with include_platform=true, and find the Coingecko ID for this token if it exists (you'll have to check the platforms property of each Coingecko record for a near-protocol key and a value that matches the FT's address - see example response below)
• If it exists on Coingecko, save Coingecko ID on Token record
• Refactor Donation.fetch_usd_prices and PotPayout.fetch_usd_prices to share a single util method (e.g. tokens.utils.fetch_usd_prices_common) that takes a token, an amount, and a timestamp and returns the amount_usd that can be saved to the Donation or PotPayout record. This common method should probably also save the historical token price.
• When fetching usd prices, use Token.coingecko_id in the request URL instead of Token.id.id. Skip the coingecko request if Token.coingecko_id is None.
• Add "near" as coingecko_id for existing NEAR native token record (do this together with Lachlan on the EC2 instance)

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

Here is an example Coingecko response:

[
  {
    "id": "1inch",
    "symbol": "1inch",
    "name": "1inch",
    "platforms": {
      "ethereum": "0x111111111117dc0aa78b770fa6a738034120c302",
      "harmony-shard-0": "0x58f1b044d8308812881a1433d9bbeff99975e70c",
      "avalanche": "0xd501281565bf7789224523144fe5d98e8b28f267",
      "binance-smart-chain": "0x111111111117dc0aa78b770fa6a738034120c302",
      "near-protocol": "111111111117dc0aa78b770fa6a738034120c302.factory.bridge.near",
      "polygon-pos": "0x9c2c5fd7b07e95ee044ddeba0e97a665f142394f",
      "energi": "0xdda6205dc3f47e5280eb726613b27374eee9d130"
    }
  },
  ...
]

Alternatives

Coingecko tokens could alternatively be fetched and added in a recurring job, but this would create many more tokens than are actually used in PotLock, and additionally token metadata would need to be fetched for each token to get its decimals etc. I think a Token.save() method override is a simpler and more targeted solution at this point.

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

• by account id
• get total donation
• get self reported amount

Summary

Also, determine best way to index human scores and keep them in sync with contract.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Run indexer locally from Potlock start block (104052029 when registry was deployed/initialized) until present. Presumably errors will come up during this process; resolve them until it's possible to run from beginning to end, and also restart in the middle, ensuring idempotency.

Once run successfully from beginning to end, verify DB state against contract state. Identify and resolve any discrepancies.

Summary

Currently, the only admin method on PotFactory contract that is indexed is admin_add_whitelisted_deployers. We should index all admin methods, so that our Postgres DB stays up-to-date and we avoid weird errors and inconsistencies.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

The v1 pot contract has a bug that results in net_amount being returned as 0 for all Pot donations (this will be fixed in pot v2 contract).

In the meantime, we have to:

• Update handle_new_donation so that if Donation.net_amount is "0", we calculate it (e.g. update this line to if "net_amount" in donation_data and donation_data["net_amount"] != "0":)
• Add custom migration to donations app that uses run_python to update net_amount for all Donations where net_amount="0". See example.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

In order to index Grantpicks (on both Stellar and NEAR), some models will have to be added & updated to integrate with existing data storage. This is a guide that can be used as a starting point for this task.

The main construct in Grantpicks is a Round. Rounds are very similar to Pots in that they have owner/admins, receive applications, receive deposits &/or donations, and make payouts. They act as a vault that receives funds and has rules around the distribution of these funds. The main differences between Pots and Rounds are:

1. there are multiple rounds on a single Rounds contract (unlike Pots, which are unique per contract)
2. Pots utilize contributions (Donation) as the community/voting element, whereas Rounds utilize voting (specifically, PairVote as outlined below)

Since these constructs are quite similar (with notable differences), they should share some models, e.g. PotApplication* and PotPayout*. I personally wouldn't worry about renaming these models (e.g. to just Application and Payout), but start thinking of "pots" as a term that encapsulates both OG pots and also other variations on pots, like Rounds. So, applications to a round should use the PotApplication model, but it should be modified so that it can be associated with either a Pot or a Round (more on this below).

Other notes:

• Don’t worry about renaming models like PotApplication but generally consider “pots” to be a term that encapsulates OG pots and also other variations on pots, like Rounds. Rounds share many characteristics of pots, and have pretty much identical related models for Applications, Applications reviews, Payouts & Payouts Challenges.

• Create new model for Round (this will have many fields similar to Pot, but it is not exactly the same)

• Rounds, unlike pots, are NOT unique per address. A Rounds contract is a singleton that contains many rounds. So there should be a field like Round.contract (Account FK which also contains chain FK)

• Add optional round FK to PotApplication model, and make pot optional (but one of these should be required - either pot or round - this can be done in a save() method override)

• Add Blacklisted to PotApplicationStatus enum (also needed for pots v2)

• Add new model PairVote (located within pots app) that contains:

  • voter (related name pair_votes)
  • for (related name votes_for)
  • against (related name votes_against)
  • voted_at (timestamp)

• Add new model RoundDeposit (this is kind of like the equivalent of a pot matching pool donation, but should be its own model). Use fields from on-chain struct (look for DepositExternal in deposits.rs)

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

• Connect with @codingshot re. Potlock Labs AWS account
• Determine deployment / CI/CD plan (also environments, presumably we want testnet, staging & prod, though we will be deploying everything to staging initially)
• Create EC2 instance
• Set up SSH access by IP address + .pem key for myself &
• Create VPC
• Provision RDS Postgres DB
• Get indexer running using systemd
• Verify starting & stopping indexer; ensure that it restarts as desired
• Verify error logging

Summary

Add blank=True to all model fields where null=True. This is necessary for admin dashboard updates.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

• get all protocol fees generated
• based on set period of time

Summary

[Brief overview of the enhancement and why it is needed or desired]

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Summary

Currently we store Account FK primary key fields as id, which is not very intuitive, e.g. to get a Token's address you need to do Token.id.id. What we want is Token.account.id. The same goes for PotFactory and Pot models.

This will have effects all over the codebase and will need extensive review to ensure that no bugs are created! Pay attention to admin.py and serializers.py files in addition to indexing logic, populatedata script & recurring jobs.

Additionally, OpenAPI schemas & examples will need to be updated.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Set up logging service to monitor errors using Cloudwatch, with multiple log levels. Also set up monitoring of availability & performance.

• Set up EC2 instance with security group config to allow HTTP, HTTPS & SSH access
• Install Poetry, nginx, gunicorn, Redis, PostgreSQL, git & other system dependencies on EC2 instance
• Clone Django project to EC2 instance
• Configure IAM role for EC2 instance that allows Cloudwatch log writing
• Provision Postgres RDS database within VPC to only allow access from EC2 instance
• Provision 3 x ElastiCache services (1 x clustered for regular Django cache + 2 x non-clustered for Celery broker & results)
• Add env vars to ~/.bashrc
• Configure gunicorn app server using systemd
• Configure nginx web server reverse-proxying to Gunicorn (ensure appropriate permissioning)
• Configure celery service using systemd
• Configure logging (log levels, Sentry error logging, Cloudwatch logging)
• Run collectstatic
• Create superuser & log in to admin dashboard
• Start celery background service
• View celery INFO-level logs in Cloudwatch to verify successful running
• Verify saving of last-processed block in case of restart
• Simulate crashing & restarting to verify
• Observe indexing speed, determine time to catch up to present, make changes to improve speed if necessary/possible
• Test API, verify caching
• Monitor costs
• Set up CI/CD (create separate ticket for this)

• need to fetch account id and see referrals earned
• need to index every donation and sponsorship to see if their was referrer paid out

Summary

Individual exceptions should be handled to assist with debugging and improve error handling logic.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]

Describe the bug

When I call get_config() on the potfactory contract, I get the following response:

View call: v1.potfactory.potlock.near.get_config()
{
  owner: 'potlock.near',
  admins: [ 'plugrel.near', 'impact.sputnik-dao.near' ],
  protocol_fee_basis_points: 200,
  protocol_fee_recipient_account: 'impact.sputnik-dao.near',
  whitelisted_deployers: [
    'lachlan.near',        'plugrel.near',
    'wendersonpires.near', 'builddao.near',
    'james.near',          'efiz.near',
    'ossround.near',       'airound.near',
    'buildersround.near',  'africaround.near',
    'ecofund.near',        'ukraineround.near',
    'collegeround.near',   'ndcround.near',
    'marketinground.near', 'marketingdao.near',
    'carina.akaia.near'
  ],
  require_whitelist: true
}

However, when I view the PotFactory in the django admin dashboard, there are no whitelisted deployers:

This is confirmed in the django shell on dev:

We need to:

• Determine why whitelisted_deployers is not up to date (review indexing logic, including for this recent transaction which should have been indexed)
• Fix indexing logic & reindex so that correct whitelisted deployers are saved in db

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Describe the bug

TokenHistoricalPrice timestamp field is not being explicitly set when a new record is created, and defaults to timezone.now(). (I think this is my fault)

Should be explicitly set and default should be removed.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Summary

As we move to support multi-chain, we will need to make two adjustments to the Account model.

These are not urgent, especially since NEAR and Stellar addresses are not compatible, but should be done before multiple EVM chains are supported.

1. Store account address in Account.address instead of Account.id PK (this is because the same address may exist on multiple chains
2. Use AutoField for Account.id PK instead of CharField
3. Add unique_together constraint for Account.chain & Account.address to ensure that only one Account record exists per address per chain

This will require careful review and local testing before deploying & migrating dev/prod data.

Motivation

[More detailed explanation of the motivation for the enhancement, including any benefits it would provide]

Description

[Detailed description of the enhancement, including how it would work and any design considerations]

Alternatives

[Discussion of any alternative solutions that were considered and why the proposed solution is preferred]

Risks

[Identification and mitigation of any potential risks associated with the enhancement]

Acceptance Criteria

[List of criteria that must be met for the enhancement to be considered accepted]

Additional Information

[Any other relevant information, such as links to related issues or pull requests]