subquery / subql-avalanche Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Add support for websocket endpoints
Requirements:
Using the standard SubQuery Avalanche Starter project https://github.com/subquery/avalanche-subql-starter
Reported to have only started after the latest release this week
avalanche-subql-starter-subquery-node-1 | 2022-09-29T09:47:37.432Z <nestjs-0> INFO Starting Nest application...
avalanche-subql-starter-subquery-node-1 | 2022-09-29T09:47:37.723Z <nestjs-0> INFO AppModule dependencies initialized
avalanche-subql-starter-subquery-node-1 | 2022-09-29T09:47:37.724Z <nestjs-0> ERROR Nest can't resolve dependencies of the IndexerManager (StoreService, ApiService, PoiService, ?, SubqueryProject, NodeConfig, SandboxService, DynamicDsService, DsProcessorService, Subquery, ProjectService). Please make sure that the argument Sequelize at index [3] is available in the FetchModule context.
avalanche-subql-starter-subquery-node-1 |
avalanche-subql-starter-subquery-node-1 | Potential solutions:
avalanche-subql-starter-subquery-node-1 | - If Sequelize is a provider, is it part of the current FetchModule?
avalanche-subql-starter-subquery-node-1 | - If Sequelize is exported from a separate @Module, is that module imported within FetchModule?
avalanche-subql-starter-subquery-node-1 | @Module({
avalanche-subql-starter-subquery-node-1 | imports: [ /* the Module containing Sequelize */ ]
avalanche-subql-starter-subquery-node-1 | })
avalanche-subql-starter-subquery-node-1 |
avalanche-subql-starter-subquery-node-1 exited with code 1
When fully synced, avalanche will report being unhealthy. This is because the service expects the block count to increase within a certain time period, but the block production rate means this fails.
We should improve the health check so it doesn't report false positives
When running indexer with workers enabled , there are multiple block ranges that are missing due to fetch service failing
When trying to debug further , in worker.ts file, fetchBlock function is supposed to receive height as number, but sometimes it's receiving it as [number] , and with below fix in that function , the above issue is getting fixed.
async function fetchBlock(height: number): Promise<FetchBlockResponse> {
assert(workerService, 'Not initialised');
let hParmas:any;
if(isArray(height)){
hParmas=height[0]
}else{
hParmas=height
}
console.log('-------------------height---------------',hParmas)
return workerService.fetchBlock(hParmas);
}```
See https://github.com/subquery/subql/blob/main/packages/node/src/indexer/fetch.service.ts#L134 for reference as to how it should work, for some reason these changes didn't make it from the main sdk
Also of note the filter doesn't include modulo https://github.com/subquery/subql/blob/main/packages/node/src/indexer/fetch.service.ts#L178
Should handle avalanche here https://github.com/subquery/subql/blob/75bf3965dfdc574891d7242a18eec469554ff0c9/packages/cli/src/controller/codegen-controller.ts#L190
These details can help to reproduce the environment the issue is occurring
Local Environment: [You can get this information from executing subql version
.]
Query Version: [What is the version of the query service?]
Indexer Version: [What is the version of the indexer service?]
Network Details:
Example project: [A link to a minimal example that can reproduce the issue]
Expected behavior: [What you expected to happen]
Actual behavior: [What actually happened]
Is there any other information you would like to add?
[Description of the bug or feature]
I am Running a subquery project for avalanche (https://github.com/subquery/avalanche-subql-starter) - c-type chain, mainnet.
Using RDS database and onfinality API.
The issue is it shows considerable speed when the sync is started and after reaching 100s or 1000s of block sync, the speed starts decreasing as shown below:-
Even using workers it don't show any change in speed at all. Is the workers implementation not up to mark and still a work-in-progress or it requires some special set of configurations to do beforehand to deal with as database grows, current there are about 2.4Crores of blocks and 40Lacs synced so far with an estimate of days ranging between 30-1000.
Is there something that can be done to trace where the lag is?
Another observation is after sometime (as the database grows, the RPM (request per minute) to onfinality API decrease:-
From this:-
These details can help to reproduce the environment the issue is occurring
Local Environment: [You can get this information from executing subql version
.] --> @subql/cli/1.8.0 linux-x64 node-v17.2.0
Query Version: [What is the version of the query service?] --> Probably [1.9.0] - 2022-11-30, using this docker image https://hub.docker.com/layers/onfinality/subql-query/latest/images/sha256-d7419524924fd98b91b6ce9dfa8651ca26d34f3cbebaf8db6b5e40221d15b2d8?context=explore,
Indexer Version: [What is the version of the indexer service?] Probably [v1.10.5] (https://hub.docker.com/layers/onfinality/subql-node-avalanche/v1.10.5/images/sha256-02d9b8a4cb27af97a547f58e64444da619dcd82748d5c6ce8531573da37e1bf6?context=explore)
Network Details:
Example project: [A link to a minimal example that can reproduce the issue] --> https://github.com/subquery/avalanche-subql-starter
Expected behavior: [What you expected to happen] --> It should show considerable increase with speed when using workers or if not then at least show consistent speed instead of falling from 3.7K requests per minute to 100s or even 0 requests per minute
Actual behavior: [What actually happened]
the Block sync speed fall down from 40bps to 10-15 then 0bps and block sync time shows 1000days or more to sync.
Is there any other information you would like to add?
You can refer database commits per second:-
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.