Comments (6)
AMohabeer
commented on June 8, 2024
Seems that by reducing the grid size, the job completes and records are indexed. This is strange as larger gird sizes worked previously (prior to the upgrade). I think this should still be investigated as I'm not sure how upgrading ES can cause this...?
from cogstack-pipeline.
afolarin
commented on June 8, 2024
@AshaAgrawal is this resolved?
from cogstack-pipeline.
AMohabeer
commented on June 8, 2024
@afolarin Partly yes. DBLineFixer is resolved but TikaAnonymised is still very slow. Honghan will be looking at this tomorrow after which we will update this thread.
from cogstack-pipeline.
Honghan
commented on June 8, 2024
-
the ES update did cause some issues on DBLinkeFixer: string type fields without analyzer will have a length limitation of their values (less than 32766). for details, please check this: 68cb059.
-
the deid slow indexing issue was caused by the view definition. it has been changed to a stored procedure, which seems much faster. Asha is doing final validation on this. If everything is fine, I think she will close this issue.
from cogstack-pipeline.
afolarin
commented on June 8, 2024
@AshaAgrawal validation of this would be good.
@Honghan if this is a general problem then can you update the docs
from cogstack-pipeline.
AMohabeer
commented on June 8, 2024
As noted by Honghan, everything seems to be working fine, therefore closing issue.
from cogstack-pipeline.
Related Issues (20)
- Add support for PDF Form Parsing HOT 1
- [Feature] Support arbitrary parameter for SQL INSERT statement for jdbc_out
- Default for scheduler.rate does not follow the cron syntax HOT 1
- Post-processing of bio yodie result HOT 1
- De-Identification
- Test LSTM OCR Engine in Tesseract HOT 4
- ElasticsearchRest Client not working with scheduler HOT 4
- ElasticsearchRest Client will fail silently if index contains invalid character HOT 2
- PDF and Thumbnail generation will fail if Tika throws length warning
- Unable to index Docman Documents HOT 5
- Add PDF Table Extraction using Tabula
- Cogstack docker download issues HOT 6
- Refactor the build process HOT 1
- add Nginx proxy to the stack for basic Auth
- Mechanism to prevent stale CogStack structured data in Elasticsearch HOT 1
- Refactor Integration and acceptance tests HOT 1
- Can we add more than one elastic search hosts in the config ? HOT 4
- fix: read from filesystem or object-store
- Unable to view links on confluence HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cogstack-pipeline.