Comments (18)
Yes, I was aware about this, and this should be definitely optimized. Do you want to provide a PR?
The Count()
function should be added to the StorageEngine interface{} and be implemented for the two storages currently available
from clover.
Yes, I was aware about this, and this should be definitely optimized. Do you want to provide a PR? The
Count()
function should be added to the StorageEngine interface{} and be implemented for the two storages currently available
Of course I want to provide pr, but I need to wait for me to read the code and ask by the way, will there be a cache for the query results? I noticed that the first query takes much longer than the later query.
from clover.
For example, I used 190ms for my first query, and then only used 120ms
from clover.
Badger caches data internally
from clover.
For some reason, my skip function doesn't seem to work. The code is as follows
func (here *Db) SearchContent(names []string, num int, pg int) []*clover.Document {
var name string
for i, v := range names {
if i < len(names)-1 {
name += "(.*" + regexp.QuoteMeta(v) + ".*)|"
} else {
name += "(.*" + regexp.QuoteMeta(v) + ".*)"
}
}
fmt.Println(num, pg)
query := here.content.Where(clover.Field("name").Like(name))
startT := time.Now()
docs, _ := query.Skip(num * pg).Limit(num).FindAll()
fmt.Printf("time.Since(startT): %v\n", time.Since(startT))
return docs
}
from clover.
What do you mean by "doesn't seem to work"? What is your expected output and what are you getting?
from clover.
What do you mean by "doesn't seem to work"? What is your expected output and what are you getting?
I expected to skip a few doc, but it didn't.
And then the findall function returns random disorder?
from clover.
I see my problem. I didn't sort.
from clover.
But when I sort it, its query speed slows down again.
from clover.
Naturally, you have to take into account the cost of sorting
from clover.
Okay, I'll start looking at the source code tomorrow.
It is 50ms before sorting and 1.3s after sorting.😔
from clover.
I found what the problem is. I'll release soon a fix for this
from clover.
I found what the problem is. I'll release soon a fix for this
thanks ,bro
from clover.
Can you post the snipped before and after sorting?
from clover.
Is it time-consuming to sort screenshots, or something else?
The part I want to sort is the above part of the code. I need to sort and query the data with a specified number of pages.
from clover.
But when I don't sort, skip won't achieve the effect of the next page as I thought.
from clover.
I pulled the version I just submitted, then tested the count elapsed time and found that it didn't reduce much.
from clover.
Actually this is the best we can do. Without indexes (which are not currently supported), there is no faster way than iterate on each record of the collection.
from clover.
Related Issues (20)
- [Feature Request] Utilize `encoding.BinaryMarshaller` and `encoding.BinaryUnmarshaller` HOT 9
- Example Request: Unmarshal/Update/Replace HOT 6
- Cannot allocate initial memory HOT 1
- GT LT operation very slow HOT 3
- Question: Is there a way to search for a value in all fields? HOT 2
- Getting error when during bulk insert HOT 3
- Small v2 package annoyance HOT 10
- Question: Searching for all documents within time range HOT 1
- distributed? HOT 1
- Support struct in Save() method HOT 7
- Add `OpenWithOptions()` to badger store
- Memory efficient `ExportCollection()` HOT 1
- need update readme HOT 2
- Hope to support semantic query building function HOT 9
- Memory store was removed in v2 HOT 1
- doc.Unmarshal does not play well with `json` tag HOT 5
- Badger storage | RunValueLogGC() is managed with a fatal error HOT 3
- DropCollection doesn't delete the collection from disk HOT 3
- Feature: a cmdline tool to explore a cloverDB HOT 1
- v2: Constant CPU usage HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clover.