`cider-enable-flex-completion` is not a sufficient replacement for `cider-company-enable-fuzzy-completion` about cider HOT 26 OPEN

I'll repeat just in case: the problem is Emacs not displaying valid completion candidates returned by Compliment. Some of those are really important (like completing a class by its short name).

What you've proposed, if I understand correctly, would be to pass say flex to Compliment and an empty prefix and so that Compliment understands that Emacs will do the filtering and it should return everything it knows. But that would mean, for example, returning 600 thousand loaded classnames and I doubt that would be an efficient thing to do.

from cider.

Maybe later – looks like quite a bit digging is required.

We can add a ;; DO NOT DELETE #3653 comment in the meantime

I think it's fine, enough people have the context now.

from cider.

Overall, the less the frontend (editor) tries to be smart about the completion candidates, the better. We can control the UX match better on the backend. And if there are users that want flex-like fuzzy matching, we can also add that to Compliment and hide behind a toggle. That is still much more efficient and makes more sense than filtering through everything on the frontend side.

from cider.

Thanks for the issue!

We should tread carefully here because we shouldn't necessarily be smarter than (or reinvent) Emacs' mechanisms.

Most of all, it's a concern of simplicity - Emacs has an abundance of styles, and there's also an abundance of Emacs packages building up on those, so creating our own standards can possibly make things even messier for everyone involved.

I don't have a specific suggestion/stance except: we can try first hard at adhering at Emacs standards and squeezing performance out of them.

If we document reasoning/facts reflecting that that's impossible, we could keep exploring solutions.

from cider.

Quick q - would it make sense to pass the completion style to Compliment?

In my mind it would make sense - then Compliment could infer what's wanted and return fewer / more relevant results.

from cider.

What if we passed flex and what the user has typed?

So it would include (among other stuff) these 3 things:

• What Emacs wants us to complete - ""
• The style - "flex"
• What CIDER knows that the user has actually typed - "InputSt"

from cider.

We should tread carefully here because we shouldn't necessarily be smarter than (or reinvent) Emacs' mechanisms.

I think we should be smarter than Emacs mechanisms. I don't see mainline Emacs as the frontier in completion solutions; e.g., company-mode steps away from it in many regards and there is a reason we use company and not just completion-at-point.

from cider.

What if we passed flex and what the user has typed?

Then Compliment doesn't really need to know that it was flex that triggered it, the returned candidates would be the same.

from cider.

I think I understood what you mean. So we would return the same "preferred" result twice, for normal prefix, and for the empty prefix, and let Emacs show the candidates that we want to show in the empty prefix case. I mean, that would work, but it feels like a bigger hack to me than what cider-company-enable-fuzzy-completion was doing.

from cider.

company-mode steps away from it in many regards and there is a reason we use company and not just completion-at-point.

Well we (certainly you and me) use Company but many others user Corfu and whatnot.

That's where adhering to standards show their value - supporting a large divesity of users

Then Compliment doesn't really need to know that it was flex that triggered it, the returned candidates would be the same.

Then we have a potential solution, right? Have CIDER pass the actual user input as an additional k-v

(passing flex might be worthwhile for further trimming results, but I'd understand if you didn't want that duplicated work)

from cider.

let Emacs show the candidates that we want to show in the empty prefix case.

Yes

I mean, that would work, but it feels like a bigger hack to me than what cider-company-enable-fuzzy-completion was doing

It seems a clean, low-effort, low-complection solution to me. Pass additional properties from the frontend and act on them on the backend.

To me (as someone who strives to support a diverse set of users) there's high value in adhering to Emacs standards, so I'd vote for this solution unless there was a huge factual drawback.

from cider.

Actually, I tried it again, and I see two identical messages are sent to nrepl for each completion attempt:

(-->
  id                        "570"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:42:27.829532000"
  context                   "nil"
  enhanced-cljs-completion? "t"
  prefix                    "InputS"
)
(<--
  id          "570"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:42:27.832275000"
  completions ((dict "candidate" "java.io.InputStream" "type" "class") ...)
  status      ("done")
)
(-->
  id                        "571"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:42:27.855899000"
  context                   "nil"
  enhanced-cljs-completion? "t"
  prefix                    "InputS"
)
(<--
  id          "571"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:42:27.858973000"
  completions ((dict "candidate" "java.io.InputStream" "type" "class") ...)
  status      ("done")
)

How would passing extra arguments to Compliment help here?

from cider.

Please describe what's wrong in the nrepl interaction above

from cider.

The problem is that Emacs sends two identical requests to the backend (with prefix "InputS"). Compliment returns the correct result both times, but Emacs refuses to render it.

from cider.

Why does that happen?

from cider.

Do you ask me? :)

However, if the prefix is all-lowercase, then it sends the second request with an empty prefix:

(-->
  id                        "658"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:55:04.272861000"
  context                   "(__prefix__)
"
  enhanced-cljs-completion? "t"
  prefix                    "cji"
)
(<--
  id         "659"
  session    "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp "2024-05-05 21:55:04.293445000"
  status     ("done" "no-eldoc")
)
(-->
  id                        "661"
  op                        "complete"
  session                   "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp                "2024-05-05 21:55:04.353659000"
  context                   "(__prefix__)
"
  enhanced-cljs-completion? "t"
  prefix                    ""
)
(<--
  id          "661"
  session     "ecd63e82-c777-435b-8dbd-16dcf3b3a01b"
  time-stamp  "2024-05-05 21:55:04.387442000"
  completions ((dict "candidate" "*" "ns" "clojure.core" "type" "function") ...)
  status      ("done")
)

from cider.

Basically, if CIDER can control what is sent to Compliment, then you don't really have to pass flex to it and the prefix in some other field – you can just perform the regular query. The trick is to make Emacs show everything that Compliment returned, not some arbitrary subset of it.

from cider.

Do you ask me? :)

Okay, we'd have to look into this first - could plausibly be a bad implementation on our end

Without this information there's not really much decision-making to do.

from cider.

Okay, we'd have to look into this first - could plausibly be a bad implementation on our end

I thought it was Emacs completion engine that controls it. If not, then it would make things easier, I suppose.

from cider.

Without this information there's not really much decision-making to do.

As long as cider-company-enable-fuzzy-completion exists, one doesn't have to be made promptly. Just something to keep in mind before accidentally removing it. Possibly "undeprecate" it since this part of the docs now shows incorrect information: https://docs.cider.mx/cider/usage/code_completion.html#fuzzy-candidate-matching. CC @bbatsov.

from cider.

Fixing the issue would seem a good course of action - having churn in the docs can easily be confusing.

We can add a ;; DO NOT DELETE https://github.com/clojure-emacs/cider/issues/3653 comment in the meantime

For clarity - would you be interested into digging into the Elisp part?

from cider.

I thought it was Emacs completion engine that controls it. If not, then it would make things easier, I suppose.

Yeah, I'm reasonably sure the Emacs completion engine controls it. But there's always the chance we've messed something up on our end. 😅

As long as cider-company-enable-fuzzy-completion exists, one doesn't have to be made promptly. Just something to keep in mind before accidentally removing it. Possibly "undeprecate" it since this part of the docs now shows incorrect information: https://docs.cider.mx/cider/usage/code_completion.html#fuzzy-candidate-matching. CC @bbatsov.

Sure. Feel free to update the docs to reflect your findings. I guess I haven't used the new flex completion, so I didn't notice the issues with it. (I switched from Company to Corfu a while ago and I was too lazy to customize it to my liking so I went back to the basics)

from cider.

I see we also have a bunch of notes about missing functionality here:

;; defines a completion style named `cider' (which ideally would have been named `cider-fuzzy').
;; note that there's already a completion category named `cider' (grep for `(metadata (category . cider))` in this file),
;; which can be confusing given the identical name.
;; The `cider' completion style should be removed because the `flex' style is essentially equivalent.
;; (To be fair, `flex' was introduced in Emacs 27, 3 years in after our commit 04e428b
;;  which introduced `cider-company-enable-fuzzy-completion')
(add-to-list 'completion-styles-alist
             '(cider
               cider-company-unfiltered-candidates
               cider-company-unfiltered-candidates
               "CIDER backend-driven completion style."))

;; Currently CIDER completions only work for `basic`, and not `initials`, `partial-completion`, `orderless`, etc.
;; So we ensure that those other styles aren't used with CIDER, otherwise one would see bad or no completions at all.
;; This `add-to-list` call can be removed once we implement the other completion styles.
;; (When doing that, please refactor `cider-enable-flex-completion' as well)
(add-to-list 'completion-category-overrides '(cider (styles basic)))

That might be related to the problems that @alexander-yakushev has observed.

from cider.

Yes, surely if we implemented the other styles, other issues would become more evident.

from cider.

Yeah, I'm reasonably sure the Emacs completion engine controls it. But there's always the chance we've messed something up on our end. 😅

Title / OP in #3006 hints that our current state isn't quite polished.

from cider.

Ah, yeah - I had forgotten about it.

from cider.