There are a number of trac tickets related to the way GHC reports errors, with a view to improving them.

https://ghc.haskell.org/trac/ghc/ticket/8809
https://ghc.haskell.org/trac/ghc/ticket/10073
https://ghc.haskell.org/trac/ghc/ticket/10179

For tooling support, there should be an option to return errors in some kind of typed structure that can be easily an unambiguously processed by the tool.

This is in general a hard task, with deep tentacles into GHC.

Is there any low hanging fruit we can harvest in time for GHC 8.1?

As linked from the things-marginally-related list, Idris has quite a nice protocol. It'd be nice to be able to share editor integration effort with the Idris folks, but I'm not sure how much overhead would be involved in making this happen. The primary editor integration seems to be emacs, but it looks like there are official atom, sublime, and vim integrations as well.

I'm not sure how much code reuse we'd get out of this, though. Even with something closer to Haskell, like Idris, I imagine there'd be some things that would need to be done differently in the editor to support Haskell vs supporting Idris. I can imagine sharing code by doing one of the following, using idris-mode as an example:

A) Have idris-mode also directly support Haskell. Of course, things like syntax highlighting would still be the responsibility of haskell-mode.

B) Have idris-mode indirectly supoprt Haskell by providing the necessary hooks and options to make it work well with Haskell.

C) Factor out the shared common code into a separate package, relied on by both idris-mode and haskell-ide-mode (or whatever it's called). Seems like a maintenance burden.

D) Copy-modify code. Yuck!

If going this route, I think I prefer (B), as (C) and (D) seem undesirable, and (A) seems invasive to idris-mode. Considering that idris is written in Haskell, it's imaginable that the Idris folks would be keen on this idea. We'd certainly want their go ahead, as it could lead to additional complexity in their editor plugins.

This thought is inspired by Tim Ottinger's recent Dot-Programming your Object Oriented Code and some subsequent Twitter discussion. Tim paints a pretty accurate picture of how IDEs work to great advantage in OO langs.

When you write a program, you will not read the documentation. You don't have time and inclination for that. [...] You type the name of some object and you press the dot. The list that pops up is your guide to the world of the object whose name lives on the left side of the dot you just pressed.

The discoverability is so powerful that when you pick up a new library, you often only need to read the documentation far enough to instantiate one core object, and any IDE can walk you through the rest.

Coming to haskell from OO feels like there's just a soup of functions; no context-sensitivity...
--@doublehelix

There are two big reasons I see that a naive code completion feature doesn't easily fit Haskell:

1. The workflow doesn't tend to proceed strictly left-to-right, so completion isn't as simple as just inserting at the cursor. In a haskell translation of my previous example, the code z $ y x would be written from right to left if x leads to the discovery of y and then to z.
2. What the IDE should suggest is a more complicated question. The OO case is trivial - just list the members of the type. With Haskell... I'm not even entirely sure what I'm asking for.

I think anything we can do to create a discovery experience akin to OO code completion would be a huge win.

There are some problems with the GHC-API as mentioned here:
https://github.com/fpco/haskell-ide/wiki/GHC-API

I think that several of these problems could be solved by storing code, type Information, cabal data in a graph database. The idea would be to query this database instead of using GHC-API. GHC-API is just used to fill the db: Neo4j.
The reason for me to come up with a graph database in the first place was to answer the following questions:

• Which functions are most often applied to a certain type?
• How do I get from a type A to a type B? E.g. I get from String to Text by Data.Text.pack

In order to achieve this we have to store all repos from github/hackage/stackage in this database.
The are a lot of advantages:

• A graph instead of a constructor tree in GHC-API allows to find reverse dependencies (not on packages but on functions: Does anybody use this library function or can I deprecate it? With what parameters do others use this function?)
• A query language like Cypher is more powerful than a traversal of a ghc tree of constructors.
• A more general access to data sources like code, cabal-files, github

The only disadvantage is that this database would have to be maintained and therefore more likely would be a service instead of being installed locally.
I plan to implement this and have looked into ghc-exactprint and hoogle and made some experiments with GHC-API. Anything else I should look into? Would you like to have this?

At the moment a PluginDescriptor is defined as

data PluginDescriptor = PluginDescriptor
  { pdCommands        :: ![Command]
  , pdExposedServices :: ![Service]
  , pdUsedServices    :: ![Service]
  }

Add some fields to be used in the IDE UI as follows

data PluginDescriptor = PluginDescriptor
  { pdUIShortName     :: !T.Text
  , pdUIOverview      :: !T.Text
  , pdCommands        :: ![Command]
  , pdExposedServices :: ![Service]
  , pdUsedServices    :: ![Service]
  }

The pdUIShortName should be displayed in menus and the like, rather than the PluginId.
The pdUIOverview can be displayed in any kind of "more information" scenario about the plugin.

It seems that TravisCI somehow does not produce deterministic builds. master branch at https://travis-ci.org/haskell/haskell-ide-engine/builds/89591048 compiles, after a useless change in README a pull request does not compile https://travis-ci.org/haskell/haskell-ide-engine/builds/89742925. It ends up with a warning in HaRe:

    src/Language/Haskell/Refact/Utils/Monad.hs:211:10: Warning:
      No explicit implementation for
          ‘GM.gmoAsk’
        In the instance declaration for ‘GM.GmOut (StateT RefactState IO)’

Build system must be reliable and deterministic.

I have all the plumbing in place for the ghci-ng plugin, but the tests are failing with something really silly.

See alanz@2fc7448

The essence of it is that the startup routine for ghci-ng calls into

    (argv2, staticFlagWarnings) <- parseStaticFlags argv1'

Where argv1 is "--interactive" and we then get a "Blocked indefinitely on an MVar" in the test.

Help

Requirements:

1. Follows generally accepted json practices.
2. Outside languages are untyped, so is json.
3. Works in async manner.
4. Enables easy error handling on Editor's side.

There seems to be a consensus that we would like to use some hints for types in the json itself, we can base a simple idea roughly on inverse of Type-indexed rows.

Lets define a set of possible responses as an open sum with following examples:

{"error": {"ui_message": ..., "code": ...}}
{"type_info": {"type":"IO () -> IO ()"}}
{"diff": {"XYZ.hs" : { diff description}}
{"plugins": {"p1": {?}, "p2": {?}}
... more follow

this translates to Haskell data declarations:

data Error = Error { ui_message :: Text, code :: Int }
instance ValidResponse Error where ...
data TypeInfo = TypeInfo { typem :: Text }
instance ValidResponse TypeInfo where ...
data Diff = Diff { diff :: ??? }
instance ValidResponse Diff where ...
data Plugins = Plugins { plugins :: ??? }
instance ValidResponse Plugins where ...
...etc

Rationale:

1. On the editors side there is no confusion when using labels: response.type_info.type
2. On Haskell side label name map one to one with type (on the same level of nesting in json).
3. Error handling on editors side is pretty easy, just handle one case "error".
4. When there is a lack of error handling on editors side there is no confusion (right now "contents" may be valid response or error string, only known after inspecting "tag")

If there are multiple hie engines started they all try to listen on the same port.

If the only interface used is stdin/stdout it is not wise to make startup fail just due to inet port being busy.

Process needs to be sorted out, for example:

• ways to communicate within the project
• what are github issues for and how to use them
• how github issues are pushed forward in process and when are they closed
• describe github issue labels and operational meaning of the labels (if used)
• who, how, where and how often publishes progress report
• who, how, where and how often publishes release notes
• what is current project focus
• standards for code contributions (checklist)
• pull request review and merging process
• documentation standards

Create a general type that can be returned to the IDE containing a diff between to file.

Perhaps base it on http://hackage.haskell.org/package/filediff

This would be used to convert the list of changed files from HaRe into a set of diffs on those files, as well as in any other plugin requiring this feature.

All IDEs can also implement standard processing for this type.

Some plugins need to do some initial startup, and then store some state.

We need to come up with a way to do this.

This will probably include some kind of life-cycle methods exposed as part of the plugin, together with a way to store the state.

It may be simplest to define the state as being of type Data.Dynamic, as each plugin will know how to convert it for its own internal use.

So we add

startPlugin :: IO Dynamic

and each CommandFunc gets an additional Dynamic parameter passed in.

My thinking is that we have an "inner" and an "outer" protocol. The inner/logical one is processed by all the plugins, the outer one comes about when a specific inner message is wrapped in a transport for delivery on the wire.

A possible version of the inner protocol is defined in https://github.com/haskell/haskell-ide-engine/blob/plugins-definitions-play/haskell-ide-plugin-api/Haskell/Ide/PluginDescriptor.hs.

A IdeRequest is defined as

data IdeRequest = IdeRequest
  { ideCommand :: CommandName
  , ideSession :: SessionContext
  , ideContext :: CommandContext
  , ideParams  :: Map.Map ParamId ParamVal
  } deriving Show

type ParamId = String
type ParamVal = String

This initially expects any additional parameters required for a command are String values.

The response is defined as

data IdeResponse = IdeResponseOk String -- ^ Command Succeeded
                 | IdeResponseFail String -- ^ Command Failed
                 | IdeResponseError String -- ^ some error in haskell-ide-engine
                                           -- driver. Equivalent to HTTP 500
                                           -- status
                 deriving Show

The String type is a problem.

What is a good way to represent the value for the IdeResponse inner protocol so that it can be serialised/deserialied via an arbitrary transport mechanism, such as JSON, MSGPACK, etc.

Loose comments from editor (emacs) perspective:

1. Buffers (open files) can be not yet saved and therefore differ from what is on disk currently. Generally we want to operate with the visible contents.
2. Files-while-editing have the tendency to by not-valid Haskell. Or at least no type correct. There were multiple issues raised so that haskell-mode works with somewhat valid Haskell instead of totally not working.
3. Editors have cursor position that can serve as indicator what to act on.
4. Editors have marked region that can serve as indicator what to act on.
5. Smooth interaction requires latency hiding because 200ms is when people start to notice non-smoothness. Editor needs to be ready to handle longer-running queries, actually any query can run 'too long'. This means communication needs to be asynchronous.

At the moment if HIE is killed for any reason, it discards any logs that have not been flushed.

Add a set of signal handlers to flush the logs on SIGTERM etc.

Possible resource: http://zguide.zeromq.org/hs:interrupt

https://hackage.haskell.org/package/logging-3.0.2/docs/Control-Logging.html
https://hackage.haskell.org/package/fast-logger-2.4.1/docs/System-Log-FastLogger.html

Note: there is nothing holy about this logging package, it was just convenient.

I suggest we use the same label system as for stack repo.

As of now, it's harder to understand where the project is going without them.

Milestones

https://github.com/commercialhaskell/stack/milestones

Labels

• copy-paste labels (keep color scheme)
• create milestones
• tag existing issues with labels
• give repo access to some people to speedup the tagging process

I am currently working on a ghci-ng plugin.

The work is happening in the following places

• https://github.com/alanz/haskeline
• https://github.com/alanz/ghci-ng

The initial intent is to

• expose ghci-ng as a library
• modify haskeline to support a Behaviour that writes to/from a Chan instead of a file Handle
• make a set of commands in a plugin to interact with these Chan s.

First pass will be to just wrap stdio/stdin to ghci as a proof of concept, allowing an embedded ghci session in a supporting IDE

Second pass will be to extend the data type in the Chan for the various commands supported in ghci, so that they can be used as first-class commands in the IDE.

The ghci threading model to support this is unknown right now, but it is a proof of concept.

Feedback welcome.

I am keen to get an initial release out by end Nov, even if it is just of the plugin API so we can start writing plugins as things exposed by a tool, rather than somehing in HIE.

The things that need to be in it should be captured in https://github.com/haskell/haskell-ide-engine/milestones/P1:%20Must

Ideally any kind of IDE integration would be good too, but I know that that is the hardest part initially.

What all should be in there?

At the moment HIE has a list of dependencies in its cabal file, for plugins available to it.
Then in the MainHIE.hs it has a variable of type Plugins which enumerates the plugins to be available.

plugins :: Plugins
plugins = Map.fromList
  [
    -- Note: statically including known plugins. In future this map could be set
    -- up via a config file of some kind.
    ("eg2",  example2Descriptor)
  , ("hare", hareDescriptor)
    -- The base plugin, able to answer questions about the IDE Engine environment.
  , ("base", baseDescriptor)
  ]

A possible future configuration process would be to have a separate exe that can read a HIE config file, containing a list of (plugin name,plugin descriptor,originating package). This exe would then generate a cabal file and Main.hs that calls for the packages to be installed, and sets up the plugins variable.

Hence a local configuration can have whatever plugins are required. This can work since we assume we are doing haskell dev, and have a compiler on hand.

Just want to mention these while things are still in the planning phase.

this may be spurious (or obvious), but making sure that its easy to have nonblocking interaction with editors and their plugins. (i've definitely seen a number of haskell plugins which have bad cpu/memory usage or needless editor blocking from bad integration)

A PluginDescriptor lists the available commands and the accepted contexts / parameters each can have.

Modify doDispatch to validate these in the incoming IdeRequest and return an appropriate IdeResponseFail value if the validation fails.

At the moment the plugin callback function for a particular command is defined as

type Dispatcher = forall m. (MonadIO m,GHC.GhcMonad m,HasIdeState m) 
                         => IdeRequest -> m IdeResponse

The monad used inside HIE is defined as

newtype IdeM a = IdeM { unIdeM :: GM.GhcModT (GM.GmOutT (StateT IdeState IO)) a}
      deriving ( Functor
               , Applicative
               , Alternative
               , Monad
               , MonadPlus
               , MonadIO
               , GM.GmEnv
               , GM.GmOut
               , GM.MonadIO
               , ExceptionMonad
               )

data IdeState = IdeState
  {
    idePlugins :: Plugins
  } deriving (Show)

There is a specific haskell-ide-plugin-api package for the Dispatcher type, and IdeM is defined
in the main haskell-ide-egine package.

There are a number of options for this.

1. Leave things as they are. But then the GhcModT is not available to a plugin action.
2. Make Dispatcher have the signature IdeRequest -> IdeM IdeResponse
   This means the IdeM type has to be in haskell-ide-plugin-api, and the various instances
   required become orphans, or need to be in haskell-ide-plugin-api.
3. Include GhcModT in the constraints on the Dispatcher

Another consideration is that the haskell-ide-engine looks like it may be a
general IDE backend, usable for other languages e.g. PureScript, Idris, et al.
In this case having anything GHC related is unneccessary. In this case,
perhaps the type should be simply

type Dispatcher = forall m. (MonadIO m,HasIdeState m) 
                         => IdeRequest -> m IdeResponse

At the moment in MainHie.hs we have

    -- TODO: pass port in as a param from GlobalOpts
    _ <- forkIO (jsonHttpListener cin)

Modify Options.hs to have a flag to enable the listener, and only launch it if the flag is set.

This should be a copy-paste of the option for "--console", just renamed.

As a bonus, consider adding a port parameter too.

via stack:

jpmoresmau:~/Documents/haskell-ide-engine$ stack exec hie
{"cmd":"ghcmod:type","params":{"file":{"file":"/home/jpmoresmau/Documents/haskell-ide-engine/src/Haskell/Ide/Engine/Types.hs"},"start_pos":{"pos":[21,9]}}}
^B
{"type_info":"21 5 21 17 \"TChan ChannelResponse\"\n21 5 21 17 \"ChannelRequest -> TChan ChannelResponse\"\n21 5 21 17 \"ChannelRequest\"\n21 5 21 17 \"TChan ChannelResponse\"\n"}

This seems ok.
Via the executable directly:

jpmoresmau:~/Documents/haskell-ide-engine$ .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/hie/hie
{"cmd":"ghcmod:type","params":{"file":{"file":"/home/jpmoresmau/Documents/haskell-ide-engine/src/Haskell/Ide/Engine/Types.hs"},"start_pos":{"pos":[21,9]}}}
^B
EXCEPTION: types:
       Could not find module ‘Data.Aeson’
       Perhaps you meant Data.Version (from base-4.8.1.0)
       Use -v to see a list of the files searched for.
       Could not find module ‘Control.Concurrent.STM.TChan’
       Perhaps you meant
         Control.Concurrent.Chan (from base-4.8.1.0)
         Control.Concurrent.MVar (from base-4.8.1.0)
       Use -v to see a list of the files searched for.
       Could not find module ‘Haskell.Ide.Engine.PluginDescriptor’
       Use -v to see a list of the files searched for.
{"type_info":""}

What does stack exec do that calling the executable directly doesn't do? Surely calling the executable directly should also work.

The PluginDescriptor includes function references. It should never be sent over a wire anywhere.

Instead, for each Command the CommandDescriptor should be sent.

Based on an IRC discussion with @hamishmack
#61 And #66 are also relevant.

There are two different but related issues

1. Given that we end up with a specific set of well-known types that can be managed by a given IDE, we need to be able to manage versioning of these, if the format changes.
2. If we end up with a set of semantic types / widgets that can be meaningfully applied in an IDE, we will end up with uneven implementation of these in the set of supporting IDEs.

Both of these seem to require some kind of content negotiation.

A possible solution would be to include a description of the content types (in the sense of semantic type / widget) together with explicit versions in each request from the IDE for a given command. The supported types for a given command could potentially be returned as part of the CommandDesc.

Now that stack and stack-ide are here, it should be fairly straightforward to create a web based ide.

I can think of

• websocket proxy between web ui and stack-ide process
• ace9 for editor (see the kitchen sink demo and choose haskell https://ace.c9.io/build/kitchen-sink.html)
• git integration for free via an open source tool like git-webui
• hoogle, HaRE, haddock, hlint, codex all setup by stack directly and proxied by ide rest api
• terminal for free with some open source tool like shellinabox or whatever found on github (https://github.com/search?utf8=%E2%9C%93&q=web+terminal+)

We can also leverage docker for packaging and deploying components, or even offering users some quick scripts to package apps, etc.

What do you think ?

Because JSON is used for external interfaces for many of the transports, the JSON generated must be stable. Hence it should not be generated by automated means that may change with compiler or library version.

Replace all ToJSON instances with hand-written ones, which make the encoding explicit.

We are hand coding the JSON instances.

Provide quickcheck tests that confirm that we can successfully roundtrip all JSON instances.

i.e. decode . encode == id

At the moment the HaRe plugin only supports the "rename" refactoring.

Add

• rename
• demote
• dupdef
• iftocase
• liftOneLevel
• liftToTopLevel

based on the way rename is done

We can either use / extend an existing protocol, or have an entirely new one. Lets discuss what will work best.

Pick an existing protocol

If we pick an existing protocol, then ideally, this would be one that has many editor integrations. So, the primary benefit of doing this would be to get lots of code reuse and editor support "for free". Unfortunately, it doesn't seem like there is yet a de-facto standard here. Even if there was, I'm thinking that picking something which is intended for a non-haskell-ey language will be too much impedance mismatch.

There may well be some potential for cross-polination with haskell-like languages, though. Idris's protocol seems like an interesting possibility (see #3).

New protocol

Do a new protocol. To do this I think it'd make sense to take a look at all the existing protocols, and consider the attributes of its design. Here are a few GHC specific protocols to take a look at:

1. stack-ide, which also use the types from ide-backend-common. The particularly nice thing here is the amount of juicy details yielded by ide-backend-common, particularly in IdInfo. However, it also ends up being up to the editor what to do with all this this info.

Something I think should be done differently than this protocol is to provide less mutation. Instead, the query protocol should just give ways to ask for info about code, based on what's in the filesystem and the CLI arguments. Being able to change GHC flags on the fly is pretty cool, but getting it perfectly correct is tricky.

2. ghc-server has some rather nice protocol types, but for Chris has been superseded by

3. ghci-ng, which simply adds a few new ghci commands which yield machine-readable output. This is also the approach taken by Idris's repl based IDE integration.

I've got some more thoughts on what such a protocol ought to look like / provide, but I'll save it for later :)

I'm thinking of calling this haskell-ide-plugin-info (these names are going to get unwieldy, maybe abbreviated to hie-plugin-info or hiep-info.

ide-backend has some pretty excellent type information for sub-expressions. So, I'm thinking it would be a good first step to try implementing that, copying / referring to ide-backend's implementation. So, this will look like having a plugin with the following command:

  CommandDescriptor
  { cmdName = "type"
  , cmdUiDescription = "get the type of a sub-expression"
  , cmdContexts = [CtxRegion]
  , cmdAdditionalParams = []
  }

It'll return a string with the type pretty printed.

We now have an ability to run an async plugin, through

data CommandFunc resp = CmdSync (SyncCommandFunc resp)
                      | CmdAsync (AsyncCommandFunc resp)
                        -- ^ Note: does not forkIO, the command must decide when
                        -- to do this.

type SyncCommandFunc resp = forall m. (MonadIO m,GHC.GhcMonad m,HasIdeState m)
                => [AcceptedContext] -> IdeRequest -> m (IdeResponse resp)

type AsyncCommandFunc resp = forall m. (MonadIO m,GHC.GhcMonad m,HasIdeState m)
                => (IdeResponse resp -> IO ()) -> [AcceptedContext] -> IdeRequest -> m ()

So an async command function gets given a function to call in IO to return the IdeResponse, but is otherwise unconstrained.

The intention is that there can either be a single command function which can server as a dispatcher to other async processes, or the function can fork off a separate process to handle the request.

In either case we potentially have multiple processes running, all wanting to make use of information from the underlying project.

At the moment a PluginDescriptor is defined as

data PluginDescriptor = PluginDescriptor
  { pdCommands        :: [Command]
  , pdExposedServices :: [Service]
  , pdUsedServices    :: [Service]
  }

The Service type is not really defined an unused at this point.

The intention is that a Service can be used as a synchronisation mechanism, and can expose access functions that are specifically multiprocess aware.

So there would be a base service providing access to the underlying project, which would make sure that e.g. more than one command does not trigger ghc-mod to reconfigure the project.

The question is, how should a Service be defined in concrete terms.

pipes-parse doesn't consume the input if it doesn't parse which results in us repeatedly trying to parse the same input which results in an perpetual flood of error messages.

My pipes foo is too weak to figure out how to fix this right now.

Initial work on Emacs menu:

This issue tracks the progress and will be closed by merging this work to master.

The context variables should rather come through as standard parameters

TravisCI reports:

The command "./travis_long stack +RTS -N2 -RTS build --test --pedantic" exited with 1

This prevents next line of script to be run so my Emacs tests aren't run.

If there is an error in any of the plugin commands, catch the error and return it wrapped in IdeResponseError

This should be done in the main dispatcher loop:

doDispatch :: PluginId -> PluginDescriptor -> Dispatcher
doDispatch pn desc req = do
  plugins <- getPlugins
  debugm $ "doDispatch:desc=" ++ show desc
  debugm $ "doDispatch:req=" ++ show req
  case Map.lookup (pn,ideCommand req) (pluginCache plugins) of
    Nothing -> return (IdeResponseError (toJSON $ "No such command:" <> ideCommand req))
    Just cmd -> (cmdFunc cmd) req

The call to (cmdFunc cmd) req should be wrapped in an error handler.

Each command should be as self-contained as possible, It should preferably not depend on hidden state at all. Visible state, like contents of visible files, is less of a problem.

Encourage stateless programming in what is supposed to be async system.

Encourage using full absolute paths to files because current working directory is state.

Where should the top of the project set

Language.Haskell.Support
Language.Haskell.Daemon
Language.Haskell.IDE.Daemon
Tools.Haskell.Daemon
Tools.Haskell.Tools

Let the bikeshedding begin

Following a discussion on IRC between @alanz and @gracjan, we think commands should be explicit in the type of response they return, and we should offer some concrete types out of the box, so that plugin authors can reuse the types for responses, which in turn means that editors using the API can have a good chance at supporting a wide range of plugin commands.

The idea is to define a type class that contains a function to serialize to something generic like JSON.

I have started some code and it looks promising (the fact that command definition carry the type they return). I obviously need a bit more time.

Here is an other tool to add in the list of tools to incorporate: hsimport
Extend the import list of a Haskell source file
https://hackage.haskell.org/package/hsimport-0.7
https://github.com/dan-t/hsimport

can't do the PR now

When HIE starts up, it is presented with a list of Plugins.

Each of these defines commands in a CommandDescriptor

For each CommandDescriptor, check that the parameter names introduced via cmdContexts and cmdAdditionalParameters are unique.

If this is not the case, exit with an error.

At the moment the Console is incredibly basic.

It should be reworked to provide the following basics

• History. Pressing up/down should scroll through prior history, allow editing
• Better command parsing
  At the moment the command parsing is as simple as possible, for the POC. This should be extended in some way, perhaps by generating parsers from plugin command descriptors UiCommand
• Better representation of output
• Built in commands to browse the plugins, and get help on them. This should just be wrappers around the functions exposed via the base plugin
• Anything else

The fledgeling HIE has a JSON API. This includes some introspection as to what plugins are available, and what commands they provide, together with their parameters.

Provide an application that can interact with the API, and present the results in a meaningful way.

i.e.

$ stack build
$ stack exec hie -- --repl -d

Then access http://localhost:8001/ as in

curl -v http://localhost:8081/req/base -X POST -H Content-Type:application/json --data-binary '{"ideParams":{},"ideCommand":"version","ideContext":{"ctxEndPos":null,"ctxCabal":null,"ctxStartPos":null,"ctxFile":null}}'

Possibly consider using http://swagger.io, especially as there is now servant-swagger

Or any other appropriate technology. It needs to be able to run locally though.

cc @kritzcreek

haskell-ide has a potentially very large scope.

A useful approach to managing this is to identify some useful, uncontroversial, small functionality that can test our architectural choices.

I think there are three broad categories of things that can be provided via haskell-ide

• A place where a tool writer can host a tool as a plugin, and make use of standardised services to access the raw project and communication with the IDE, thus minimising this effort for all to get the tool integrated into multiple environments.

• A way to query information about a project, and present it to the user. This in turn is a large topic, and can cover anything from contents of available libraries, to type information of loaded modules, to information to support syntax highlighting.

  This may include on the fly error checking

• Providing access to a REPL

So I suggest we come up with something from one of these categories, and aim to let it communicate with at least 2 different IDEs via the process identified in #2 / #3

What should these be?

At the moment the plugin API package is called haskell-ide-plugin-api.

Before this goes on to hackage, we must agree on an appropriate package name for it, and a naming scheme for packages in general.

Some ideas to kick off the discussion

• haskell-ide-plugin-api (i.e. stays as is)
• haskell-ide-engine-api
• hie-plugin-api
• others?

In a related fashion, a given plugin can be called

• haskell-ide-plugin-hare
• hie-hare
• hiep-hare
• hie-plugin-hare

Note: I fully expect future tools to provide the PluginDescriptor as part of the normal package, so HaRe would simply export

hareDescriptor :: PluginDescriptor

I'm thinking that a good way to implement plugins is by emulating ghc's plugin system, perhaps even calling functions from it. Based on a quick look at the DynamicLoading module, it seems like it'd be a good approach to just copy+modify the loadPlugin function.

It makes sense to have something similar to the Plugin datatype, and have a field per "callback". Perhaps this is getting into details too early, but maybe these callbacks should be wrapped in Maybes, like the aptly named Hooks datatype. It's informative to know which things a particular plugin hooks into.

A CommandFunc is defined as

type CommandFunc = forall m. (MonadIO m,GHC.GhcMonad m,HasIdeState m)
                => [AcceptedContext] -> IdeRequest -> m IdeResponse

It will only be called by the dispatcher if at least one context matches, and and the required parameters are present and of the right type.

At the moment the command processing function is very messy

checkCmd :: CommandFunc
checkCmd _ctxs req = do
  case getParams ["file"] req of
    Left err -> return err
    Right [ParamFile fileName] -> do
      liftIO $ runGhcModCommand (GM.checkSyntax [T.unpack fileName])
    Right x -> error $ "GhcModPlugin.checkCmd: got unexpected file param:" ++ show x

In this case there is only one valid context, so it it is called at all the "file" parameter is guaranteed to be valid. The getParams call returns a list of ParamValue in the order of the param names given to it, or a error result if they are missing. This call is actually redundant now.

It may be simpler to define the following types

data ActualRequiredParams =
    AR0
  | AR1 ParamVal
  | AR2 ParamVal ParamVal
 ...

data ActualOptionalParams =
    AO0
  | AO1 (Maybe ParamVal)
  | AO2 (Maybe ParamVal) (Maybe ParamVal)
 ...

as well as something similar for the matching contexts.

The CommandFunc would then be defined as

type CommandFunc = forall m. (MonadIO m,GHC.GhcMonad m,HasIdeState m)
                => [ActualContextContext] -> ActualRequiredParams -> ActualOptionalParams -> m IdeResponse

This will still leave is with "missing case" warning messages though.

Is there some clever way to do this with types?