Currently if I want to create a CRUD style resource in Roxy I have to code the controller by hand. Please add a scaffolding command to do this for me.

ml create resource controller-name [format]

This should create all of the following functions and views for each with the specified format. If none is supplied as a format then no views will be created.

declare function c:index()
{
};

declare function c:new()
{
};

declare function c:create()
{
};

declare function c:show()
{
};

declare function c:edit()
{
};

declare function c:update()
{
};

declare function c:destroy()
{
};

Default is GET, which leaves the password in cleartext on the location bar. The code appears to be in user-lib.xqy, function uv:build-login.

If any of the standard users or roles (nobody, app-user) are used in the build.properties file, then they are removed during 'wipe', If the definitions for these built-in items haven't been copied into ml-config.xml, then they won't be restored by bootstrap. In addition, any other application that relies on them will be broken until they've been restored.

An enhancement would be to skip over the standard users and roles during wipe.

The code modules say that Roxy is licensed under Apache License, Version 2.0. It would be good to make this more prominent by adding this to the readme file and/or as a separate license file.

Hi all,

I would like to file a feature request:

In the scenario of having several applications that offer different views over the same content, it will be extremely handy to be able to "synchronize" somehow the database properties (like range indexes and such).

Some ideas:

• Add a parameter to build.properties so we can explicitly indicate the ml-config.xml file to use.
• Add a parameter to ml.rb so we can explicitly indicate the ml-config.xml file to use.
• Mark application via build.properties as "slave" of certain application (so when deploying it can at least give a warning of why the local ml-config.xml is being ignored).

Regards,
Miguel

P.S: I could create a pull request for the first option if you think it's good enough.

I'd like to be able to deploy to a cluster, which means specifying the number of forests per host in the cluster. I'd like to configure what I need in the properties file, maybe like this:

cluster-nodes=ml-box-1,ml-box-2,ml-box-3
forests-per-node=2

If you set up a rewrite element like this:

declare variable $c:ROXY-ROUTES :=
  <routes xmlns="http://marklogic.com/appservices/rest">
    <request uri="^/bug" endpoint="/my/module.xqy"/>
    {
      $def:ROXY-ROUTES/rest:request
    }
  </routes>;

and then hit a URL like this:

/bug?uri=foo.xml

then the rewriter drops the uri parameter.

Seems that they weren't updated to handle the new load_properties function.

if you add a self-closing collation element on a string element range index as follows

    <range-element-index>
      <scalar-type>string</scalar-type>
      <namespace-uri></namespace-uri>
      <localname>KeywordEN</localname>
      <collation/>
      <range-value-positions>false</range-value-positions>
    </range-element-index>

then having run ml local bootstrap , the index is created with a unicode codepoint collation (http://marklogic.com/collation/codepoint). Running normal lexicon queries against it will fail saying the required index with the collation http://marklogic.com/collation was not found.

Does is make sense to change Roxy add the above root collation instead of the unicode collation when the collation element is not specified? Might be a safer default.

Document setup and deploying.

Add support for bootstrapping Range Path indexes in 6.0. Should fail gracefully for < 6.0

I'm working on a project where some of the configuration information is stored in the modules db, rather than the content db. In order to unit test this code, it is necessary to have a separate test-modules db, to prevent overwriting the configuration data in the production/dev db.

In several environments it is against the security policy for a password to be saved in cleartext.

I would suggest a method for Roxy to prompt the user for the password. Also, when invoking recordloader and other child processes, the password should not be passed in where a "ps -ef" will show the password.

forcing an app -reinitialization doesn't seem to work.

This is missing :-(

Add a word lexicon to ml-config.xml:

<word-lexicons>
    <word-lexicon>http://marklogic.com/collation/</word-lexicon>
</word-lexicons>

Run bootstrap and all's well. Run it a second time and you'll get an error.

Modify the deployer so that bootstrap will set up a scheduled task.

Note: I'm not sure whether it will work to have this as part of bootstrap, since the source code might not be deployed yet. If that doesn't work, then it might need to be a different command (ml {env} deploy tasks), but hopefully it can be part of bootstrap.

I want to set my default controller and function to /customresearch/search
But when I overwrite the default controller in config.xqy with "customresearch" I notice that using http://localhost:8040 doesn't work .. The reason being that the function is defaulting to main() which is fine for app builder but not for my case..

When I overwrite that as follows

config.xqy:

(: the default function that the URL http://server:port/ maps to :)
declare variable $DEFAULT-FUNCTION := "search";

/roxy/rewrite.xqy

<rest:request uri="^/([\w\d_\-]*)/?([\w\d_\-]*)\.?(\w*)/?$" endpoint="/roxy/query-router.xqy">
  <rest:uri-param name="controller" default="{$config:DEFAULT-CONTROLLER}">$1</rest:uri-param>
  <rest:uri-param name="func" default="{$config:DEFAULT-FUNCTION}">$2</rest:uri-param>
  <rest:uri-param name="format">$3</rest:uri-param>
  <rest:http method="GET"/>
  <rest:http method="HEAD"/>
</rest:request>

Then all works fine. It might be an idea to build this into the default deployment and set DEFAULT-FUNCTION to main() so that appbuilder still works out of the box.

Fully removing app servers from MarkLogic requires a restart. If you do "ml {env} wipe" followed by "ml {env} bootstrap", you're told that a restart is necessary. You can then to "ml {env} restart". However, this only restarts the target box, not the cluster, so other servers in the cluster may still be in a funky state. Update the restart command to restart the cluster. Update documentation accordingly.

Much like we do for missing controllers we should show a handy, dandy error message for SEC-PRIV that directs the Roxy user where to add the missing privileges.

I was trying to setup a element word lexicon to use with search:suggest and I tried adding the following which seemed to make sense:

Though this didn't work. Can we get this added to the roxy deployer?

Thanks,

Andy...

Something like: "ml local capture", which would use the application name, servers, and ports, look at the target server, and build deploy/ml-config.xml. It would probably make sense for this to capture particular features (db settings, range indexes) rather than the whole config, so that the substitutions could remain in place.

A MISSING-VIEW (XDMP-MODNOTFOUND) error is thrown, when trying to use:
ch:use-view("welcome")

from a controller method.

WORKAROUND: This does work when you specify the controller in the use-view method as shown below:

ch:use-view("controller/welcome")

The latest version of MarkLogic, ML6, now support Path Range Indexes and this is not currently supported within Roxy.

The controller function code created by the scaffolding needs a () or something to not blow 500 errors.

Some shops will want to automate the source code repository retrieval process.

Can you please add a new command to retrieve the latest code from an SVN and VSS repository?

Example:

ml local getsvn
ml local getvss

Example build.properties for SVN:

svn-url=svn+ssh://[email protected]/project
svn-user=guest
svn-passwd=password

Example build.properties for VSS:

vss-dir=\\avalon\VSS\VSS_LSP
vss-user=guest
vss-passwd=password

would suggest changing

line 36

ml init

to

./ml init

I would like to add read permissions to my content when I do a "ml dev deploy content".

The list of environments is currently hard coded. Please allow users to dynamically specify the list of environments in build.properties.

It will be quite handy that during setup:apply-database-settings at xquery/setup.xqy, instead of doing nothing whenever a value it's not set, it would default to some other value.

Like this, we would make sure that the database configuration and the configuration are perfectly aligned after a bootstrap.

Example:

let $value := setup:get-setting-from-database-config-as-string($database-config, "directory-creation")
let $admin-config :=
   if ($value) then
       admin:database-set-directory-creation($admin-config, $database, $value)
  else
       admin:database-set-directory-creation($admin-config, $database, "manual")

After a bootstrap it won't matter if somebody changed the setting manually or somebody removed the line from the config file. The setting will default to Manual and automatic directory creation will be disabled.

Thanks!

Again: I could send a pull request if you think it's interesting for everybody else.

In a Roxy application I've create an element range index with the default collation (root collation -> http://marklogic.com/collation/) this resulted to the following:

• the unit tests were working just fine
• the application itself was giving the error that the element range index can not be found.

Switched the collation type of the index to unicode codepoint (http://marklogic.com/collation/codepoint):

• the application was working fine
  • the unit tests were giving the error that the element range index can not be found.

I've tested the application using the test DB, this won't happen in the feature, but it would be very uncomfortable to have two DB configs to maintain - one for the test DB and one for the app DB and just keep on changing the index collations.

Thank you!
Silvia

The code inside the xdmp:eval() in setup:configure-http-server() in deploy/lib/xquery/setup.xqy needs a namespace declaration for err (declare namespace err="http://marklogic.com/xdmp/error";). The namespace is used in the catch clause.

dcassel@madison:roxy $ ml init my-app --force
ERROR: Init has already been run. Use --force to rerun it.

ERROR: Missing environment for --force

When inserting documents into the modules database (and potentially the content database also), ignore the filesystem's path, and just place the files under "src" into the database.

/Users/pmcgowan/Documents/workspace/marklogic-roxy//custom/config/datasource-config.xqy

should be

/custom/config/datasource-config.xqy

On "ml local bootstrap", the script responds with "Bootstrap Complete" with no error message, even if there was an error. The error can be seen with "ml local bootstrap -v". Having an error in deploy/ml-config.xml (for instance, a mismatched closing tag) will trigger the problem.

When bootstrapping, the setup script looks for any privileges referred to by deploy/ml-config.xml. For any it finds, it first deletes them, then recreates them based on the specification in deploy/ml-config.xml. I think the motivation for this is to avoid accidentally having a definition of an execute privilege that doesn't match what's in your config file. The problem is that if I have a role X with privilege A, and then I bootstrap a project that creates role Y with privilege A, then because A gets deleted and recreated, role X no longer has it. A couple possible solutions come to mind:

1. Don't delete then recreate. If a privilege exists, just leave it alone.
   -- Possible side effect: if the name of a privilege in my project conflicts with the name of another project's privilege, my project will accidentally end up using the one defined by the other project. This would solve the problem of standard privileges getting removed from roles. Encouraging a naming convention for project-specific privileges might be enough here.
2. As part of setup:create-privilege(), first record which roles have the privilege, then restore when the privilege gets recreated.
   -- Side effects possible here. Suppose my project has a privilege with some URI and another project uses the same name but a different URI. The URI associated with the name will depend on which project has bootstrapped more recently. Again, this would solve the problem for standard privileges.
3. Change the config to explicitly define the privileges you want, then just refer to them by name when assigning them to roles. So you might have /configuration/sec:privileges/sec:privilege to define an app-specific privilege, then in /configuration/sec:roles/sec:role/sec:privileges, you'd only have a sec:privilege-name, not the action or type.
   -- Conflicts with other project-specific privileges would still be possible, but at least the standard privileges would be left alone. We should encourage naming conventions to avoid project-specific privilege name collisions.

I'm thinking #1 would be sufficient. Comments?

Adding a geospatial-element-child-index to deploy/ml-config.xml does not let "ml {env} bootstrap" create that index

• Set up the deploy script so that it will build the index if provided
• Add a sample index comment in the deploy/ml-config.xml file

Adding a geospatial-element-index to deploy/ml-config.xml does not let "ml {env} bootstrap" create that index

• Set up the deploy script so that it will build the index if provided
• Add a sample index comment in the deploy/ml-config.xml file

Add samples for range-element-attribute-index and geospatial indexes

Clearing out the pre-installed CA's takes twice as long as it takes to install MarkLogic. Would be nice to have the ability to clear the default CA's, and also load a group of CA's from a text file, similar to the functionality provided using the import feature on the ML CA admin page.

Should be able to specify which group of nodes to deploy content database on, in case cluster has separate E and D groups.

It would be very useful to have the bootstrap process be able to create groups and assign hosts to those groups. The configuration would need to allow you to specify which group or groups you want your application installed on.

Following the instructions in the README the user is instructed to modify app-modules-db. The option is not in the build.properties generated after executing "ml init".

It's not clear how you can call a view from a view.
ch:use-layout((), "xml"),
ch:use-layout("...", "html")
a bit more information how to use those helpers in Single page websites

declare function c:register() as item()*
{
ch:use-view("welcome", ("html")),
lib:something-useful()
};
This says that when a client requests /user/register.html, use the view at /src/views/register.html.xqy.

This should use the view at: /src/views/welcome.html.xqy

Some of the catch clauses in deploy/lib/xquery/setup.xqy log the exception, but many do not. It would be very helpful to log at least some of these that are not currently logged. The two in particular that would have saved me some time are in setup:apply-database-settings() and setup:configure-http-server(). In general, I would think any failure that would cause a significant portion of the configuration not to succeed (like configuring a database or app server) should be logged.

I'd like to be able to set the number of forests for the content database for different environments.

Workaround: call "ml {env} wipe" twice, the first time gets the app servers and databases, the second time gets the forests.

Instead of creating 1 xcc server per every app instance, allow apps to share a single xcc server. Helps to reduce frivolous xcc servers.