As a user i would like to use Multidimensional analysis tools in MDCS, currently they are not support can it be added

https://pro.arcgis.com/en/pro-app/tool-reference/image-analyst/an-overview-of-the-multidimensional-analysis-toolset-in-image-analyst.htm

Hello Xin Xia,

Unfortunately I'm a consumer of this library as yourself, but I don't maintain it. It is managed by Esri's Imagery Workflows team. But you've got a good question that might help others so I'm going to post it as an issue on the Esri/mdcs-py repository, so the Workflows team will be able to check it out as well.

Thanks,
@mfunk

Hi Matt,

I found your email through the Mosaic Dataset Configuration Script (MDCS) GitHub. I have had success with running it on Command Prompt. However, what I really need is to make a tool out of the scripts to work in Model Builder. How do I go about loading all the separate python script files into a tool? How does the xml configuration file go into being used by a tool? Do they all need to be inputs to MDCS.py? Can this be done in a script tool or does it need to be done in a python toolbox tool?

Thank you,
Xin Xia

I'm using mdcs along with optimize rasters to create and then convert TIF overviews to MRF overviews. Then I want to run mdcs again to then swap the TIF overviews for the MRFs in the mosaic dataset. It works fine except the last step of adding the MRFs - below is the error and I've attached my batch file, screen shot of overviews folder and the remove raster config for mdcs. Any advice?

Adding rasters:
				<Error>
					<type>critical</type>
					<text>No new mosaic dataset items added to dataset (Test_Data). Verify the input data path/raster type is correct</text>
				</Error>
				<Error>

remove_raster_config.txt

example_batch.txt

We have all our mrf-data in eu-west-1 but are currently moving it to eu-north-1 to get lower latency in Norway.
This means that all our mosaics with embedded proxies need to change the bucket reference inside the embedded proxies.
Currently the real path to the mrf-data is something like this: euwestbucket/subfolder/mrf/...
In the future it will look something like this: eunorthbucket/subfolder/mrf/...
I've tried to export paths with mdtools, but only the old external mrfproxy-paths are shown e.g.:
/servicedata/Markfuktighet_DTW/mrfproxy/Markfuktighet_DTW.Overviews/Ov_i04_L01_R000000AE_C0000003B.tif
So my guess is that I need to add some special formatting before the bucketname to allow the MD to know where to look.
But what formatting is that?

The link on README under "Resources - ElevationScripts from ArcGIS Resource Center" is not working

ElevationScripts from ArcGIS Resource Center
[http://www.arcgis.com/home/item.html?id=f1e1020f94ab40219cf0d2f22a62ceee/]
takes you to login request, then informs you that you do not have permisson.

It should point to:

[http://www.arcgis.com/home/item.html?id=d2a055e12af14258a931fdc3ecf2c8b4]
Python script for creating, configuring, and populating mosaic datasets.

For those workflows (w/fs) that require to class initialize any user defined variables using the internal MDCS (data) object.

Presently when the user creates reference mosaic dataset raster function is necessary, add the functionality to allow the creation if it is not specified

Need to update Initialize_Script_Paths.bat to initilize for ArcGIS 10.5 and 10.6.

The layer name created when calculating a field in line 934 of solutionsLib.py can cause an error when there are multiple calculations and they all run quickly. The name is based on the timestamp to the nearest second. When two calculations are closer than a second from each other, they can have the same layer name.

Also, the day is used in place of the month in the layer name. This probably doesn't cause any problems.

Proposed solution: change format of layer name to include milliseconds and also put month in the right place:

lyrName = 'lyr_%s_%s' % (str(self.m_base.m_last_AT_ObjectID),datetime.strftime(datetime.now(),"%Y%m%d%H%M%S%f"))

Currently the default log files log the mosaic name , tool name and total time. It would be good to have the entire MDCS command logged in the log file as it will be useful to have it preserved in the log file especially when there are a lot of user variables used in the command line.

Having an additional issue with empty TIFS being created from the BO command, config is attached along with log file - runs without errors however the TIFs are quite small and nothing draws in ArcGIS Pro.

create_md_build_overview.txt
create_build_overviews_20220627T111706.txt

When I define and build overviews in ArcPro they come out fine:

arcpro_py_export.txt

The master.xml is not in sync with the new tools that have been added to the solution lib

Need to update the pdf file with updated documentation.
File name MDCS.PDF
Delete MDCS_v5.pdf

Having issues in making codeblocks work for the calculate value tool - followed the master.xml template but keep getting a Name Error:

	<CalculateValue>
                 <query></query>
		 <fieldName>CAPTURESTART</fieldName>
		 <expression>todate(!Name!)</expression>
		 <expression_type>PYTHON</expression_type>
		<code_block>"""def todate(value):\n import dateutil.parser as dparser\n  date = dparser.parse(next((x for x in (value.split("_")) if x[0].isdigit())), fuzzy=True, ignoretz=True)\n  return date"""</code_block>
	</CalculateValue>

Results in:

ERROR 000539: Traceback (most recent call last):
File "", line 1, in
NameError: name 'todate' is not defined

Can confirm same codeblock works in ArcPro calculate field. Any advice?

Curious what the outlook for being able to install MDTools on Arc Pro 3 is.

Thanks for your work!

Unable to create a Referenced MD using this command:

python.exe "C:\Image_Mgmt_Workflows\mdcs-py\scripts\MDCS.py" -i:"C:\Image_Mgmt_Workflows\mdcs-py\Parameter\Config\create_ref_md.xml"

With this template:

<?xml version="1.0" encoding="utf-8"?>
<Application>
	<Name>create_ref</Name>
	<Command>CR</Command>
	<Workspace>
		<WorkspacePath>C:\data\Services\Aerial\</WorkspacePath>
		<Geodatabase>Aerial_Public.gdb</Geodatabase>
		<MosaicDataset>
			<MosaicDatasetType>source</MosaicDatasetType>
			<Name>Aerial_Public</Name>
			<CreateReferencedMosaicDataset>
				<in_dataset>Aerial_Public</in_dataset>
				<out_mosaic_dataset>Aerial_Public_R</out_mosaic_dataset>
				<SRS></SRS>
				<number_of_bands></number_of_bands>
				<pixel_type></pixel_type>
				<where_clause>Latest = 1</where_clause>
				<in_template_dataset></in_template_dataset>
				<extent></extent>
				<select_using_features></select_using_features>
				<lod_field></lod_field>
				<minPS_field>MinPS</minPS_field>
				<maxPS_field>MaxPS</maxPS_field>
				<pixelSize></pixelSize>
				<build_boundary>BUILD_BOUNDARY</build_boundary>
			</CreateReferencedMosaicDataset>
			<AddRasters>
			</AddRasters>
			<Processes>
			</Processes>
		</MosaicDataset>
	</Workspace>
</Application>

There's no errors in the logfile, but no reference is created. I've tried hardcoding paths in <in_dataset> and <out_mosaic_dataset>, as well as using m: and p: in the console to override the mosaic and output but no luck. What am I missing?

Not urgent, but I noticed that when running the mdtools.exe executable without any arguments, the following arguments are shown:

This does not match the list of arguments (or the case/naming) from the PDF documentation - I assume the .exe just needs a quick update on the allowed params, since it includes a --PD argument that is not in the docs, and doesn't break out the --op, for example.

When using a batch file to create a series of similar Mosaic datasets from the same configuration file (see extract below) I am not able to use variables in the batch and configuration files.

The variables seem to be ignored and the default value is always used. Also: how is a variable used when the variable name is non-unique in the configuration XML? (as is the case with <Name> which occurs twice at the top of the file). I am getting a
log-critical: Error: <Name> should be the first child-element in <MosaicDataset>
and then the MDCS script fails.

The help specifies the following syntax:
<defaultValue>;$<variableName>

which in my case for below configuration file looks as follows when I want the <MosaicDataset><Name>, <dataset_id> and <filter> variables to be dynamically specified:

c:\python27\ArcGIS10.3\python.exe C:\Image_Mgmt_Workflow\MDCS\scripts\mdcs.py -i:C:\Image_Mgmt_Workflow\MDCS\Parameter\Config\MPI_CreateMosaicDataset_GENERIC_AlbersEqualArea.xml -p:CatchMapper_MPI_AlbersEqualArea_MDCS_Demo_Ficticious$MosaicDataset/Name -p:CatchMapper_MPI_AlbersEqualArea_MDCS_Demo_Year01$dataset_id -p:year=1$filter

<?xml version="1.0" encoding="utf-8"?>
<Application>
  <Name>CatchMapper_MPI_AlbersEqualArea_MDCS_Demo_Year_GENERIC</Name>
  <Command>CM+AR+AF+BB+ERF+SP+CC+CV+AI+DO+BO</Command>
  <Workspace>
    <WorkspacePath>md</WorkspacePath>
    <Geodatabase>CatchMapper_MPI_Demo_YearlyMosaics</Geodatabase>
    <MosaicDataset>
     <Name>CatchMapper_MPI_AlbersEqualArea_MDCS_Demo_Year_GENERIC;$</Name>
      <MosaicDatasetType>Source</MosaicDatasetType>
      <SRS>PROJCS['NZ_Albers_Equal_Area',GEOGCS['GCS_WGS_1984',DATUM
['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM
['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Albers'],PARAMETER
['false_easting',0.0],PARAMETER['false_northing',0.0],PARAMETER
['central_meridian',175.0],PARAMETER['standard_parallel_1',-30.0],PARAMETER
['standard_parallel_2',-50.0],PARAMETER['latitude_of_origin',-40.0],UNIT['Meter',1.0]]</SRS>
      <num_bands>1</num_bands>
      <pixel_type>8_BIT_UNSIGNED</pixel_type>
      <AddRasters>
       <AddRaster>
         <dataset_id>MPI8bit_mosaics_Albers_Year05Grouped;$</dataset_id>
         <raster_type>Table @ C:\Image_Mgmt_Workflow\MDCS\MD\CatchMapper_MPI_Demo_YearlyMosaics.gdb\CatchMapper_MPI_AlbersEqualArea_MDCS_Demo\RasterType.OBJECTID=3</raster_type>
      <Sources>   <data_path>C:\Image_Mgmt_Workflow\MDCS\MD\CatchMapper_MPI_Demo.gdb\CatchMapper_MPI_AlbersEqualArea_MDCS_Demo</data_path>
      </Sources>
      <filter>year=1;$</filter>
      <duplicate_items_action>OVERWRITE_DUPLICATES</duplicate_items_action>
    </AddRaster>
    </AddRasters>

Thanks for any help on this.

I uninstalled ArcMap and so the folder C:/Python27 was also uninstalled.
I have Pro on my machine but the Python repo is not installed on C:/ as ArcMap.
The tool doesn't work if this file is not available...

C:\work\orto>"c:\Program Files\ArcGIS\Pro\bin\mdtools.exe" --md=MosaicDataset_ep.gdb\s3047 --embedrasterproxy
. Initializing ArcObjects...
...FAILED to initialize license...(Line: 52)
C:\work\orto>

Update the support for raster type for .art files

In Set Environment for Pyramid level default value should be -1. When it passes blank or null in config xml like

For copy raster command to get CRF doesn't get all the pyramid values.
So it has to be passed

-1

I can see MakeOPeNDAPRasterLayer. Is there way to create a mosaic dataset using a set of OPeNDAP URLs as raster source via MDCS?

Currently when creating mosaic, I must use a well-known text string representation (like "PROJCS['WGS_1984_Web_Mercator_Auxiliar...."). But I would love to use Factory code directly (4326) and for both projections (horizontal and vertical)
Here is the possible solution to change script in CreateMD.py

and in xml file:

Current MDTools is failing for advanced named user license. @imrankhan-pathan has fix. Need to build a new Setup file with this fix

On a machine with both ArcGIS Desktop 10.8.1 and ArcGIS Pro 2.8.0, running MDTools_Setup.exe only shows the option to install components into Desktop, not Pro.

Hi,

I've just downloaded and unzipped the tool.
When I ran the MDTool_Setup.exe, I only saw the option for ArcMap 10.7 components. However, while I have Pro 2.5 installed, I would like to utilize the tools in Pro.

I further looked up C:\Program Files\ArcGIS\Pro\bin, but I still failed to find the MDTool.

Is the tool support Pro 2.5?

Best,
April

C:\Image_Mgmt_Workflows\MDCS\Sentinel-2-L2A-AWS\scripts\MDCS.py:208: SyntaxWarning: "is not" with a literal. Did you mean "!="?

if verMessage is not 'Ignore':

I am trying to use the RemoveRastersFromMosaicDataset and where_clause is supported in this. But when I put the following string in the where_clause for the StdTime column which is Date format it throughs an error: "not well-formed (invalid token): line 125, column 80"
The have tried:

1. <where_clause>StdTime >= timestamp '2021-01-01 00:00:00' And StdTime <= timestamp '2021-01-10 00:00:00'</where_clause>
2. <where_clause>"StdTime >= timestamp '2021-01-01 00:00:00' And StdTime <= timestamp '2021-01-10 00:00:00'"</where_clause>
   But when I use the same clause in the ArcPro using the Remove Rasters From Mosaic Dataset it work. See the screen grab below:
   

Tool Link: https://pro.arcgis.com/en/pro-app/latest/tool-reference/data-management/transfer-files.htm

Hi I just tried publishing a service using publishing commands and it worked.

I also wanted to select service Capabilities like WCS and WMS selected at time of publishing. tried to look in documentation and online did not find any clue how to pass Capabilities information into service definition or service definition draft.

update mdtool setup to support ArcGIS pro version 2.3 and 2.4

Initialize_Script_Paths.bat needs to be updated to support later version of Desktop and Pro.

Adding new tools and updating existing ones as updated in 10.4

Steps to reproduce :

• Open ArcGIS Pro

• Launch the Publish Image service GPtool

• Fill in required parameters

• Open your server manager create the required folder

• In ArcGIS Pro type the same folder name

• publish the service, the tool fails

in this case the tool fails to create the folder as it exists, ideally it should check if the folder exists continue else, create it if does not exists

When using the 'Add Rasters' geoprocessing tool in ArcGIS Pro, you can provide a filepath to a folder that contains sub-folders where the images are stored. This is useful in cases where rasters are stored in groups (according to extents or grid references etc.), as it prevents a user from having to rearrange their file structure so that all of the images files are stored within a single folder.

The MDCS script does not currently allow for a filepath to a folder that contains sub-folders, instead, you have to provide a filepath to a folder that contains all of the rasters you want to add to a mosaic dataset. This means that a user needs to rearrange their file structure so that all the data is stored in a single folder, which can be quite time consuming.

Is there any way around this? Could this be handled as a bit of custom code in the MDCS_UC script?

Thanks,
Alex

Hello,
Import Field Value command ('IF' is its code) is not listed in the documentation ... I've found its node <<ImportFieldValues> into the "master.xml" file.
Anyway, I have seen that in its node there is the parameter .
Unfortunately this parameter is not considered by the script code of this command.
Looking at its content, I discovered that all fields of the join table are copied in to the the target table ( that is the footprint AMD table).
I think this is quite dangerous if the join table contains the same field name of the target table and you don't want to copy its content in to the target table ...

presently some of the input parameters are not exposed in the master.xml for create referenced Mosaic dataset, add support to them

The file MDCS_UC.py have some examples among which I can find "sample02."

def sample02(self, data):
log = data['log'] # How to use logging within the user function.
log.Message('Returning multiple values', 0)
data['useResponse'] = True
data['response'] = ['msg0', 'msg1', 'msg2']
data['status'] = True # overall function status
return True # True must be returned if data['useResponse'] is required. data['response'] can be used to return multiple values.

Is there anyone who knows how to actually the values in data['response']?
Is there any documentation?

I'm building my own custom commands and want to return som values from such a command. Just need to know how to do it. :)

//Anders

Hello,
I did not found any command to clip the geometry of boundary or footprints and replace their original geometry with the new clipped geometry.
Is it under development ?
Thanks a lot

Logger Module is not able to print messages with unicode characters in it with python 3.7.10 in ArcGIS Pro 2.8+
Affects all workflows using this module.

Not all customers use MDCS from a machine that is connected to the internet. When they aren't connected to the internet the MDCS version checker hangs and slows the process down by 5 or 10 minutes. Remove those lines so that we don't include it.

I am trying to publish a mosaic dataset on an EC2 machine (where our ESRI Enterprise and Image Server resides) using raster proxies created with the Optimize raster tool. The source .MRF images have been loaded into an AWS S3 bucket, I have provided the AWS credentials to the EC2 machine using the AWS CLI. The mosaic dataset fails to reference the S3 bucket. I have tried loading the mosaic using a .csv table and altering the source URL to include (\VSIS3\Http:\) and it still fails to reference the imagery. Is there anything obvious that I am missing that will solve this issue?

Hello,
I'm experiencing a trouble in using the MTC processing code.
The Tile Cache seems to computed correctly (I can load and visualize) even if I get the bad message "Failed".
Looking at the code in the script SolutionLib.py I guess that the reason of the failure is due to two procedures that are not defined:
returnLevelDetails and modifyConfProperties.
Is it so ?
Thanks in advance for your attention
Giuseppe

As a user when i write my own user command i would like to have an access to all the input command line argument as well as have a way to update them
currently only Mosaic dataset name is allowed.

New setup of MDTools which fixes the license error when standard license of ArcGIS Pro is used to perform the tasks such as embedrasters, search and replace, export raster paths.