Oracle seem to have changed the /otn-pub/ component of the JDK download paths to /otn/. Checked for JDK8 and JDK11. Perhaps change that part of the url search string to /otn.*/, to be more resilient?

Looks like they added an additional link on the download page for NotepadPlusPlus that confuses the URLTextSearcher. I switched to using the following already in the recipe which seems to have fixed the download issue:



Processor
URLTextSearcher
Arguments

url
https://notepad-plus-plus.org/update/getDownloadUrl.php
re_pattern
(?P[0-9.]+)]]>



Processor
URLTextSearcher
Arguments

url
https://notepad-plus-plus.org/update/getDownloadUrl.php
re_pattern
(?Phttp.*)<\/Location>]]>

Hi - I've just run the Aquamacs.download.recipe and gotten the following error:
Error in com.github.hansen-m.download.aquamacs: Processor: URLDownloader: Error: Couldn't download https://github.com/davidswelt/aquamacs-emacs/releases/download/Aquamacs-3.2/Aquamacs-Emacs-3.2.dmg: HTTP Error 404: Not Found
It seems like the version numbers have drastically changed for this location.

Would it be possible to revamp this to download from here?
http://aquamacs.org/download-release.shtml

Thanks,

Jake

Hi there, I noticed a tiny error in the XML. Got some errors when trying to make an override locally...

line 16 starts with starts with string but ends with key

The latest version of Epson Printer Drivers from Apple (2.18) uses a different file name: EPSONPrintDriver2.18.dmg (as opposed to EPSONPrinterDriver in previous releases). Not certain whether this is a permanent change or not from Apple, but the recipe fails in its current form, not seeing the new update.

One example: running GoogleChrome-Win.download recipe fails with error "Unknown processor 'BESRelevanceProvider'." I believe this is because it references the processor as 'BESRelevanceProvider' and not by the full SharedProcessor path 'com.github.hansen-m.SharedProcessors/BESRelevanceProvider' instead.

Apologies if the culture surrounding this sort of thing should be "fix it yourself and make a pull request" but I just started with git and don't know how to do any of that.

Just started getting these errors on the Prey.munki recipe which references your download recipe.

Processor: URLTextSearcher: Error: No match found on URL: https://preyproject.com/download

Im not great at REGEX I noticed that the format of the download link may have changed slightly.

The current structure is

https://downloads.preyproject.com/prey-client-releases/node-client/1.7.5/prey-mac-1.7.5-x64.pkg

(?P<url>https://downloads.preyproject.com/prey-client-releases/node-client/(?P<version>[0-9.]+)/prey-mac-[0-9.]+-x64.pkg)

Looks correct but its failing.

BoxEdit.download.recipe fails because it looks like Box changed the download URL.

Old URL: https://e3.boxcdn.net/box-installers/boxedit/mac/currentrelease/BoxEditInstaller.dmg
New URL: https://e3.boxcdn.net/box-installers/boxedit/mac/currentrelease/BoxToolsInstaller.dmg

Looks like the package referenced by CodeSignatureVerifier has moved.

$ autopkg run -v "CrashPlanPROeClient.download"
Processing CrashPlanPROeClient.download...
URLTextSearcher
URLTextSearcher: Found matching text (version): 4.4.1
URLTextSearcher: Found matching text (match): 4.4.1
URLDownloader
URLDownloader: Storing new Last-Modified header: Wed, 30 Sep 2015 20:56:22 GMT
URLDownloader: Downloaded /Users/elliot/Library/AutoPkg/Cache/com.github.hansen-m.download.CrashPlanPROeClient/downloads/CrashPlanPROe.dmg
EndOfCheckPhase
CodeSignatureVerifier
CodeSignatureVerifier: Mounted disk image /Users/elliot/Library/AutoPkg/Cache/com.github.hansen-m.download.CrashPlanPROeClient/downloads/CrashPlanPROe.dmg
CodeSignatureVerifier: Invalid input path: /Users/elliot/Library/AutoPkg/Cache/com.github.hansen-m.download.CrashPlanPROeClient/downloads/CrashPlanPROe.dmg/Install CrashPlanPROe.pkg
Invalid input path
Failed.
Receipt written to /Users/elliot/Library/AutoPkg/Cache/com.github.hansen-m.download.CrashPlanPROeClient/receipts/CrashPlanPROeClient-receipt-20151005-104640.plist

The following recipes failed:
    CrashPlanPROeClient.download
        Error in com.github.hansen-m.download.CrashPlanPROeClient: Processor: CodeSignatureVerifier: Error: Invalid input path

The following new items were downloaded:
    Download Path                                                                                                     
    -------------                                                                                                     
    /Users/elliot/Library/AutoPkg/Cache/com.github.hansen-m.download.CrashPlanPROeClient/downloads/CrashPlanPROe.dmg  

Unfortunately the download recipe fails with

Curl failure: The requested URL returned error: 404 Not Found (exit code 22)

It seems like you can no longer download JDK 9 from the Oracle Website and only JDK 10.

I am having an error that Pat G was having. He suggested to contact you. I have the engine installed and I am getting the MSI but I am failing on 7zip. I do have 7zip installed. Not sure what I am missing.

The following recipes failed:
/Users/ladmin/Library/AutoPkg/RecipeRepos/com.github.autopkg.patgmac-recipes/Google Chrome/GoogleChrome-Win64.LANrev.recipe
Error in com.github.patgmac.lanrev.GoogleChrome-Win64: Processor: com.github.hansen-m.SharedProcessors/WinInstallerExtractor: Error: Command '['7z', 'x', '-y', u'-o/Users/ladmin/Library/AutoPkg/Cache/com.github.patgmac.lanrev.GoogleChrome-Win64/InstallerExtract', u'/Users/ladmin/Library/AutoPkg/Cache/com.github.patgmac.lanrev.GoogleChrome-Win64/downloads/GoogleChrome x64.msi']' returned non-zero exit status 2

Let me know if anything comes to mind. I know 7zip but I just don't know what.

Thanks
-Jeff

Hello,

I'm receiving this error when attempting to use my Box Edit recipe which seems like the code signature may have changed.

Processor: CodeSignatureVerifier: Error: Invalid input path

More of an FYI, Inkscape now needs MacPorts to compile: https://inkscape.org/release/inkscape-0.92.4/mac-os-x/macports/dl/

It looks like they changed the format on the "whats-new" page.

sysadmin@beta: ~ 38$ autopkg run Unity3D.munki
Processing Unity3D.munki...
Unable to find latest version string in http://unity3d.com/unity/whats-new.
Failed.

The following recipes failed:
Unity3D.munki
Error in local.munki.Unity3D: Processor: Unity3DComponetsURLProvider: Error: Unable to find latest version string in http://unity3d.com/unity/whats-new.

Nothing downloaded, packaged or imported.

http://sites.box.com/sync4 seems to redirect to a login page.

Setting URL to https://community.box.com/t5/Using-Box-Sync/Installing-Box-Sync/ta-p/85 fixes the recipe for now, though I have my doubts about this being a stable URL. (One can just insert this URL in a recipe override.)

https://www.box.com/resources/downloads has no mentions of Box Sync, so I'm wondering if this is now or soon to be a deprecated product.

I noted today that the Rhinoceros.download recipe was raising an error:
Error in com.github.jps3.munki.Rhinoceros: Processor: URLTextSearcher: Error: No match found on URL: http://www.rhino3d.com/download/wenatchee/5.0/wip?current_page=download_new

I dug around a little bit, and I thought some of the following could save you some time updating the recipe file. Note I have not actually downloaded the resultant *.dmg and checked to see how or where this new license key code is used, as my Internet connection at home today is very slow and with frequent lags and dropouts. (Otherwise I would have been happy to go further with this.)

That URL now forwards to http://www.rhino3d.com/download/rhino-for-mac/5.0/wip which contains a form asking for an email address. The form appears to accept pretty much any input of the form [email protected] (even [email protected] worked). When the Next button is clicked, the next page displays a license key (of form XXXX-XXXX-XXXX-XXXX-XXXX-XXXX) and a link to the *.dmg file.

Using curl, I could post a request of the initial form:

curl -sLf -d 'email=x%40y.z&direction_next=Next+%3E&current_page=license_info' http://www.rhino3d.com/download/rhino-for-mac/5.0/wip

The returned HTML contains:

<div style='text-align:center'>
<p style="border:1px solid #ddd; background-color:#ffffcc; padding:10px;">Use the following Rhino For Mac 5.0 license key to install<br/><b><nobr>XXXX-XXXX-XXXX-XXXX-XXXX-XXXX</nobr></b></p><p>This license key has been emailed to <b>[email protected]</b></p>
<a class='btn download_link' style='display:inline-block;text-align:center;'  href="http://files.mcneel.com/Releases/Rhino/5.0/Mac/rhinoceros_wip_5A683.dmg">Download  Now</a>

• The download URL for the rhinoceros_wip_XXXXX.dmg file does not appear to need any special request headers, cookies, etc to work, which makes things much simpler.
• The request appears that it must be POST to get the download URL in the returned HTML

I've forked your repo here and have started to work on a RhinocerosUrlProvider but I am not finished yet. (Python’s urllib and etree libraries drive me crazy, I wish the requests and BeautifulSoup ones was standard!).

Alternatively, Since I have recipes based off of your Rhinoceros.download (*.pkg and *.munki), I can just make a new, updates *.download and place it in my repo. I do not wish to make any assumptions, step on toes, etc. It's been great leveraging off of the *.download recipe you'd made.

-Jason

We started seeing the following error a few days ago; I'm wondering if the 'terms of service' checkbox at https://store.unity.com/download is the problem? Manually loading https://store.unity.com/download/thank-you?thank-you=personal&os=osx&nid=1389 does result in a download.

Error in local.munki.Unity3D: Processor: Unity3DComponetsURLProvider: Error: No match found on URL: https://store.unity.com/download/thank-you?thank-you=personal&os=osx&nid=1389

Thanks!

I get this error related to SSL. (autopkg 0.5.0)

The following recipes failed:
RStudio.munki
    Error in local.munki.RStudio: Processor: URLDownloader: Error: Couldn't download https://download1.rstudio.org/RStudio-0.99.467.dmg: <urlopen error [Errno 1] _ssl.c:507: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>

Download URL changed to https://d.keka.io/

The currently used Mac download page references version 0.92.1 as the latest version, but:

There are currently no official .dmg files available for the Inkscape 0.92 series yet.

This appears out of date and that there's a new download page.

https://inkscape.org/en/release/ currently redirects to https://inkscape.org/en/release/0.92.2/ which can then take you to https://inkscape.org/en/release/0.92.2/mac-os-x/ by clicking the Mac link. Eventually getting you a DMG download.

Your SubEthaEdit recipe links to a download URL with a hard-coded version. I can see two potential issues with this:

• If version 3.5.5 is ever released, your URL won't grab it.
• Major version 4 is now available, and it's only in the App Store.

It might be time to retire the SubEthaEdit recipe, given the above.

The Inkscape recipe takes the latest version (0.92.3 at the moment), and attempts to download the latest DMG from https://inkscape.org/en/release/0.92.3/mac-os-x/. It fails because there are no downloads available there yet. Though https://inkscape.org/en/release/0.92.2/mac-os-x/ has DMG download available.

Running UbuntuMate 16.04. When using Ibus-Pinyin to input Chinese and English characters in texmaker , shift-key (_L or _R）doesn't shift the input mode. It works in all other applications. I can still use Ctrl-space (or Super-space) to toggle between English and Chinese keyboards. Same problem was reported in other versions of (at least) Ubuntu.

Here is the new URL for Amazon Music as it downloads a newer version:

https://www.amazon.com/gp/dmusic/desktop/downloadPlayer/

When I run the GoogleChrome-Win.download recipe, I get a "unknown processor BESRelevanceProvider'.

Known issue?

I was playing around with this processor and the Unity download recipe today in order to get the Unity components downloaded as part of autopkg, but my Python skills need work ... As it stands, there are a couple of issues I'm experiencing with the Unity3DComponetsURLProvider processor:

1. Can't provide anything other than Unity for the component to be downloaded. I'd really like to be able to specify this in the download recipe input, rather than have Unity be hardcoded.
2. If it's undesirable (or too much trouble) to update Unity3DComponetsURLProvider for this, then it'd be great to add the ability to add revision as an output variable. Looking at the code, it appears that revision is used, however this is not setup to be part of the output variables, but this is crucially needed in order to download.

If you go to https://unity3d.com/unity/whats-new/latest, this will redirect you to the latest available release version info page, which includes the direct URLs to the Unity Editor, as well as the various component installers.

However, the Unity3DComponetsURLProvider crucially collects the revision number, which is needed in the download URL. Below is what I've come up with for the Facebook Games Support installer if a revision output variable were added.

<dict>
    <key>Processor</key>
    <string>URLTextSearcher</string>
    <key>Arguments</key>
    <dict>
        <key>url</key>
        <string>https://unity3d.com/unity/whats-new/latest</string>
        <key>re_pattern</key>
        <string>(?P&lt;url&gt;https:\/\/download\.unity3d\.com\/download_unity\/%revision%\/MacEditorTargetInstaller\/UnitySetup-Facebook-Games-Support-for-Editor-(?P&lt;full_version&gt;[\w.]+)\.pkg)</string>
    </dict>
</dict>

Ideally, the re_pattern would be an input variable so that this one recipe could be used for different component packages, but if `%revision% is to be used in the variable as it is above, I don't think this is possible ...

The URL scheme for each component package is as follows:

https://download.unity3d.com/download_unity/<revision>/MacEditor[Installer|TargetInstaller]/[Unity|UnitySetup]-<Name of component>-<CFBundleVersion>.pkg

And while not huge, the processor itself is misspelled slightly.

Inkscape download recipe does not download latest version located at
https://inkscape.org/en/gallery/item/11269/Inkscape-0.92.2-1-x11-10.7-x86_64.dmg

Looks like Stellarium has moved from SourceForge to GitHub:

Now we're seeing errors in the recipe:
`The following recipes failed:

Stellarium.munki.recipe

    Error in local.munki.Stellarium: Processor: URLTextSearcher: Error: No match found on URL: https://sourceforge.net/projects/stellarium/rss?path=/Stellarium-MacOSX

`

All recipes I'm trying that utilize the BESRelevanceProvider shared processor are producing this error during autopkg runs:

com.github.hansen-m.SharedProcessors/BESRelevanceProvider
BESRelevanceProvider: QnA Error: (/usr/local/bin/QnA) -- list index out of range
BESRelevanceProvider: version = None

I've tested with Firefox-Win.download, GoogleChrome-Win.download, and AdobeFlashPlayerActiveX-Win.download. Launching QnA on its own kicks out this error:

$ /usr/local/bin/QnA
libc++abi.dylib: terminating with uncaught exception of type MacintoshPlatform::CannotCreateSpecialFolder: 
Abort trap: 6

Any advice?

It would be interesting to add etag data to the generated BigFix task in the metadata, perhaps in a MIME field, and/or an action script comment. This could be used by autopkg or other automation of BigFix content for de-duplication and other purposes.

I could see this as useful metadata for all results of autopkg recipes to carry through.

This would allow the thing doing the automation to be more stateless other than the generated results that are already in the distribution platform anyway.

run -v Fitbit.download
Processing Fitbit.download...
URLTextSearcher
No match found on URL: http://www.fitbit.com/start?platform=mac
Failed.
Receipt written to /Users/USERNAME/Library/AutoPkg/Cache/com.github.hansen-m.download.Fitbit/receipts/Fitbit-receipt-20160902-080940.plist

The following recipes failed:
    Fitbit.download
        Error in com.github.hansen-m.download.Fitbit: Processor: URLTextSearcher: Error: No match found on URL: http://www.fitbit.com/start?platform=mac

Nothing downloaded, packaged or imported.

The regex expression still looks fine, though.

I hope you don't mind, but I used your GimpOnOSX.download.recipe as a parent recipe for my .munki version of it. I haven't gone into full testing, but I wanted to check with you before completely finalizing it on my recipe-repo. Is that alright with you?

I'm getting a URL 404 error with the RStudio download recipe; not sure if changes to the mirror broke your search regex?

Texmaker appears to have new version with new link.
Current download link points to http://www.xm1math.net/texmaker/assets/files/texmaker-5.0.2.dmg

Texmakerd.download AutoPkg recipe downloads version 4.5 at http://www.xm1math.net/texmaker/TexmakerMacosxLion.zip

after running autopkg install QnA and then trying to run QnA, I got the following error:

BESClientActionMastheadPath not set and no masthead found
Missing library: Dynamic library failed to load (/Users/james/QnA/libBEScrypto.dylib): dlopen(/Users/james/QnA/libBEScrypto.dylib, 6): image not found

I am guessing that this is due to not having BigFix installed on the machine.

• https://github.com/autopkg/hansen-m-recipes/blob/master/IBM/QnA.install.recipe

The relevance for the BES relevance processor in both Google Chrome download recipes the year needs to be updated from 2011 to 2018 so the version can be obtained. Looks like Google changed the copyright year in the latest release.

Lately, I've been getting an error whenever running your Win recipes from within AutoPkgR.

Errno 2 No such file or directory.

Mar 14 09:47:14 col-pgalla2-11v AutoPkgr[91214]: Run status: Processing com.github.hansen-m.download.FirefoxESR-Win... Mar 14 09:47:14 col-pgalla2-11v AutoPkgr[91214]: Processing com.github.hansen-m.download.FirefoxESR-Win... Mar 14 09:47:15 col-pgalla2-11v AutoPkgr[91214]: Error [1] Error running recipes Mar 14 09:47:17 col-pgalla2-11v AutoPkgr[91214]: [DEBUG] Completed AutoPkg Task: /usr/bin/python /usr/local/bin/autopkg run com.github.hansen-m.download.FirefoxESR-Win --report-plist /var/folders/y2/zc31smvd50j8jtwqg14lj8kjrbxhc7/T/com.lindegroup.AutoPkgr/20160314094714 -v

The same recipes run fine from the command line. I'm assuming this a a path issue somewhere?
I did update p7zip via brew and 7z is in /usr/local/bin.

The latest version of MeshLab (2016.12) is not being downloaded by the recipe. It looks like they have moved from SourceForge to GitHub. Basically, the recipe needs to be rewritten from scratch. I can do that work it and post to my own repo (since I'll also be writing a pkg recipe as well) or I can put in a pull request. Take your pick.

Hi
Inkscape.download.recipe downloads Inkscape version 0.48.5 from 28 July 2014
Current available is 0.91 from February 2015

I am getting the following error:

The following recipes failed:
RStudio.munki
Error in local.munki.RStudio: Processor: URLDownloader: Error: Curl failure: Could not resolve host: %DOWNLOAD_MIRROR% (exit code 6)

I am calling this through the recipe in joshua-d-millers RStudio.munki.recipe. Am I missing something?

iTunes-Win.download isn't finding the latest version. It seems the URL the recipe uses isn't being updated anymore. https://www.apple.com/itunes/download/ seems to include the correct URL to download the exe, but autopkg can't find the URL when it curls it. I can find the url in the curl, but can't find any reason the regex isn't working.

$ pkgutil --check-signature /Volumes/ADE\ 4.5/Digital\ Editions\ 4.5\ Installer.pkg
Package "Digital Editions 4.5 Installer.pkg":
   Status: no signature

Possibly an oversight? I've tweeted @adobe but no idea if that will have any effect. If anyone has a better support route, please can they try and let them know?

Looks like when the Google Chrome msi is extracted, the "[5] Summary Information" does not contain the string looked for. It might be an issue with the extraction or the file itself as when extracted with 7zip as the character set seems off when viewed in a text editor.

The Adobe Flash download links for URLTextSearcher appear to be not working.

I recommend deleting the Picasa recipe from your repo and adding a Readme that mentions Picasa is no longer being developed.

https://picasa.google.com/

Seems like they changed something on the website.

autopkg run -v com.github.hansen-m.download.mega
Processing com.github.hansen-m.download.mega...
URLTextSearcher
No match found on URL: http://www.megasoftware.net/releases/?C=M;O=D
Failed.
Receipt written to /Users/userxy/Library/AutoPkg/Cache/com.github.hansen-m.download.mega/receipts/com.github.hansen-m.download-receipt-20161121-122329.plist

The following recipes failed:
    com.github.hansen-m.download.mega
        Error in com.github.hansen-m.download.mega: Processor: URLTextSearcher: Error: No match found on URL: http://www.megasoftware.net/releases/?C=M;O=D

Nothing downloaded, packaged or imported.

Looks like the download is now behind a EULA, so the recipe fails

I have been trying to work your Windows recipes on a new Mac install of AutoPkg and I am running into a stone wall. I have wine and 7zip installed via MacPorts but I am still seeing the following errors. Any assistance would be appreciated.

AutoPkgr:~ ladmin$ autopkg run /Users/ladmin/Library/AutoPkg/RecipeRepos/com.github.autopkg.hansen-m-recipes/Google/GoogleChrome-Win64.download.recipe
Traceback (most recent call last):
File "/usr/local/bin/autopkg", line 36, in
import autopkglib.github
File "/Library/AutoPkg/autopkglib/init.py", line 627, in
import_processors()
File "/Library/AutoPkg/autopkglib/init.py", line 530, in import_processors
mydirname + '.' + name, fromlist=[name]), name)
File "/Library/AutoPkg/autopkglib/AutoPkgBESEngine.py", line 23, in
import requests

Is it possible to get the version from the Firefox-Win and FirefoxESR-Win download recipes?

I suspect it has to do with the 301 permanent redirect. The recipe now gives this error:

Processing Kindle.download...

URLTextSearcher

No match found on URL: http://www.amazon.com/gp/kindle/mac/download

Failed.

Receipt written to /Users/admin/Library/AutoPkg/Cache/com.github.hansen-m.download.Kindle/receipts/Kindle-receipt-20150729-095146.plist

The following recipes failed:

Kindle.download

    Error in com.github.hansen-m.download.Kindle: Processor: URLTextSearcher: Error: No match found on URL: http://www.amazon.com/gp/kindle/mac/download

Nothing downloaded, packaged or imported.