Giter Club home page Giter Club logo

fs's Introduction

babashka.fs

Clojars Project bb built-in

File system utilities. This library can be used on the JVM and is also included in babashka (>= 0.2.9).

Why

Babashka is a scripting utility. It's convenient to have cross platform file system utilities available for scripting. The namespace clojure.java.io already offers a bunch of useful features, but it predates java.nio. The nio package isn't that nice to use from Clojure and this library should help with that.

The main inspirations for this library are clojure.java.io, clj-commons/fs and corasaurus-hex/fs.

API docs

See API.md.

Usage

(require '[babashka.fs :as fs])
(fs/directory? ".") ;;=> true

Examples

The glob function takes a root path and a pattern. The pattern is interpreted as documented here.

(map str (fs/glob "." "**{.clj,cljc}"))

Output:

("project.clj" "test/babashka/fs_test.clj" "src/babashka/fs.cljc")

The function exec-paths returns all entries from PATH as Paths. To search all these directories for an executable, e.g. java, you can combine it with list-dirs which searches files directly in the directories using an (optional) glob pattern:

(str (first (filter fs/executable? (fs/list-dirs (filter fs/exists? (fs/exec-paths)) "java"))))
"/Users/borkdude/.jenv/versions/11.0/bin/java"

For convenience, the above use case is also supported using the which function:

(str (fs/which "java"))
"/Users/borkdude/.jenv/versions/11.0/bin/java"

Test

$ bb test

License

Copyright Β© 2020-2021 Michiel Borkent

Distributed under the EPL License, same as Clojure. See LICENSE.

fs's People

Contributors

benjamin-asdf avatar bobisageek avatar borkdude avatar corasaurus-hex avatar duzunov avatar eamonnsullivan avatar eval avatar holyjak avatar hugoduncan avatar ikappaki avatar jjttjj avatar jlesquembre avatar kineolyan avatar kiramclean avatar lread avatar pavlosmelissinos avatar retrogradeorbit avatar sohalt avatar tekacs avatar teodorlu avatar thenonameguy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

fs's Issues

glob: not recursive by default?

It seems a lot of environments do not have recursive globbing by default. We should revisit this.

borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ ls **/*.md
CHANGELOG.md                                   babashka.nrepl/CHANGELOG.md                    depstar/README.md                              doc/news.md                                    pods/examples/pod-lispyclouds-sqlite/README.md
CONTRIBUTING.md                                babashka.nrepl/README.md                       doc/build.md                                   doc/projects.md                                process/README.md
README.md                                      babashka.nrepl/doc/intro.md                    doc/dev.md                                     doc/surveys/2020-11.md                         sci/CHANGELOG.md
babashka.curl/CHANGES.md                       deps.clj/README.md                             doc/examples.md                                examples/README.md                             sci/README.md
babashka.curl/README.md                        depstar/CHANGELOG.md                           doc/libraries.md                               pods/README.md                                 sci/doc/libsci.md
borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ bash
bash-3.2$ ls **/*.md
babashka.curl/CHANGES.md	babashka.nrepl/README.md	depstar/README.md		doc/examples.md			doc/projects.md			process/README.md
babashka.curl/README.md		deps.clj/README.md		doc/build.md			doc/libraries.md		examples/README.md		sci/CHANGELOG.md
babashka.nrepl/CHANGELOG.md	depstar/CHANGELOG.md		doc/dev.md			doc/news.md			pods/README.md			sci/README.md

read-all-lines fails when encoding not UTF-8

The function read-all-lines fails when the encoding is not UTF-8

With the file attached, iso-8859.txt, the function produces this error:

(fs/read-all-lines (fs/file "iso-8859.txt"))
: Input length = 1 user

If you enclose in a try-catch, the exception is java.nio.charset.MalformedInputException

(tested with babashka v0.8.2)

Ideally, the function should provide an encoding optional param to deal with these cases.

Allow for more declarative use of `xdg-*-home`

(fs/xdg-config-home) is mostly never useful as-is as it's typically used to get the config-folder for a specific application: (fs/path (fs/xdg-config-home) "some-app").

How about we add a 1-arity option for the fs/xdg-*-home functions?

(fs/xdg-config-home "clj-kondo")
;; => #object[sun.nio.fs.UnixPath 0x37548587 "/Users/gert/.config/clj-kondo"]`

`which` searches $PATH for relative paths (i.e. different from *nix which command)

Probably a low priority, as it's a somewhat esoteric situation.
I'm reporting this here because in my opinion fs/which is sort of the root cause, but there are several layers here, and maybe a couple of options as to where to fix (if we agree it's an issue).

presenting symptom/problem statement
When running native babashka tests in my Windows environment, I found that the tests end up using my installed babashka (on my PATH) rather than the freshly-built babashka in the current directory. I believe this has been the case since a fix for babashka/babashka#1078 went in. The native tests use babashka.process and execute "./bb", but despite the explicit relative path, $PATH is still being searched.

The desired outcome (from my standpoint) would be that calling process with a relative path (not just a command/file name) would respect the relative path, to more closely mimic the behavior of the command line. With that said, the Windows PATHEXT problem (needing the tack on the exe/com/cmd extension) would still exist in that case, so maybe there's a case for separating determining candidate directory paths and checking the candidate extensions.

babashka.process
On Windows, p/process searches $PATH if program is a relative path:
https://github.com/babashka/process/blob/8d72f2097a1dc49b81a52af6733b8beb836efbdc/src/babashka/process.cljc#L167-L168

Given a situation where the current directory contains an executable with the same name as an executable on $PATH, calling process on ./executable-name will use the one from $PATH instead of the one from the current directory:

C:\Users\user\code\clojure\babashka>.\bb --version
babashka v0.7.9-SNAPSHOT

C:\Users\user\code\clojure\babashka>bb
user=> (require '[babashka.process :as p])
user=> (-> (p/process ["./bb" "--version"] {:out :string})
         deref
         (select-keys [:out :cmd]))
{:out "babashka v0.7.8\r\n", :cmd ["C:\\Users\\user\\scoop\\shims\\.\\bb.exe" "--version"]}

... note that the process invocation uses the bb executable from $PATH. In my non-windows environment, both the shell command and the process invocation use the bb from the current directory.

fs/which
fs/which always searches $PATH, whereas the which unix util appears to only search $PATH if the argument is exactly a file name (i.e. not a path that specifies any directories)

# /home/bob/code is on my $PATH
$ echo $PATH | awk -F: '{print $NF}'
/home/bob/code

# /home/bob/code/bin/bb exists
$ ls /home/bob/code/bin/bb
/home/bob/code/bin/bb

# which doesn't find 'bin/bb' on $PATH from an arbitrary directory
$ pwd
/home/bob/code/clojure
$ which bin/bb
$

# but which does find 'bin/bb' if it's relative to the current directory
$ pwd
/home/bob/code
$ which bin/bb
bin/bb

but fs/which always looks through $PATH:

$ pwd
/home/bob/code/clojure
# this which fails at the command line
$ bb "(require '[babashka.fs :as fs]) (fs/which \"bin/bb\")"
#object[sun.nio.fs.UnixPath 0x1dff8382 "/home/bob/code/bin/bb"]

Possible resolution
If the aim would be to make fs/which behave more like unix which (while still accounting for Windows extensions), then I imagine it might work something like this:

; figure out candidate paths
(let [prog-path (path program)]
   (if (= prog-path (.getFileName prog-path)) ; if the whole path is just the file name, search $PATH, otherwise treat path as relative
       (exec-paths)
       ["."]))

... and then the existing search (with windows extensions on Windows) through the newly-determined candidate paths. One possible problem with the .getFileName approach is that it would also see a directory name as just a file name, but that seems to also be true currently.

That's just an idea - I figure logic that's getting complicated like this might warrant some design discussion before acting, so I just wanted to lay out the problem statement and get some feedback.

Clarify posix-file-permissions option

heow Today at 3:42 PM
anyone know the permissions format for (fs/create-dir "foo") ? The docs just reference Files which is a bit opaque

3 replies
borkdude:clj-kondo: 3 minutes ago
The docs could be improved here. In another function I see:
The :posix-file-permissions option is a string like "rwx------"
πŸ‘
1

borkdude:clj-kondo: 1 minute ago
Please post an issue about this.
borkdude:clj-kondo: < 1 minute ago
Or I will do this myself now, actually ;).

Consider adding cljdoc config

Because it has cljc files, cljdoc assumes that babashka/fs is a Clojure and ClojureScript project. Explicitly telling cljdoc that this is a Clojure only project will allow docs to build successfully on cljdoc.

I you are ok with adding this config, I am very happy to follow up with a PR.

Add common XDG-paths

The XDG spec defines various base paths to look for config/data/cache files.
Having written some scripts with this logic, I thought it might be convenient to have this in babashka/fs.

Looking at what paths are commonly used, it seems adding the following fns would suffice:

  • xdg-config-home
  • xdg-cache-home
  • xdg-data-home
  • xdg-state-home

exists? should never throw

On Windows:

user=> (fs/exists? "<")
Execution error (InvalidPathException) at sun.nio.fs.WindowsPathParser/normalize (WindowsPathParser.java:182).
Illegal char <<> at index 0: <

Handle google storage uri strings without need for additional wrapping

babashka.fs works well with google storage paths with a little bit of additional treatment

(require '[babashka.fs])
(import [java.nio.file Paths] [java.net URI] [java.nio.file.spi FileSystemProvider])

;; quickly check that `gs` is registered correctly
(->> (FileSystemProvider/installedProviders)
     (map (fn [p] (.getScheme p)))
     (into #{})
     println)
;; => #{"jrt" "gs" "jar" "file"}

(def uri-str "gs://bucket/filename")
(def uri (URI/create uri-str))
(def path (Paths/get uri))
(babashka.fs/exists? path) ;=> true

but I'm wondering if it would be possible to have the following two cases also work, where the argument is a URI or a string representing a uri:

(babashka.fs/exists? uri) ;=> currently fails with "unknown protocol: gs"
(babashka.fs/exists? uri-str) ;=> currently results in `false`

we spoke offline and you tracked the issue down to

user=> (babashka.fs/path uri)
Execution error (MalformedURLException) at java.net.URL/<init> (URL.java:652).
unknown protocol: gs
;; then
user=> (clojure.java.io/as-url uri)
Execution error (MalformedURLException) at java.net.URL/<init> (URL.java:652).
unknown protocol: gs
;; and lastly
user=> (.toURL uri)
Execution error (MalformedURLException) at java.net.URL/<init> (URL.java:652).
unknown protocol: gs

your work in 47f5642 helps the following work

(babashka.fs/exists? uri) ;=> true

but this still fails:

(babashka.fs/exists? uri-str) ;=> false

copy-tree doesn't create nested destination directories

As discussed here: https://clojurians.slack.com/archives/CLX41ASCS/p1641396190148600
Tested with bb v0.7.0 and v0.7.3

Repro steps:

➜  ~ mkdir foo
➜  ~ touch foo/bar1
➜  ~ touch foo/bar2
➜  ~ bb -e '(fs/copy-tree "foo/" "foo2/foo")'
----- Error --------------------------------------------------------------------
Type:     java.nio.file.NoSuchFileException
Message:  /Users/dimitaruzunov/foo2/foo
Location: <expr>:1:1

Expected result: to create the foo2 directory

Or to change the docstring:
Copies entire file tree from src to dest. Creates dest if needed using create-dirs, passing it the :posix-file-permissions option. Supports same options as copy.
https://babashka.org/fs/babashka.fs.html#var-copy-tree

Explicitly support `:win-exts` on `which` function

Currently

@borkdude has a comment on :win-exts being unsupported for the which function:

fs/src/babashka/fs.cljc

Lines 780 to 787 in 5a63586

(defn which
"Locates a program in (exec-paths) similar to the which Unix command.
On Windows it tries to resolve in the order of: .com, .exe, .bat,
.cmd."
([program] (which program nil))
([program opts]
;; :win-exts is unsupported, if you read and use
;; this, let me know, it may break.

But...

I actually made use of :win-exts on Etaoin to help me locate .ps1 files.

So...

As requested in the comment, I reached out to borkdude on Slack.
To which he responded:

I'm fine with removing the comment and support :win-exts

Next Action

I'll follow up with a PR.
Only comment and docstring should change.

Feature request: Extension utils

This seems to most up-to-date JVM Clojure filesystem library, but I noticed there are couple of differences from some of the other ones. I was wondering what you think of adding extension utils to this library, like split-ext or ext, like ones from e.g. the clj-commons lib.

They're really simple so maybe it doesn't make sense to include here. Or maybe you want to stick to mirroring the java file APIs. Anyway, I can see why they might not belong here, but if this is something you'd consider I'd be happy to help implement it.

Thanks for yet another useful library!

move-tree

Do we need this? When moving a file on the same partition, moving an entire dir should already work.
Workaround: copy-tree + delete-tree on the original.

Get the filename without an extension/suffix

Linux includes a handy function called basename which lets you get a file name from a path and optionally remove a suffix at the same time. Is this something babashka.fs should include?

I have already opened #28 for this but it should be considered a WIP when/if it's decided what should be included.

Some possible options:

  1. Include it as part of the file-name function just as was done in raynes.fs.
  2. Add a new function for doing just the removal part as the PR does.
  3. Add a function that strips the extension or a suffix from a path and returns the entire path, not just the file-name.
  4. Not include this at all?

Looking for input and thoughts from the community, thanks!!

split-ext/extension fail on files with trailing space on windows

It seems that this works on linux but on windows fails.

(spit "test .txt" "hi")
(fs/extension "test .txt")

The exception thrown is

InvalidPathException Trailing char < > at index 4: test 
	sun.nio.fs.WindowsPathParser.normalize (WindowsPathParser.java:191)
	sun.nio.fs.WindowsPathParser.parse (WindowsPathParser.java:153)
	sun.nio.fs.WindowsPathParser.parse (WindowsPathParser.java:77)
	sun.nio.fs.WindowsPath.parse (WindowsPath.java:92)
	sun.nio.fs.WindowsFileSystem.getPath (WindowsFileSystem.java:232)
	java.io.File.toPath (File.java:2396)
	babashka.fs/as-path (fs.cljc:39)
	babashka.fs/as-path (fs.cljc:34)
	babashka.fs/path (fs.cljc:53)
	babashka.fs/path (fs.cljc:47)
	babashka.fs/split-ext (fs.cljc:728)
	babashka.fs/split-ext (fs.cljc:720)

This is due to the call to path on something with a trailing space, ie "test ", here, this is an invalid path in windows, so it makes sense that calling path on that should throw on windows. "test .txt" as a path is valid on windows or at least it works with the rest of api.

I'm not quite sure what the correct behavior here for extension should be though. Figured I would report it and see what you think.

.zip functionality

Hi,

I wonder if you've considered adding .zip/.jar functionality to this lib. I would be able to contribute PRs.

Some needs/features that come to mind:

  • zip-seq, akin to file-seq
  • uncompressing a file into the filesystem
  • uncompressing a file into memory

WDYT?

Cheers - V

`expand-home` is wrong on windows when passed a string with a forward slash

(fs/expand-home "~/my-file") ;; "C:\\Users\\my-file" incorrect, missing username directory when using forward slash in the argument
(fs/expand-home "~\\my-file") ;; "C:\\Users\\justin\\my-file" correct
(fs/expand-home (fs/path "~/my-file")) ;; "C:\\Users\\justin\\my-file" correct

This is because the current expand-home implementation just looks for the index of the path separator in the input. Calling as-path on the input within expand-home should fix this.

I will provide a PR for this today or tomorrow

issue with glob matching at the last part of path

I've run into a problem when trying to use fs/glob to match a path which matches the end of the pattern. For instance
(fs/glob "/Users/derek/tmp/file1" "file1") => [] or
(fs/glob "/Users/derek/tmp/file1" "**file1") => [] both return nothing

but pointing to its parent dir works
(fs/glob "/Users/derek/tmp" "file1") => [#object[sun.nio.fs.UnixPath 0x62a73d2b "/Users/derek/tmp/file1"]]

The above is counter-intuitive to me. Also I can match in this scenario using the standard Java FileSystems matcher and get the expected behavior:
(def jfs (FileSystems/getDefault))
(def match3 (.getPathMatcher jfs "glob:**file1"))
(def f1 (io/file "/Users/derek/tmp/file1"))
(.matches match3 (fs/canonicalize f1)) => true

Maybe I'm misunderstanding something about how you intended glob to be used, but this sounds like a bug to me.

Thanks for taking a look at this

Derek

Add test case for attributes

 (-> (fs/set-attribute "/tmp/foo.clj" "basic:lastModifiedTime" (fs/millis->file-time 100)) (fs/read-attributes "*") :lastModifiedTime fs/file-time->millis) ;;=> 100

Wrap fs/create-dir with fs/exists?

Hey, I really like the babshka.fs namespace, thanks for creating.

In my scripts I tend to wrap create-dir like this

(defn create-dir [path]
  (when-not (fs/exists? path)
    (fs/create-dir path)))

Just wanted to check if you think this would be a useful addition to the babshka.fs namespace?

expand-home is not working as expected on Windows

Small example:

(require '[babashka.fs :as fs])
(fs/expand-home "~/hello-world.bat")

prints

#object[sun.nio.fs.WindowsPath 0x42897f2f "C:\\Users\\hello-world.bat"]

But in my case the string part should be like:

C:\\Users\\alesn\\hello-world.bat

This means that actual user directory is omitted from the expansion.

Add dir-name

dirname(1) is a common command in shell scripts. It would be convenient to have its direct equivalent in babashka.fs. It would be also nice to have something that functioned like corasaurus_hex.fs/split-path. I don't think they are redundant, since a dirname function would handle a path with no parent directory specially (returning "."). Joker, for example, has joker.filepath/dir as well as split and split-list.

clarify docs for create-dir and create-dirs

javahippie For create-dir and create-dirs, the docs just reference the Files docs, maybe it’s helpful to point out, what the difference there is directly in the docs? In some API, this behavior is controlled with a flag, so it might not be clear to some users (edited)

<1m
borkdude πŸ‘ I'll do that

create-temp-file

I find myself needing temp-file just to create a temporary file somewhere.

Consider allowing non-relative paths for fs/which

Background

On Windows babashka.fs/which will automatically find foo.exe (and other :win-exts) for foo.

This is very helpful. We often don't really care about the Windows executable extension.

Problem

which currently throws on non-relative paths.

For example, let's say I want to find the executable for java on Windows.
Instead of finding the java executable on the PATH, I want to use JAVA_HOME.

From a Windows 10 in PowerShell:

PS C:\Users\lee> clj --version
Clojure CLI version (deps.clj) 1.11.1.1252
PS C:\Users\lee> clj -Sdeps "{:deps {babashka/fs {:mvn/version \`"0.3.17\`"}}}"
Clojure 1.11.1
user=> (require '[babashka.fs :as fs])
nil

Finding java on the path works great:

user=> (fs/which "java")
#object[sun.nio.fs.WindowsPath 0x3e3861d7 "C:\\Users\\lee\\scoop\\apps\\graalvm22-jdk17\\current\\bin\\java.exe"]

But maybe we want to find it via JAVA_HOME.
First, let's show that java.exe exists through JAVA_HOME:

user=> (fs/exists? (fs/file (System/getenv "JAVA_HOME") "bin" "java.exe"))
true

Ok, now let's try to find java.exe through JAVA_HOME using which:

user=> (fs/which (fs/file (System/getenv "JAVA_HOME") "bin" "java"))
Execution error (IllegalArgumentException) at babashka.fs/path (fs.cljc:57).
C:\Users\lee\scoop\apps\graalvm22-jdk17\current\bin\java.com is not a relative path

This does not work because which does not currently work on non-relative paths.

But if I contrive a relative path I can get which to find the exe:

user=> (System/getProperty "user.dir")
"C:\\Users\\lee"
user=> (fs/which (fs/file (fs/relativize (System/getProperty "user.dir") (System/getenv "JAVA_HOME")) "bin" "java"))
#object[sun.nio.fs.WindowsPath 0x61af1510 "scoop\\apps\\graalvm22-jdk17\\current\\bin\\java.exe"]

Proposal

On Windows, which could resolve the executable extension even for non-relative paths.

Additional Idea

Perhaps which should allow non-relative paths in general?
If the passed in program already resolves to an executable, which could return it.

Given a foobar.bat in the current directory, we see this behaviour when looking up programs that specify a relative path:

user=> (fs/which "./foobar")
#object[sun.nio.fs.WindowsPath 0x11b455e5 ".\\foobar.bat"]
user=> (fs/which "./foobar.bat")
#object[sun.nio.fs.WindowsPath 0x6d025d1d ".\\foobar.bat"]

(Tangent: I wonder though, should we be returning the absolute or canonical path here?)

Operating on absolute paths seems to match the which zsh/bash behaviour on macOS/Linux:

$ which java
/home/lee/.sdkman/candidates/java/current/bin/java
❯ which /home/lee/.sdkman/candidates/java/current/bin/java
/home/lee/.sdkman/candidates/java/current/bin/java

And the get-command behaviour on Windows PowerShell

PS C:\Users\lee> get-command java

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     java.exe                                           17.0.7.0   C:\Users\lee\scoop\apps\graalvm22-jdk1...


PS C:\Users\lee> get-command C:\\Users\\lee\\scoop\\apps\\graalvm22-jdk17\\current\\bin\\java.exe

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     java.exe                                           17.0.7.0   C:\Users\lee\scoop\apps\graalvm22-jdk17\current\bin\java.exe

And I guess kinda sorta the where behaviour Windows CMD shell:

C:\Users\lee>where java
C:\Users\lee\scoop\apps\graalvm22-jdk17\current\bin\java.exe
C:\Users\lee\scoop\apps\temurin19-jdk\current\bin\java.exe
C:\Users\lee\scoop\apps\temurin8-jdk\current\bin\java.exe
C:\Users\lee\scoop\apps\temurin11-jdk\current\bin\java.exe

C:\Users\lee>where "C:\Users\lee\scoop\apps\graalvm22-jdk17\current\bin:java.exe"
C:\Users\lee\scoop\apps\graalvm22-jdk17\current\bin\java.exe

Next Steps

Lemme know if I've missed some alternatives/nuances,etc that you see.

If this idea makes sense to you, happy to help out with a PR.

Add fs/zip

We can look at the tools build implementation for this as an example.

Async-capabilities for fs/glob and fs/match

It would be useful if the library provided async-capabilities for finding files, such as a callback-based API for fs/glob and fs/match. That way we could traverse large file-trees in the background without blocking the main thread.

Ideas for more functions

I came across f.el and some of these functions look really useful for when you're messing around with the filesystem.

`babashka.fs/glob` crashes when encountering permission errors

On my Mac, I get the following behavior:

user=> (babashka.fs/glob "/private" "**/deps.edn")
java.lang.Exception: Visiting /private/etc/cups/certs failed [at <repl>:6:1]

I use babashka.fs/glob to find all deps.edn files in a subfolder in neil-quickadd. In my case, I'd prefer ignoring files and folders with permission errors.

https://github.com/teodorlu/neil-quickadd/blob/7bb1629670966d4e40adf458bc157f08eb2dc8f1/src/teodorlu/neil_quickadd.clj#L31

The glob folder traversal logic appears to be in babashka.fs/match:

https://github.com/babashka/fs/blob/master/src/babashka/fs.cljc#L250-L314

expand-home

Should we add an expand-home function that expands a leading ~ into (System/getProperty "user.home")?

Questions:

  • On Windows this doesn't mean anything. Should we also read ~/foo/bar as C:\Users\borkdude\foo\bar on Windows
  • Do we need to expand more than tilde?
  • Do we need to expand tilde is if's not the leading character?

support for google storage file attribute metadata

Hello, I'm trying to use google cloud storage's nio interface with babashka.fs. While a lot of it is just working (:tada:), I'm having issues accessing file metadata attributes.

The direct java invocation would look like:

(Files/readAttributes 
 (Paths/get (URI/create "gs://bucket/filename"))
 com.google.cloud.storage.contrib.nio.CloudStorageFileAttributes 
 (into-array [LinkOption/NOFOLLOW_LINKS]))

but when I try to use babashka.fs/read-attributes as follows I encounter two failures

(babashka.fs/read-attributes 
 (Paths/get (URI/create "gs://bucket/filename")) 
 CloudStorageFileAttributes 
 {:nofollow-links true})

failure A

; eval (current-form): (babashka.fs/read-attributes path CloudStorageFileAttributes {:nofollow-links true})
; (err) Execution error (ClassCastException) at babashka.fs/read-attributes (fs.cljc:596).
; (err) class java.lang.Class cannot be cast to class java.lang.String (java.lang.Class and java.lang.String are in module java.base of loader 'bootstrap')

failure B

#error {
 :cause "Don't know how to create ISeq from: com.google.cloud.storage.contrib.nio.CloudStorageObjectAttributes"
 :via
 [{:type java.lang.IllegalArgumentException
   :message "Don't know how to create ISeq from: com.google.cloud.storage.contrib.nio.CloudStorageObjectAttributes"
   :at [clojure.lang.RT seqFrom "RT.java" 557]}]
 :trace
 [[clojure.lang.RT seqFrom "RT.java" 557]
  [clojure.lang.RT seq "RT.java" 537]
  [clojure.core$seq__5467 invokeStatic "core.clj" 139]
  [clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 24]
  [clojure.core.protocols$fn__8226 invokeStatic "protocols.clj" 75]
  [clojure.core.protocols$fn__8226 invoke "protocols.clj" 75]
  [clojure.core.protocols$fn__8178$G__8173__8191 invoke "protocols.clj" 13]
  [clojure.core$reduce invokeStatic "core.clj" 6886]
  [clojure.core$into invokeStatic "core.clj" 6958]
  [clojure.core$into invoke "core.clj" 6950]
  [babashka.fs$read_attributes invokeStatic "NO_SOURCE_FILE" 585]
  [babashka.fs$read_attributes invoke "NO_SOURCE_FILE" 577]
...
}

For quick context, I marked the failure points:

(ns babashka.fs)

(defn read-attributes
  "Reads attributes via Files/readAttributes. Keywordizes string keys into keywords. This can be changed by passing a :key-fn in the options map."
  ([path attributes]
   (read-attributes path attributes nil))
  ([path ^String attributes {:keys [:nofollow-links :key-fn]}] ;; failure A due to ^String annotation
   (->>  (Files/readAttributes (as-path path)
                               attributes
                               (->link-opts {:nofollow-links nofollow-links}))
         (into {}) ;; failure B
         (keyize (or key-fn keyword)))))

Thanks!

support `java.nio.file.Files/write`

I've been wanting to use babashka.fs for writing files using nio

Might supporting those be a sensible addition? Would also require an implementation ->standard-open-option analogous to ->copy-opts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.