Comments (9)
Haha nice to see you over here. Small world.
from arquero.
It's handy for libraries that share constructed objects between each other. Example:
LibA {"dependencies": "arrow@^14"}
LibB {"dependencies": "arrow@^15"}
App depends on LibA and LibB
npm install: no warnings etc
result in:
node_modules/
LibA
node_modules/[email protected]
LibB
node_modules/[email protected]
Becomes an issue when in the app:
import {createArrow} from 'LibA';
import {useArrow} from 'LibB';
const myArrow = createArrow(); // constructed with v14 internally
useArrow(myArrow); // the object will be operated on by a library leveraging v15 internally
Versus with peerDependencies:
LibA {"peerDependencies": "arrow@^14"}
LibB {"peerDependencies": "arrow@^15"}
App depends on LibA and LibB
npm install: gives an error saying that peer dependencies cannot be satisfied among my used libraries
The app must either:
- Ask nicely that the libs update their support arrow range (which hopefully comes with testing from the libraries that the new major version doesn't break existing behavior)
- Manually define overrides in the App package.json to assume the responsibility of forcing the libs to use a shared version of arrow 14 or 15 or manually specify that they each use a separate version.
from arquero.
This breaks compatibilty with arrow 15 (using table / toIPC breaks)
from arquero.
Which methods specifically? Arrow 14 and 15 use the same IPC format and the APIs are very similar.
from arquero.
I see what you mean. The binary format somehow changes: #347.
from arquero.
I sent a pull request to change the default nullable for typed arrays back to not nullable (and also fixed a bug in null count): apache/arrow#40852. I think we could either update to arrow 15 now with my fix to the tests and then later revert, or wait for arrow 16. Either way, arquero should work either version.
from arquero.
I could make a separate issue for it, but it would also be helpful if arrow was a peer dependency instead of a direct dependency (helps make sure an app only ends up with a single copy of arrow shared across dependencies).
"peerDependencies": {
"apache-arrow": "^15.0.0"
},
If arrow is good at backwards compatibility (although maybe the issues you ran into are reason enough to not assume that and need manual update across major versions) could consider a broader range specifier.
"peerDependencies": {
"apache-arrow": ">=15.0.0"
},
That would be similar to the pattern of libraries like @geoarrow/geoarrow-js
.
from arquero.
The arrow ipc is backwards compatible. The arrow library is best effort backwards compatible. But yes, that's a separate issue for arquero. Why do we need a peer dependency? Couldn't we specify a broader range for the dependency so that npm can dedupe?
from arquero.
Couldn't we specify a broader range for the dependency so that npm can dedupe?
Critical difference is, is it okay if npm sometimes doesn't dedupe or is it potentially an issue if multiple copies of the library exist in the app at different versions. For arrow as a dependency I think it's safer for end users to indicate that an app should really only resolve to a single shared copy of arrow and peerDependencies let you specify that.
Particularly because arquero does have APIs for sharing constructed instances of the shared arrow library (if it was completely internal then would make sense as just a dependency).
from arquero.
Related Issues (20)
- arquero op.first_value function issue HOT 2
- table.print() should return table to enable chaining HOT 2
- Citation for academic paper HOT 2
- Nuxt 2.0 build failing due to apache-arrow exports HOT 6
- Add op functions to work with Map and Set objects HOT 1
- `fromCSV` fails with uncaught TypeError on CSV with headers only HOT 1
- Join ignores empty string as suffix
- Table expressions do not support underscores as numeric separators in numeric literals
- CSV parse functions don't get run on null values
- Failing during production build: minification problem? HOT 10
- Verb to drop columns by name? HOT 2
- derive can not handle string? HOT 2
- Table concatenation results in empty table
- Problems getting Arquero to find it's types in Typescript HOT 5
- array_agg and undefined/none values HOT 3
- fromArrow -> Unrecognized type: "undefined"
- COUNTIF-like aggregate function
- NextJS swcMinify is breaking arquero
- docs: broken links HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arquero.