Comments (5)
kimjoaoun
commented on June 17, 2024
2
Thank you @philss and @cigrainger. Sorry for opening the useless issue. D:
from explorer.
cigrainger
commented on June 17, 2024
1
Well that's a big relief @philss, thank you. I was very confused over here. Closing, as I've confirmed it:
iex(1)> IEx.configure(inspect: [charlists: :as_lists])
:ok
iex(2)> df = Explorer.Datasets.fossil_fuels()
#Explorer.DataFrame<
[rows: 1094, columns: 10]
year integer [2010, 2010, 2010, 2010, 2010, "..."]
country string ["AFGHANISTAN", "ALBANIA", "ALGERIA", "ANDORRA", "ANGOLA",
"..."]
total integer [2308, 1254, 32500, 141, 7924, "..."]
solid_fuel integer [627, 117, 332, 0, 0, "..."]
liquid_fuel integer [1601, 953, 12381, 141, 3649, "..."]
gas_fuel integer [74, 7, 14565, 0, 374, "..."]
cement integer [5, 177, 2598, 0, 204, "..."]
gas_flaring integer [0, 0, 2623, 0, 3697, "..."]
per_capita float [0.08, 0.43, 0.9, 1.68, 0.37, "..."]
bunker_fuels integer [9, 7, 663, 0, 321, "..."]
>
iex(3)> df |> Explorer.DataFrame.slice(0, 1) |> Explorer.DataFrame.to_map()
%{
bunker_fuels: [9],
cement: [5],
country: ["AFGHANISTAN"],
gas_flaring: [0],
gas_fuel: [74],
liquid_fuel: [1601],
per_capita: [0.08],
solid_fuel: [627],
total: [2308],
year: [2010]
}from explorer.
cigrainger
commented on June 17, 2024
gas_fuel appears to also be corrupted, returning 'J'. Looking into this with urgency.
Edit: it appears this is happening only up to a certain slice size. For example:
iex(6)> df |> Explorer.DataFrame.slice(0, 2) |> Explorer.DataFrame.to_map()
%{
bunker_fuels: '\t\a',
cement: [5, 177],
country: ["AFGHANISTAN", "ALBANIA"],
gas_flaring: [0, 0],
gas_fuel: 'J\a',
liquid_fuel: [1601, 953],
per_capita: [0.08, 0.43],
solid_fuel: [627, 117],
total: [2308, 1254],
year: [2010, 2010]
}
iex(7)> df |> Explorer.DataFrame.slice(0, 3) |> Explorer.DataFrame.to_map()
%{
bunker_fuels: [9, 7, 663],
cement: [5, 177, 2598],
country: ["AFGHANISTAN", "ALBANIA", "ALGERIA"],
gas_flaring: [0, 0, 2623],
gas_fuel: [74, 7, 14565],
liquid_fuel: [1601, 953, 12381],
per_capita: [0.08, 0.43, 0.9],
solid_fuel: [627, 117, 332],
total: [2308, 1254, 32500],
year: [2010, 2010, 2010]
}And, just to confirm, there appears to be no problem prior to converting to a map:
iex(2)> df |> Explorer.DataFrame.slice(0, 1)
#Explorer.DataFrame<
[rows: 1, columns: 10]
year integer [2010]
country string ["AFGHANISTAN"]
total integer [2308]
solid_fuel integer [627]
liquid_fuel integer [1601]
gas_fuel integer [74]
cement integer [5]
gas_flaring integer [0]
per_capita float [0.08]
bunker_fuels integer [9]
>
iex(3)> df |> Explorer.DataFrame.slice(0, 2)
#Explorer.DataFrame<
[rows: 2, columns: 10]
year integer [2010, 2010]
country string ["AFGHANISTAN", "ALBANIA"]
total integer [2308, 1254]
solid_fuel integer [627, 117]
liquid_fuel integer [1601, 953]
gas_fuel integer [74, 7]
cement integer [5, 177]
gas_flaring integer [0, 0]
per_capita float [0.08, 0.43]
bunker_fuels integer [9, 7]
>
from explorer.
philss
commented on June 17, 2024
@kimjoaoun @cigrainger hey guys, I think this is just a matter of how Elixir is printing the lists. If you try this:
df = Explorer.Datasets.fossil_fuels()
df |> Explorer.DataFrame.slice(0, 1) |> Explorer.DataFrame.to_map() |> IO.inspect(charlists: :as_lists)You can see that the inspected value is returned correctly. This is because by default IEx will try to infer the type of a list. Sometimes it is detected as charlist. See https://hexdocs.pm/elixir/Inspect.Opts.html for details.
To disable this behavior in IEx, you can run IEx.configure(inspect: [charlists: :as_lists]).
For more details, please check this Stackoverflow question: https://stackoverflow.com/questions/30037914/elixir-lists-interpreted-as-char-lists
from explorer.
philss
commented on June 17, 2024
@kimjoaoun no problem :)
from explorer.
Related Issues (20)
- Add support for log and exp operations in DF.mutate HOT 5
- Support returning infinity and nan from aggregation functions
- Allow summarize to work without groups
- Allow stable sorting to be turned off HOT 1
- RFC: Rename functions from_ndjson and load_ndjson HOT 1
- Improvements to categorical to series Rust implementation
- Series.and/2 and Series.or/2 behaviours HOT 1
- Use of `..` nullary operator in Release 0.5.3 introduced Elixir 1.4 dependency
- ISO 8601 DateTime format with timezone info can't be parsered HOT 5
- Allow diff of date and datetime series HOT 20
- DataFrame.iter_rows method? HOT 3
- libexplorer-v0.5.6-nif-2.16-aarch64-apple-darwin.so nif panic HOT 6
- Min not working for datetime[NS] HOT 4
- Largish CSV hangs on from_csv/load_csv HOT 8
- Return {:error, string} instead of {:error, {:polars, string()}} HOT 3
- [idea] Allow pivot_wider on categorical values HOT 12
- load_csv for a single-column csv drops the first row if nil (nit) HOT 1
- Missing Window Functions from Polars HOT 6
- AVX support HOT 1
- Support relocating columns
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from explorer.