Giter Club home page Giter Club logo

Comments (7)

blink1073 avatar blink1073 commented on June 3, 2024 1

Ah, right, Binary type support was added in https://jira.mongodb.org/browse/ARROW-52, which has not been released yet.

from mongo-arrow.

blink1073 avatar blink1073 commented on June 3, 2024

Hi @mertbakir, can you please post the ValueError traceback that you're seeing?

from mongo-arrow.

mertbakir avatar mertbakir commented on June 3, 2024

@blink1073 hello, here it is.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[14], line 1
----> 1 schema = Schema({'_id': ObjectId,
      2                  'client_id': Binary,
      3                  'location': {
      4                     'accuracy': int,
      5                     'battery': int,
      6                     'bearing': int,
      7                     'speed': int,
      8                     'total_distance': int,
      9                     'location': {'coordinates': list}
     10                     }
     11                  })

File (file:///C:/Users/.../venv/lib/site-packages/pymongoarrow/schema.py:45), in Schema.__init__(self, schema)
     38 """Create a :class:`~pymongoarrow.schema.Schema` instance from a
     39 mapping or an iterable.
     40 
     41 :Parameters:
     42   - `schema`: A mapping.
     43 """
     44 if isinstance(schema, abc.Mapping):
---> 45     normed = type(self)._normalize_mapping(schema)
     46 else:
     47     raise ValueError("schema must be a mapping or sequence")

File (file:///C:/Users/.../venv/lib/site-packages/pymongoarrow/schema.py:58), in Schema._normalize_mapping(mapping)
     56 normed = {}
     57 for fname, ftype in mapping.items():
---> 58     normed[fname] = _normalize_typeid(ftype, fname)
     59 return normed

File (file:///C:/Users/.../venv/lib/site-packages/pymongoarrow/types.py:168), in _normalize_typeid(typeid, field_name)
    166     return normalizer(typeid)
    167 else:
--> 168     raise ValueError("Unsupported type identifier {} for field {}".format(typeid, field_name))

ValueError: Unsupported type identifier  for field client_id

from mongo-arrow.

blink1073 avatar blink1073 commented on June 3, 2024

We can't infer an arrow data type from a list, since we need to know what type of data it contains.

The schema should look something like:

from pyarrow import list_, float64
schema = Schema({'_id': ObjectId,
	'client_id': Binary,
	'location': {
	   'accuracy': int,
	   'battery': int,
	   'bearing': int,
	   'speed': int,
	   'total_distance': int,
	   'location': {'coordinates': list_(float64())}
	   }
})

This would make a good example to add to our docs for 1.0.

from mongo-arrow.

mertbakir avatar mertbakir commented on June 3, 2024

@blink1073 but client_id Binary throws error too. I am on 0.7, should I try it with 1.0?

from mongo-arrow.

mertbakir avatar mertbakir commented on June 3, 2024

Ah, right, Binary type support was added in https://jira.mongodb.org/browse/ARROW-52, which has not been released yet.

So my only option is to wait for the release? I thought I could install the in-dev. version.

from mongo-arrow.

blink1073 avatar blink1073 commented on June 3, 2024

For now, you can install from source.

from mongo-arrow.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.