Giter Club home page Giter Club logo

mongo-types's Introduction

mongo-types PyPI

Type stubs for mongoengine.

Allows for autocomplete and static typing.

install

pip install mongo-types

Monkey patch mongoengine's QuerySet so we can type it with a generic argument at runtime:

import types
from mongoengine.queryset.queryset import QuerySet

def no_op(self, x):
    return self

QuerySet.__class_getitem__ = types.MethodType(no_op, QuerySet)

usage

After installing and monkey patching, the types should work for the most part, but you'll probably need to change how you write some things.

getting objects to work

By default, the base document is typed to not have an objects property so that each document can type it properly.

Here's a helper class that's useful for simple cases which don't modify the QuerySet.

from typing import Generic, Type, TypeVar
from mongoengine import QuerySet, Document

U = TypeVar("U", bound=Document)

class QuerySetManager(Generic[U]):
    def __get__(self, instance: object, cls: Type[U]) -> QuerySet[U]:
        return QuerySet(cls, cls._get_collection())

class Page(Document):
    meta = {
        "collection": "pages",
    }

    objects = QuerySetManager["Page"]()

    organization = fields.StringField()

replacing usages of queryset_class

before:

from typing import Type
from mongoengine import QuerySet, Document

class PostQuerySet(QuerySet):
    def for_org(self, *, org: str) -> QuerySet:
        return self.filter(organization=org)

    def exists(self) -> bool:
        return self.count() > 0

class Post(Document):
    meta = {
        "collection": "posts",
        "queryset_class": SMSLogQuerySet,
    }

    organization = fields.StringField()
    # --snip--

after:

from typing import Type
from mongoengine import QuerySet, Document

class PostQuerySet(QuerySet["Post"]):
    def for_org(self, *, org: str) -> QuerySet["Post"]:
        return self.filter(organization=org)

    def exists(self) -> bool:
        return self.count() > 0


class QuerySetManager:
    def __get__(self, instance: object, cls: Type[Post]) -> PostQuerySet:
        return PostQuerySet(cls, cls._get_collection())


class Post(Document):
    meta = {
        "collection": "posts",
    }

    objects = QuerySetManager()

    organization = fields.StringField()
    # --snip--

replicating @queryset_manager behavior

before:

from mongoengine import Document, QuerySet, queryset_manager, fields

class UserQuerySet(QuerySet):
    def for_org(self, *, org: str) -> QuerySet:
        return self.filter(organization=org)

class User(Document):
    meta = {
        "collection": "users",
        "queryset_class": UserQuerySet,
    }

    is_active = fields.BooleanField()

    # --snip--

    @queryset_manager
    def objects(self, queryset: QuerySet) -> QuerySet:
        return queryset.filter(is_active=True)

    @queryset_manager
    def all_objects(self, queryset: QuerySet) -> QuerySet:
        return queryset

maybe_user = User.all_objects.first()

after:

from __future__ import annotations
from typing import Type
from mongoengine import QuerySet, Document

class UserQuerySet(QuerySet["User"]):
    def for_org(self, *, org: str) -> UserQuerySet:
        return self.filter(organization=org)


class QuerySetManager:
    def __get__(self, instance: object, cls: Type[User]) -> UserQuerySet:
        return UserQuerySet(cls, cls._get_collection()).filter(is_active=True)


class User(Document):
    meta = {
        "collection": "users",
    }

    is_active = fields.BooleanField()

    # --snip--

    objects = QuerySetManager()

    @classmethod
    def all_objects(cls) -> UserQuerySet:
        return UserQuerySet(cls, cls._get_collection())

maybe_user = User.all_objects().first()

fixing "Model" has no attribute "id"

Mongoengine will define an id field for you automatically. Mongo-types require you specify your id explicitly so that the types can be more strict.

class User(Document):
    meta = {
        "collection": "users",
    }

# becomes

class User(Document):
    meta = {
        "collection": "users",
    }
    id = fields.StringField(db_field="_id", primary_key=True, default=default_id)

# or if you prefer ObjectIds

class User(Document):
    meta = {
        "collection": "users",
    }
    id = fields.ObjectIdField(db_field="_id", primary_key=True, default=ObjectId)

dev

poetry install

# run formatting, linting, and typechecking
s/lint

# build
poetry build -f wheel

# build and publish
poetry publish --build

related

mongo-types's People

Contributors

chdsbd avatar glennolsson avatar gomonuk avatar last-partizan avatar phil-airship avatar sbdchd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

mongo-types's Issues

remove `__getitem__` from Mongoengine models

__getitem__ is typed to return Any since we don't have a way to derive overloads based off the class properties.

Instead of __getitem__ users can use getattr if they really want to access properties dynamically.

port argument of register_connection() is typed as str | None, but mongoengine expects int

mypy reports that the port argument for the register_connection() function needs to be either a string or None:

Argument "port" to "register_connection" has incompatible type "int"; expected "str | None"  [arg-type]

Which is what the mongoengine-stubs/connection.pyi file dictates. But when I replace it with a string representation of the port number, mongoengine reports the port must be an integer.

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/mongoengine/connection.py", line 348, in _create_connection
    return mongo_client_class(**connection_settings)
  File "/usr/local/lib/python3.10/site-packages/pymongo/mongo_client.py", line 715, in __init__
    raise TypeError("port must be an instance of int")
TypeError: port must be an instance of int

According to pymongo on github, the type check on the port has been there for 11 years, so I think this is just an oversight.

`get_database` method in `MongoClient` does not have any arguments defined

get_database method in MongoClient actually has arguments defined like

    def get_database(
        self,
        name: Optional[str] = None,
        codec_options: Optional[CodecOptions] = None,
        read_preference: Optional[_ServerMode] = None,
        write_concern: Optional[WriteConcern] = None,
        read_concern: Optional["ReadConcern"] = None,
    ) -> database.Database[_DocumentType]:

But in the stub file, this method doesn't have any arguments

Missing stubs in mongoengine connection

Steps to reproduce:

  1. Create a module that includes the following code:
import mongoengine

mongoengine.register_connection(
    alias="core",
    name="foobar",
    username="username",
    password="password",
)

connection = mongoengine.get_connection("core")
  1. Run mypy against it:
python -m mypy ./mongo-types-mypy_issue.py
  1. Observe that mypy does not trigger on register_connection, but does trigger on get_connection:
[snip ...]/mongo-types-mypy_issue.py:10: error: Module has no attribute "get_connection"  [attr-defined]
Found 1 error in 1 file (checked 1 source file)

Expected Behavior: Since __all__ exported by the connection.py module includes get_connection that there would be a stub for it.

`upsert` missing from `replace_one`

As of pymongo 4.3.3 the Collection.replace_one method has the upsert parameter, but it's missing from the stub definitions:

https://github.com/sbdchd/mongo-types/blob/main/pymongo-stubs/collection.pyi#L76

    def replace_one(
        self,
        filter: Mapping[str, Any],
        replacement: Mapping[str, Any],
        bypass_document_validation: bool = ...,
        collation: Optional[Collation] = ...,
        hint: Optional[Any] = ...,
        session: Optional[ClientSession] = ...,
    ) -> UpdateResult: ...

This leads to the following false positive from mypy (v0.991):

error: Unexpected keyword argument "upsert" for "replace_one" of "Collection"  [call-arg]

QuerySetManager: 'type' object is not subscriptable

Hi,

I am using Python 3.10.4 and Poetry 1.1.13 and I have the following dependencies:

black           22.3.0
click           8.1.3 
mongo-types     0.15.1
mongoengine     0.24.1
mypy-extensions 0.4.3  
pathspec        0.9.0  
platformdirs    2.5.2 
pymongo         4.1.1 
tomli           2.0.1 

And a simple test project:

# main.py

from typing import Generic, Type, TypeVar

from mongoengine import Document, QuerySet, connect, fields

connect(host="mongodb://root:[email protected]:27017/my_db?authSource=admin")

U = TypeVar("U", bound=Document)


class QuerySetManager(Generic[U]):
    def __get__(self, instance: object, cls: Type[U]) -> QuerySet[U]:
        return QuerySet(cls, cls._get_collection())


class Shot(Document):
    meta = {
        "collection": "shots",
    }
    objects = QuerySetManager["Shot"]()
    entity_id = fields.StringField()

I have the following error when launching it:

$ poetry run python main.py
Traceback (most recent call last):
  File "/home/josephhenry/mongo-types-test/main.py", line 10, in <module>
    class QuerySetManager(Generic[U]):
  File "/home/josephhenry/mongo-types-test/main.py", line 11, in QuerySetManager
    def __get__(self, instance: object, cls: Type[U]) -> QuerySet[U]:
TypeError: 'type' object is not subscriptable

Did I do something wrong?

Thanks!

MongoDB's `motor` official library support?

I am currently developing an asynchronous app with motor instead of pymongo and mypy. As far as I know, it's not that different from PyMongo, and, in some places uses it under the hood. I wonder if you can review the possibility of writing type-hints for it too. Thank you for your work on mongo-types!

remove `required=False` as a keyword arg

required=False is the default behavior. We should remove this option to prevent mistakes where callers mistakenly think required=True is the default behavior.

The following definitions are equivalent

class Post:
    author = fields.StringField()

class Post:
    author = fields.StringField(required=False)

This is different.

class Post:
    author = fields.StringField(required=True)

With this change, it would only be possible to write:

class Post:
    author = fields.StringField()

class Post:
    author = fields.StringField(required=True)

The following would error:

class Post:
    author = fields.StringField(required=False)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.