Giter Club home page Giter Club logo

maxminddb.cr's Introduction

๐Ÿ‘‹ Hello!

๐Ÿ“Š Stats

maxminddb.cr's People

Contributors

delef avatar gewo avatar jgillich avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

maxminddb.cr's Issues

Error: can't find file 'crystal/datum'

In BindRPZCollector.cr:23:1

 23 | require "geoip2"
      ^
Error: while requiring "geoip2"


In lib/geoip2/src/geoip2.cr:2:1

 2 | require "./geoip2/database"
     ^
Error: while requiring "./geoip2/database"


In lib/geoip2/src/geoip2/database.cr:1:1

 1 | require "maxminddb"
     ^
Error: while requiring "maxminddb"


In lib/maxminddb/src/maxminddb.cr:1:1

 1 | require "./maxminddb/reader"
     ^
Error: while requiring "./maxminddb/reader"


In lib/maxminddb/src/maxminddb/reader.cr:3:1

 3 | require "./decoder"
     ^
Error: while requiring "./decoder"


In lib/maxminddb/src/maxminddb/decoder.cr:1:1

 1 | require "./any"
     ^
Error: while requiring "./any"


In lib/maxminddb/src/maxminddb/any.cr:1:1

 1 | require "crystal/datum"
     ^
Error: can't find file 'crystal/datum'

If you're trying to require a shard:
- Did you remember to run `shards install`?
- Did you make sure you're running the compiler in the same directory as your shard.yml?
$ crystal -v
Crystal 0.32.1 (2020-05-15)

LLVM: 9.0.1
Default target: x86_64-portbld-freebsd12.1

whats is the problem.. searching 'crystal/datum' does not lead me to anyware

Improve: IPAddress.ipv6_to_bytes Musl Platform support

Summary

Solution

    {% if flag?(:darwin) || flag?(:openbsd) || flag?(:freebsd) %}
      ipv6_address = pointer.value.sin6_addr.__u6_addr.__u6_addr8
      memory.write ipv6_address.to_slice
    {% elsif flag?(:linux) && flag?(:musl) %}
      ipv6_address = pointer.value.sin6_addr.__in6_union.__s6_addr
      memory.write ipv6_address.to_slice
    {% elsif flag?(:linux) %}
      ipv6_address = pointer.value.sin6_addr.__in6_u.__u6_addr8
      memory.write ipv6_address.to_slice
    {% else %}
      return
    {% end %}

Unnecessary dependencies?

Summary

Improve

module MaxMindDB
  class Reader
    def get(address : String | Int)
      get Socket::IPAddress.new address, 0_i32
    end

    def check_ip_type!(address : Socket::IPAddress)
      case {metadata.ipVersion, address.family}
      when {4_i32, Socket::Family::INET6}
        message = String.build do |io|
          io << "Error looking up " << "'" << address.to_s << "'" << ". "
          io << "You attempted to look up an IPv6 address in an IPv4-only database."
        end

        raise ArgumentError.new message
      end
    end

    def get(address : Socket::IPAddress)
      check_ip_type! address

      pointer = find_address_in_tree address
      return resolve_data_pointer pointer if 0_i32 < pointer

      Any.new Hash(String, Any).new
    end

    private def find_address_in_tree(address : Socket::IPAddress) : Int32
      raise InvalidAddress.new unless raw_address = MaxMindDB.ip_address_to_bytes address

      # raw_address = address.data
      bit_size = raw_address.size * 8_i32
      node_number = start_node bit_size

      bit_size.times do |i|
        break if node_number >= metadata.nodeCount

        index = raw_address[i >> 3_i32]
        bit = 1_i32 & (index >> 7_i32 - (i % 8_i32))

        node_number = read_node node_number, bit
      end

      return 0_i32 if node_number == metadata.nodeCount
      return node_number if node_number > metadata.nodeCount

      raise InvalidDataBase.new "Something bad happened"
    end
  end
end
module MaxMindDB
  class InvalidAddress < Exception
  end

  def self.ipv4_address_to_bytes(ip_address : Socket::IPAddress) : Bytes
    buffer = IO::Memory.new 4_i32

    split = ip_address.address.split "."
    split.each { |part| buffer.write Bytes[part.to_u8] }

    buffer.to_slice
  end

  def self.ipv6_address_to_bytes(ip_address : Socket::IPAddress) : Bytes?
    return unless ip_address.family.inet6?

    pointer = ip_address.to_unsafe.as LibC::SockaddrIn6*
    memory = IO::Memory.new 16_i32

    {% if flag? :darwin %}
      ipv6_address = pointer.value.sin6_addr.__u6_addr.__u6_addr8
      memory.write ipv6_address.to_slice
    {% else %}
      ipv6_address = pointer.value.sin6_addr.__in6_u.__u6_addr8
      memory.write ipv6_address.to_slice
    {% end %}

    memory.to_slice
  end

  def self.ip_address_to_bytes(ip_address : Socket::IPAddress) : Bytes?
    case ip_address.family
    when .inet6?
      ipv6_address_to_bytes ip_address
    else
      ipv4_address_to_bytes ip_address
    end
  end
end

undefined constant Any::Type

Not sure if this was surfaced by Crystal 0.35 or just nobody noticed so far, but:

def initialize(@value : Any::Type)

Any::Type was removed in b209806.

 13 | GeoIP2.open(io)
             ^---
Error: instantiating 'GeoIP2:Module#open(IO::Memory)'


In lib/geoip2/src/geoip2.cr:7:14

 7 | Database.new(db, locales)
              ^--
Error: instantiating 'GeoIP2::Database.class#new(IO::Memory, Array(String))'


In lib/geoip2/src/geoip2/database.cr:7:27

 7 | @reader = MaxMindDB.open(db)
                         ^---
Error: instantiating 'MaxMindDB:Module#open(IO::Memory)'


In lib/maxminddb/src/maxminddb.cr:12:12

 12 | Reader.new(input, cache_max_size)
             ^--
Error: instantiating 'MaxMindDB::Reader.class#new(IO::Memory, Nil)'


In lib/maxminddb/src/maxminddb/reader.cr:23:26

 23 | @metadata = Metadata.new(@buffer)
                           ^--
Error: instantiating 'MaxMindDB::Metadata.class#new(MaxMindDB::Buffer)'


In lib/maxminddb/src/maxminddb/metadata.cr:70:50

 70 | metadata = Decoder.new(buffer, start_offset).decode(start_offset).as_any
                                                   ^-----
Error: instantiating 'MaxMindDB::Decoder#decode(Int32)'


In lib/maxminddb/src/maxminddb/decoder.cr:57:5

 57 | decode
      ^-----
Error: instantiating 'decode()'


In lib/maxminddb/src/maxminddb/decoder.cr:66:5

 66 | decode_by_type(data_type, size)
      ^-------------
Error: instantiating 'decode_by_type(MaxMindDB::Decoder::DataType, Int32)'


In lib/maxminddb/src/maxminddb/decoder.cr:78:7

 78 | decode_pointer(size)
      ^-------------
Error: instantiating 'decode_pointer(Int32)'


In lib/maxminddb/src/maxminddb/decoder.cr:149:17

 149 | return Node.new(pointer) if @pointer_test
                   ^--
Error: instantiating 'MaxMindDB::Decoder::Node.class#new(Int32)'


In lib/maxminddb/src/maxminddb/decoder.cr:31:29

 31 | def initialize(@value : Any::Type)
                              ^
Error: undefined constant Any::Type

Use Crystal 0.34 Datum feature instead of MaxMindDB::Any?

Summary

  • Crystal introduced a Datum feature in version 0.34, which provides support for Log, and can also be used in other places.
  • I have used it in my branch repository and it works very well.

Improve

  • Reuqire Datum
require "crystal/datum"
  • MaxMindDB::Any
module MaxMindDB
  struct Any
    Crystal.datum types: {nil: Nil, bool: Bool, s: String, i: Int32, u16: UInt16, u32: UInt32, u64: UInt64, u128: UInt128, f: Float32, f64: Float64, aany: Array(Any)}, hash_key_type: String, immutable: false

    def initialize(@raw : Type)
    end

    def found?
      size > 0_i32
    end

    def empty?
      !found?
    end

    def to_json(json : ::JSON::Builder)
      raw.to_json json
    end

    def as_i : Int32
      raw.as(Int).to_i
    end
  end
end

References

Error: expecting ';', 'end' or newline after enum member

Summary

Seems to need to release new version.

Problem

In lib/maxminddb/src/maxminddb/decoder.cr:11:7

 11 | Extended,
      ^
Error: expecting ';', 'end' or newline after enum member

Version

  • Crystal
Crystal 0.33.0 (2020-02-16)

LLVM: 9.0.1
Default target: x86_64-apple-macosx

GeoLite2 download Problem

Summary

This issue has nothing to do with this shard, I'm sorry for any interruptions.

  • Some people may not be able to download MaxmindDB.
    • Sorry, we were not able to create your account. Please ensure that you are using an email that is not disposable, and that you are not connecting via a proxy or VPN.
  • Fortunately someone mirrored this file before.

Dowmload

archive.org MD5 Hash Last-Modified Header
GeoLite2-City.tar.gz 7ba2c58b4e0eac6c08c6399aebdae26c Tue, 24 Dec 2019 17:46:00 GMT
GeoLite2-Country.tar.gz dc6224c648350d90f344a0c5c3ca5474 Tue, 24 Dec 2019 17:40:42 GMT
GeoLite2-ASN.tar.gz f3c9c5775fd226db6e8098675ff65861 Tue, 24 Dec 2019 14:16:58 GMT
GeoLite2-City-CSV.zip 89e5bf01a970b5668d74cf29a242d546 Tue, 24 Dec 2019 17:45:52 GMT
GeoLite2-Country-CSV.zip f50b518341de54fe48c5e34f13e24e99 Tue, 24 Dec 2019 17:40:41 GMT
GeoLite2-ASN-CSV.zip 463ccd73c104c52547fa50a84fc2f86e Tue, 24 Dec 2019 14:16:58 GMT

References

Ensuring cache feature is thread safe?

Summary

  • Crystal has supported Thread for some time. (I.e. -Dpreview_mt)
  • Hash is not thread-safe, Hash is volatile.
  • Once multiple threads try to write the same key value at the same time, the program crashes.
  • This issue has appeared in my other project, so I raised an issue.

Solution

Improve

require "./any.cr"

module MaxMindDB
  struct Cache(K, V)
    property capacity : Int32
    property storage : Immutable::Map(K, V)

    def initialize(@capacity : Int32)
      @storage = Immutable::Map(K, V).new
    end

    def fetch(key : K, &block : K -> V) : V
      value = storage[key]?
      return value if value

      value = yield key

      unless full?
        _storage = storage.set key, value
        self.storage = _storage
      end

      value
    end

    def full?
      storage.size >= capacity
    end
  end
end
dependencies:
  immutable:
    github: lucaong/immutable

References

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.