delef / maxminddb.cr Goto Github PK
View Code? Open in Web Editor NEWMaxMind DB Reader for Crystal
License: MIT License
MaxMind DB Reader for Crystal
License: MIT License
In BindRPZCollector.cr:23:1
23 | require "geoip2"
^
Error: while requiring "geoip2"
In lib/geoip2/src/geoip2.cr:2:1
2 | require "./geoip2/database"
^
Error: while requiring "./geoip2/database"
In lib/geoip2/src/geoip2/database.cr:1:1
1 | require "maxminddb"
^
Error: while requiring "maxminddb"
In lib/maxminddb/src/maxminddb.cr:1:1
1 | require "./maxminddb/reader"
^
Error: while requiring "./maxminddb/reader"
In lib/maxminddb/src/maxminddb/reader.cr:3:1
3 | require "./decoder"
^
Error: while requiring "./decoder"
In lib/maxminddb/src/maxminddb/decoder.cr:1:1
1 | require "./any"
^
Error: while requiring "./any"
In lib/maxminddb/src/maxminddb/any.cr:1:1
1 | require "crystal/datum"
^
Error: can't find file 'crystal/datum'
If you're trying to require a shard:
- Did you remember to run `shards install`?
- Did you make sure you're running the compiler in the same directory as your shard.yml?
$ crystal -v
Crystal 0.32.1 (2020-05-15)
LLVM: 9.0.1
Default target: x86_64-portbld-freebsd12.1
whats is the problem.. searching 'crystal/datum' does not lead me to anyware
I curious to see how fast is this shards vs Maxmind (Node.js)?
https://github.com/runk/node-maxmind
{% if flag?(:darwin) || flag?(:openbsd) || flag?(:freebsd) %}
ipv6_address = pointer.value.sin6_addr.__u6_addr.__u6_addr8
memory.write ipv6_address.to_slice
{% elsif flag?(:linux) && flag?(:musl) %}
ipv6_address = pointer.value.sin6_addr.__in6_union.__s6_addr
memory.write ipv6_address.to_slice
{% elsif flag?(:linux) %}
ipv6_address = pointer.value.sin6_addr.__in6_u.__u6_addr8
memory.write ipv6_address.to_slice
{% else %}
return
{% end %}
I see you use sija/ipaddress.cr
in maxminddb.cr/shard.yml, I think it can be replaced.
Dependencies cause some risks (in an uncontrolled environment).
Socket::IPAddress
will make its semantic expression cleaner.Just a suggestion.
module MaxMindDB
class Reader
def get(address : String | Int)
get Socket::IPAddress.new address, 0_i32
end
def check_ip_type!(address : Socket::IPAddress)
case {metadata.ipVersion, address.family}
when {4_i32, Socket::Family::INET6}
message = String.build do |io|
io << "Error looking up " << "'" << address.to_s << "'" << ". "
io << "You attempted to look up an IPv6 address in an IPv4-only database."
end
raise ArgumentError.new message
end
end
def get(address : Socket::IPAddress)
check_ip_type! address
pointer = find_address_in_tree address
return resolve_data_pointer pointer if 0_i32 < pointer
Any.new Hash(String, Any).new
end
private def find_address_in_tree(address : Socket::IPAddress) : Int32
raise InvalidAddress.new unless raw_address = MaxMindDB.ip_address_to_bytes address
# raw_address = address.data
bit_size = raw_address.size * 8_i32
node_number = start_node bit_size
bit_size.times do |i|
break if node_number >= metadata.nodeCount
index = raw_address[i >> 3_i32]
bit = 1_i32 & (index >> 7_i32 - (i % 8_i32))
node_number = read_node node_number, bit
end
return 0_i32 if node_number == metadata.nodeCount
return node_number if node_number > metadata.nodeCount
raise InvalidDataBase.new "Something bad happened"
end
end
end
module MaxMindDB
class InvalidAddress < Exception
end
def self.ipv4_address_to_bytes(ip_address : Socket::IPAddress) : Bytes
buffer = IO::Memory.new 4_i32
split = ip_address.address.split "."
split.each { |part| buffer.write Bytes[part.to_u8] }
buffer.to_slice
end
def self.ipv6_address_to_bytes(ip_address : Socket::IPAddress) : Bytes?
return unless ip_address.family.inet6?
pointer = ip_address.to_unsafe.as LibC::SockaddrIn6*
memory = IO::Memory.new 16_i32
{% if flag? :darwin %}
ipv6_address = pointer.value.sin6_addr.__u6_addr.__u6_addr8
memory.write ipv6_address.to_slice
{% else %}
ipv6_address = pointer.value.sin6_addr.__in6_u.__u6_addr8
memory.write ipv6_address.to_slice
{% end %}
memory.to_slice
end
def self.ip_address_to_bytes(ip_address : Socket::IPAddress) : Bytes?
case ip_address.family
when .inet6?
ipv6_address_to_bytes ip_address
else
ipv4_address_to_bytes ip_address
end
end
end
the application won't build on the alpine docker image crystallang/crystal:0.35.1-alpine
maxminddb.cr/src/maxminddb/ip_address.cr
Line 42 in 6ed46a1
Not sure if this was surfaced by Crystal 0.35 or just nobody noticed so far, but:
maxminddb.cr/src/maxminddb/decoder.cr
Line 31 in 4eb492e
Any::Type
was removed in b209806.
13 | GeoIP2.open(io)
^---
Error: instantiating 'GeoIP2:Module#open(IO::Memory)'
In lib/geoip2/src/geoip2.cr:7:14
7 | Database.new(db, locales)
^--
Error: instantiating 'GeoIP2::Database.class#new(IO::Memory, Array(String))'
In lib/geoip2/src/geoip2/database.cr:7:27
7 | @reader = MaxMindDB.open(db)
^---
Error: instantiating 'MaxMindDB:Module#open(IO::Memory)'
In lib/maxminddb/src/maxminddb.cr:12:12
12 | Reader.new(input, cache_max_size)
^--
Error: instantiating 'MaxMindDB::Reader.class#new(IO::Memory, Nil)'
In lib/maxminddb/src/maxminddb/reader.cr:23:26
23 | @metadata = Metadata.new(@buffer)
^--
Error: instantiating 'MaxMindDB::Metadata.class#new(MaxMindDB::Buffer)'
In lib/maxminddb/src/maxminddb/metadata.cr:70:50
70 | metadata = Decoder.new(buffer, start_offset).decode(start_offset).as_any
^-----
Error: instantiating 'MaxMindDB::Decoder#decode(Int32)'
In lib/maxminddb/src/maxminddb/decoder.cr:57:5
57 | decode
^-----
Error: instantiating 'decode()'
In lib/maxminddb/src/maxminddb/decoder.cr:66:5
66 | decode_by_type(data_type, size)
^-------------
Error: instantiating 'decode_by_type(MaxMindDB::Decoder::DataType, Int32)'
In lib/maxminddb/src/maxminddb/decoder.cr:78:7
78 | decode_pointer(size)
^-------------
Error: instantiating 'decode_pointer(Int32)'
In lib/maxminddb/src/maxminddb/decoder.cr:149:17
149 | return Node.new(pointer) if @pointer_test
^--
Error: instantiating 'MaxMindDB::Decoder::Node.class#new(Int32)'
In lib/maxminddb/src/maxminddb/decoder.cr:31:29
31 | def initialize(@value : Any::Type)
^
Error: undefined constant Any::Type
Log
, and can also be used in other places.require "crystal/datum"
module MaxMindDB
struct Any
Crystal.datum types: {nil: Nil, bool: Bool, s: String, i: Int32, u16: UInt16, u32: UInt32, u64: UInt64, u128: UInt128, f: Float32, f64: Float64, aany: Array(Any)}, hash_key_type: String, immutable: false
def initialize(@raw : Type)
end
def found?
size > 0_i32
end
def empty?
!found?
end
def to_json(json : ::JSON::Builder)
raw.to_json json
end
def as_i : Int32
raw.as(Int).to_i
end
end
end
Seems to need to release new version.
In lib/maxminddb/src/maxminddb/decoder.cr:11:7
11 | Extended,
^
Error: expecting ';', 'end' or newline after enum member
Crystal 0.33.0 (2020-02-16)
LLVM: 9.0.1
Default target: x86_64-apple-macosx
This issue has nothing to do with this shard, I'm sorry for any interruptions.
Sorry, we were not able to create your account. Please ensure that you are using an email that is not disposable, and that you are not connecting via a proxy or VPN.
archive.org | MD5 Hash | Last-Modified Header |
---|---|---|
GeoLite2-City.tar.gz | 7ba2c58b4e0eac6c08c6399aebdae26c | Tue, 24 Dec 2019 17:46:00 GMT |
GeoLite2-Country.tar.gz | dc6224c648350d90f344a0c5c3ca5474 | Tue, 24 Dec 2019 17:40:42 GMT |
GeoLite2-ASN.tar.gz | f3c9c5775fd226db6e8098675ff65861 | Tue, 24 Dec 2019 14:16:58 GMT |
GeoLite2-City-CSV.zip | 89e5bf01a970b5668d74cf29a242d546 | Tue, 24 Dec 2019 17:45:52 GMT |
GeoLite2-Country-CSV.zip | f50b518341de54fe48c5e34f13e24e99 | Tue, 24 Dec 2019 17:40:41 GMT |
GeoLite2-ASN-CSV.zip | 463ccd73c104c52547fa50a84fc2f86e | Tue, 24 Dec 2019 14:16:58 GMT |
Thread
for some time. (I.e. -Dpreview_mt
)Hash
is not thread-safe, Hash
is volatile.require "./any.cr"
module MaxMindDB
struct Cache(K, V)
property capacity : Int32
property storage : Immutable::Map(K, V)
def initialize(@capacity : Int32)
@storage = Immutable::Map(K, V).new
end
def fetch(key : K, &block : K -> V) : V
value = storage[key]?
return value if value
value = yield key
unless full?
_storage = storage.set key, value
self.storage = _storage
end
value
end
def full?
storage.size >= capacity
end
end
end
dependencies:
immutable:
github: lucaong/immutable
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.