Giter Club home page Giter Club logo

yql's Introduction

YQL(Yet another-Query-Language)

Build Status GoDoc

YQL is very similar with the where part of sql. You can see it as another sql which also support comparison between two sets. YQL have nearly no new concepts, so you can use it well short after reading the examples.Though it's designed for rule engine, it can be widely used in your code logic.

Install

go get github.com/caibirdme/yql

Exmaple

See more examples in the yql_test.go and godoc.

	rawYQL := `name='deen' and age>=23 and (hobby in ('soccer', 'swim') or score>90))`
	result, _ := yql.Match(rawYQL, map[string]interface{}{
		"name":  "deen",
		"age":   int64(23),
		"hobby": "basketball",
		"score": int64(100),
	})
	fmt.Println(result)
	rawYQL = `score ∩ (7,1,9,5,3)`
	result, _ = yql.Match(rawYQL, map[string]interface{}{
		"score": []int64{3, 100, 200},
	})
	fmt.Println(result)
	rawYQL = `score in (7,1,9,5,3)`
	result, _ = yql.Match(rawYQL, map[string]interface{}{
		"score": []int64{3, 5, 2},
	})
	fmt.Println(result)
	rawYQL = `score.sum() > 10`
	result, _ = yql.Match(rawYQL, map[string]interface{}{
		"score": []int{1, 2, 3, 4, 5},
	})
	fmt.Println(result)
	//Output:
	//true
	//true
	//false
	//true

And In most cases, you can use Rule to cache the AST and then use Match to get the result, which could avoid hundreds of thousands of repeated parsing process.

	rawYQL := `name='deen' and age>=23 and (hobby in ('soccer', 'swim') or score>90)`
	ruler,_ := yql.Rule(rawYQL)

	result, _ := ruler.Match(map[string]interface{}{
		"name":  "deen",
		"age":   23,
		"hobby": "basketball",
		"score": int64(100),
	})
	fmt.Println(result)
	result, _ = ruler.Match(map[string]interface{}{
		"name":  "deen",
		"age":   23,
		"hobby": "basketball",
		"score": int64(90),
	})
	fmt.Println(result)
	//Output:
	//true
	//false

Though the to be matched data is the type of map[string]interface{}, there're only 5 types supported:

  • int
  • int64
  • float64
  • string
  • bool

Helpers

In score.sum() > 10, sum is a helper function which adds up all the numbers in score, which also means the type of score must be one of the []int,[]int64 or []float64.

This repo is in the early stage, so now there are just a few helpers, feel free to create an issue about your needs. Supported helpers are listed below:

  • sum: ...
  • count: return the length of a slice or 1 if not a slice
  • avg: return the average number of a slice(float64(total)/float64(len(slice)))
  • max: return the maximum number in a slice
  • min: return the minimum number in a slice

Usage scenario

Obviously, it's easy to use in rule engine.

var handlers = map[int]func(map[string]interface{}){
	1: sendEmail,
	2: sendMessage,
	3: alertBoss,
}

data := resolvePostParamsFromRequest(request)
rules := getRulesFromDB(sql)

for _,rule := range rules {
	if success,_ := yql.Match(rule.YQL, data); success {
		handler := handlers[rule.ID]
		handler(data)
		break
	}
}

Also, it can be used in your daily work, which could significantly reduce the deeply embebbed if else statements:

func isVIP(user User) bool {
	rule := fmt.Sprintf("monthly_vip=true and now<%s or eternal_vip=1 or ab_test!=false", user.ExpireTime)
	ok,_ := yql.Match(rule, map[string]interface{}{
		"monthly_vip": user.IsMonthlyVIP,
		"now": time.Now().Unix(),
		"eternal_vip": user.EternalFlag,
		"ab_test": isABTestMatched(user),
	})
	return ok
}

Even, you can use json.Marshal to generate the map[string]interface{} if you don't want to write it manually. Make sure the structure tag should be same as the name in rawYQL.

Syntax

See grammar file

Compatibility promise

The API Matchis stable now. Its grammar won't change any more, and what I only will do next is to optimize, speed up and add more helpers if needed.

Further Trial

Though it's kinder difficult to create a robust new Go compiler, there're still some interesting things could do. For example, bringing lambda function in Go which maybe look like:

var scores = []int{1,2,3,4,5,6,7,8,9,10}
newSlice := yql.Filter(`(v) => v % 2 == 0`).Map(`(v) => v*v`).Call(scores).Interface()
//[]int{4,16,36,64,100}

If the lambda function won't change all time, it can be cached like opcode, which is as fast as the compiled code. And in most cases, who care?(pythoner?)

It's not easy but interesting, isn't it? Welcome to join me, open some issues and talk about your ideas with me. Maybe one day it can become a pre-compile tool like babel in javascript.

Attention

Lambda expression now is in its very early stage, DO NOT USE IT IN PRODUCTION.

You can take a quick preview in test case

type Student struct {
	Age  int
	Name string
}

var students = []Student{
	Student{
		Name: "deen",
		Age:  24,
	},
	Student{
		Name: "bob",
		Age:  22,
	},
	Student{
		Name: "alice",
		Age:  23,
	},
	Student{
		Name: "tom",
		Age:  25,
	},
	Student{
		Name: "jerry",
		Age:  20,
	},
}

t = yql.Filter(`(v) => v.Age > 23 || v.Name == "alice"`).Call(students).Interface()
res,_ := t.([]Student)
// res: Student{"deen",24} Student{"alice", 23} Student{"tom", 25}

Chainable

dst := []int{1, 2, 3, 4, 5, 6, 7}
r := Filter(`(v) => v > 3 && v <= 7`).Map(`(v) =>  v << 2`).Filter(`(v) => v % 8 == 0`).Call(dst)
s, err := r.Interface()
ass := assert.New(t)
ass.NoError(err)
ass.Equal([]int{16, 24}, s)

yql's People

Contributors

caibirdme avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

yql's Issues

Benchmarks

Hey,

Just stumbled across this project, looks amazing. Really curious to see how it compares against other similar projects like in terms of execution performance (not parse/check):

  1. Knetic/govaluate
  2. google/cel-go

Ambiguous import problem with the antlr module

The import needs to be updated in go.mod in accordance to the newest changes made to the antlr package, or the versioning should be limited so that it only downloads the compatible version.

github.com/test_user/test_repo imports
	github.com/caibirdme/yql imports
	github.com/antlr/antlr4/runtime/Go/antlr: ambiguous import: found package github.com/antlr/antlr4/runtime/Go/antlr in multiple modules:
	github.com/antlr/antlr4 v0.0.0-20210121092344-5dce78c87a9e (/Users/me/go/pkg/mod/github.com/antlr/[email protected]/runtime/Go/antlr)
	github.com/antlr/antlr4/runtime/Go/antlr v0.0.0-20210826220005-b48c857c3a0e (/Users/me/go/pkg/mod/github.com/antlr/antlr4/runtime/!go/[email protected])

Antlr issue: antlr/antlr4#3188

关于*_test.go的几个问题

版主,你好!
在学习你的代码的过程中,发现几个问题,想请教一下。
在文件 yql_test.go 中引用yql包中的部分接口函数时,并没有使用yql.Match 并且 使用的常量 epsilon 也是不能直接引用的,不知道是我这边环境的问题还是代码问题?

In operator has problems in some cases

If there is only one option, always return an error,example,
rawSQL := hobby in ('swim') , data := map[string]interface{}{"hobby": "swim"}
will return false

lambda.Filter能用了吗?

如题。。 我看文档里写着处于非常早期的阶段不建议生产使用。

我的场景里只有A==X || B==Y

License

Do you mind adding a license to this?

Consider migrating to "github.com/antlr/antlr4/runtime/Go/antlr/v4"

It seems that v1 module of github.com/antlr/antlr4/runtime/Go/antlr has been deprecated.

https://github.com/antlr/antlr4/blob/master/runtime/Go/antlr/go.mod

// Deprecated: Please switch to the new v4 module path: github.com/antlr/antlr4/runtime/Go/antlr/v4 - see https://github.com/antlr/antlr4/blob/master/doc/go-target.md
module github.com/antlr/antlr4/runtime/Go/antlr

The new module is github.com/antlr/antlr4/runtime/Go/antlr/v4 and it fit more properly with the idiomatic ways of Go than v1 module. For example, migrating to v4 may reduce the likehood of ambiguous import error related to antlr.

Is there any plan to migrate to github.com/antlr/antlr4/runtime/Go/antlr/v4? If so, this doc may help.

String Array

Hello, Is it possible to match string array?

rawYQL = stringArray in ('test1')
result, _ = yql.Match(rawYQL, map[string]interface{}{
"stringArray": []string{"test1","test2"},
})

support for nested fields?

Is there support for querying on nested fields?

Given this sample:

{  
   "src_ip":"127.0.0.1",
   "event":{  
      "Event":{  
         "Process":{  
            "type":2,
            "exec_filename":"/usr/bin/dig",
            "exec_command_line":[  
               "dig",
               "+short",
               "ifjeow0234f90iwefo2odj.wat.lol"
            ]
         }
      }
   }
}

Can I do something like these queries using this library?
event.Event.Process.exec_filename='/usr/bin/dig' and src_ip='127.0.0.1'

How about querying a nested array?
src_ip='127.0.0.1' and 'ifjeow0234f90iwefo2odj.wat.lol' in event.Event.Process.exec_command_line

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.