Giter Club home page Giter Club logo

goxyq's Introduction

goxyq

Goxyq is a proxy server written in Golang with built in queues that may be used to sequentialize an asynchronous system or APIs. Goxyq uses REDIS as its storage system for queuing information.

Disclaimer: Goxyq's current design is quite simple and pretty ad-hoc but with enough generality to be used in a great gamut of use cases. The system will be maintained if people show interest for it. If not... it already served me well in a production system

Usage

  • To install the tool just run (be sure that your $GOPATH is set):
go get github.com/adlrocha/goxyq

or just clone the repo.

  • Before running the proxy be sure that you have a REDIS instance running. You can achieve this by running the following in the repo's path:
./sccripts/redis_start.sh
  • Now you are ready to run the goxyq:
go run server.go
  • You can also build the tool and run it:
go build
./server

Configuration

To modify Goxyq's configuration go to ./config/config.go.

  • Port: Goxyq's listening port.
  • DestinationHost: Destination host.
  • ProxyPathPrefix: Prefix path for which the proxy will inspect traffic.
  • QueueAtrribute: Body attributes that will trigger the creation of a new queue or the stack of the request.
  • HeaderBypass: List of headers that will be bypassed by the proxy.

Potential short-term enhancements

  • Allow the use of multiple QueueAttributes.
  • Use of etcd instead of REDIS as queue storage (it better fits decentralized environments).

goxyq's People

Contributors

adlrocha avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

goxyq's Issues

Question about the project

I am looking for a asynchronous proxy server, that will allow me to tunnel requests over a message queue to a server that is behind a firewall. I do not want to allow incoming connections to the server from the internet.

I was thinking of running a proxy server on an internet exposed server, which will push the request to a message queue (Redis / RabbitMQ). A client instance running on the internal server will fetch the request from the queue and replay it locally, and save the response back into the queue.

The proxy server on the internet exposed server will fetch the response from the queue and pass it on the the original requester.

Does goxyq support the above use case?

Another question about the project (sorry)

I think this project is what I'm looking for, but I can't quite tell.

I'm looking to proxy HTTP requests from a backend server to an API, but rate-limit them. It looks you've purpose-built this for exactly that purpose. If I'm reading (skimming, actually) the code correctly, the queuing system ensures that only a single request is in-flight at any given time for any queue (queues are created for every different value of configured QueueAtrribute), but sends them out back-to-back. Requests with a header matching one in HeaderBypass do not get queued and are simply sent out as soon as they are received.

Is that all correct? In my specific case, I would need to implement a delay between each request (I'm using this to avoid hitting a rate-limit on a 3rd party API) - I would be willing to submit a PR for this, or at least attempt one, provided 1) you're open to this, 2) my Go doesn't fail me 3) it's not too complicated. I would ideally like to queue requests to several 3rd party APIs, but don't mind running multiple instances of goxyq if need be. Do you think goxyq is the right solution here?

Thanks for publishing goxyq and for taking the time to read this (and hopefully answer!)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.