Apache Thrift is a lightweight, language-independent software stack with an associated code generation mechanism for RPC.
Thrift.jl is an implementation of Thrift for Julia, including a plugin for the Thrift IDL compiler.
Install the Julia Thrift package: Pkg.add("Thrift")
.
On Linux and OSX a Julia Thrift compiler will be installed under the deps
folder. You may add it to your PATH
environment for convenience. On Windows, no pre-built Julia Thrift compiler is installed. Note that the Thrift compiler is only needed to generate Julia code from IDLs. If generated code is already available, Thrift.jl can be used with it even if a compiler is not available.
To build the Julia Thrift compiler (if needed):
- Patch the Thrift IDL compiler with the Julia code generator plugin. Any one of the following two methods can be followed:
- Download Thrift sources from Apache. Place Julia plugin
t_jl_generator.cc
intocompiler/cpp/src/generate
folder. Update makefiles to include the new source and rebuild. - Clone Thrift sources from a fork here.
- Download Thrift sources from Apache. Place Julia plugin
- Build and install the Thrift compiler. Some instructions here.
A sample Hello Julia IDL and implementation is bundled along with the Thrift.jl package. It can be found under test/hello
folder of the package.
It contains a Thrift IDL named hello.thrift
, which contains a service SayHello
with a hello
method that returns a hello message for the supplied name in a randomly chosen language.
-
Generate Julia sources from the IDL.
Run command:
thrift --gen jl hello.thrift
. This should result in agen-jl
folder with sources generated from the IDL placed in a folderhello
(named after the IDL file name). -
Examine the generated files. Below is a brief explanation of the contents of the generated files.
hello_types.jl
: contains Julia types for Thrift structs, exceptions and enums declared explicitly in the IDL along with other implicit types generated by the code generator.hello_constants.jl
: contains any constants declared in the IDLSayHello.jl
: code generated for theSayHello
service.hello.jl
: contains a module namedhello
(named after the IDL file name), that includes the above mentioned generated files. It also includes a file namedhello_impl.jl
that is not generated, but must be created by the user.
-
An implementation of the service methods are already provided as
hello_impl.jl
in thetest/hello
folder. It has an implementation ofhello
service method, that appends a randomly chosen greeting from the constant arrayGREETINGS
to the supplied name. -
Place the
hello_impl.jl
file in thegen-jl/hello
folder. -
The client and server implementations for this are already provided as
clnt.jl
andsrvr.jl
. Start the server withjulia srvr.jl
. Run the client with the commandjulia clnt.jl
.
Types used as Thrift structures are regular Julia types and the Julia syntax to set and get fields can be used on them. But with fields that are set as optional, it is quite likely that some of them may not have been present in the instance that was read. Similarly, fields that need to be sent need to be explicitly marked as being set. The following methods are exported to assist doing this:
get_field(obj, fld::Symbol)
: Getsobj.fld
if it has been set. Throws an error otherwise.set_field!(obj, fld::Symbol, val)
: Setsobj.fld = val
and marks the field as being set. The value would be written on the wire whenobj
is serialized. Fields can also be set the regular way, but then they must be marked as being set using thefillset
method.has_field(obj, fld::Symbol)
: Checks whether fieldfld
has been set inobj
.clear(obj, fld::Symbol)
: Marks fieldfld
ofobj
as unset.clear(obj)
: Marks all fields ofobj
as unset.
The thriftbuild method makes it easier to set large types with many fields:
thriftbuild{T}(::Type{T}, nvpairs::Dict{Symbol}()=Dict{Symbol,Any}())
copy!(to, from)
: shallow copy of objectsisfilled(obj, fld::Symbol)
: same ashas_field
isfilled(obj)
: whether all mandatory fields are setfillset(obj, fld::Symbol)
: mark field fld of object obj as setfillunset(obj)
: mark all fields of this object as not setfillunset(obj, fld::Symbol)
: mark field fld of object obj as not setenumstr(enumname, enumvalue::Int32)
: returns a string with the enum field name matching the value
The generated code largely follows the scheme used in other languages, e.g. Python and C++. Each Thrift program (IDL file) is placed into a separate folder. The program (IDL file) name must be different from any of the service names defined in the program. Generated files may get clobbered if that is violated, because of filename clashes.
A Julia module is also generated bundle all sources together. Using the module is optional, though convenient in most cases. The example in test/calculator
illustrates how to include multiple thrift generated services in a single Julia module, without using the autogenerated modules.
The generated service Processor
now assumes that the implemented methods are present in the current module. Thus the generated code is not a complete module and requires the user to supply a service implementation to be complete. An alternative would be to make the generated code a complete module, and have the user supply an implementation module.
Service extensions are supported. The thrift processor on the server side passes on any methods it can not handle to the processor it extends from. Extensions of service clients are supported through Julia type extension.
The code generator can be tweaked in the future towards any preferred way of usage that may appear with further usage.
Thrift serialization can be customized for a type by defining a meta
method on it. The meta
method provides an instance of ThriftMeta
that allows specification of optional fields, field numbers, and default values for fields for a type. The Thrift code generator generates appropriate meta
methods wherever required. The below information will however help in understanding and tweaking the generated code if required.
Defining a specialized meta
is done simply as below:
import Thrift.meta
meta(t::Type{MyType}) = meta(t, # the type which this is for
Symbol[:intval], # optional fields
Int[8, 10], # field numbers
Dict{Symbol,Any}(:strval => "default value")) # default values
Without any specialized meta
method:
- All fields are marked as required.
- Field numbers are assigned serially starting from -1, and decremented in the order of their declaration.
- No default values are assigned.
When the default behavior is fine, just passing empty values would do. E.g., if just field numbers need to be specified, the following would do:
meta(t::Type{MyType}) = meta(t, [], [8,10], Dict())
Following is the status of protocols, transports and servers supported in the current implementation:
Protocol | Implemented as | |
---|---|---|
Binary | TBinaryProtocol | |
Compact | TCompactProtocol |
Transport | Implemented as | |
---|---|---|
Socket | TSocket and TServerSocket | |
Framed | TFramedTransport | |
SASL | TSASLClientTransport | Only client side implementation as of now |
Memory | TMemoryTransport | Can't be used with servers as of now |
File | TFileTransport | Can't be used with servers as of now |
Server | Implemented as | |
---|---|---|
Blocking. Single Task. | TSimpleServer | Single process, blocking |
Non Blocking Tasks. | TTaskServer | Single process. Asynchronous task spawned for each connection. |
Non Blocking Multi Process. | TProcessPoolServer | Multi process, non blocking. |