An implementation of Google's Protocol Buffers for Erlang, based on ngerakines/erlang_protobuffs.
License: Apache License 2.0
This project, erlang_protobuffs, is no longer actively maintained. Please use gpb instead. gpb has erlang_protobuffs compatibility and is actively maintained by @tomas-abrahamsson
README_ORIG.md contains this project's original README.md content.
Hey,
I was wondering if this project could add a license file, so that everyone could know where it stands in using it.
One suggestion would be the Apache License 2.0, but there are others you might feel as appropriate.
Or examples which cover most of the exposed APIs
is there any plan to support protobuf3?
Hello! I don't know if there is a bug or feature.
I got files extend.proto and extensions.proto from the test/erlang_protobuffs_SUITE_data/ directory.
Then I scaned file:
protobuffs_compile:scan_file("extend.proto").Then I ran test:
extension_test() ->
Message0 = #extendable{},
{ok, Message1} = extend_pb:set_extension(Message0, bar, 25),
?assertEqual(1, extend_pb:extension_size(Message1)),
?assertEqual({ok, 25}, extend_pb:get_extension(Message1, bar)),
?assert(extend_pb:has_extension(Message1, bar)), %% <<< Why does it return false?!
ok.Test fails in last assert.
I checked extend_pb.erl. It contains:
...
has_extension(#extendable{'$extensions' = Extensions},
bar) ->
dict:is_key(bar, Extensions);
...
set_extension(#extendable{'$extensions' = Extensions} =
Record,
bar, Value) ->
NewExtends = dict:store(126,
{optional, Value, sint32, none}, Extensions),
{ok, Record#extendable{'$extensions' = NewExtends}};
...In function has_extension() it tries to search for atom key in the dictionary.
But set_extension() inserts only numeric key in dictionary.
It seems to me it's bug in the code generator.
Hi
I was trying to build a .proto with an embedded enum in it, but the compilation step fails. I noted that the provided enum.proto(https://github.com/basho/erlang_protobuffs/blob/master/test/erlang_protobuffs_SUITE_data/proto/enum.proto) file fails as well:
$ ../bin/protoc-erl enum.proto
=INFO REPORT==== 16-Mar-2013::18:58:14 ===
Writing header file to "enum_test_pb.hrl"
escript: exception error: bad argument
in function list_to_atom/1
called as list_to_atom(["EnumMsg_Values"])
in call from protobuffs_compile:atomize/1 (src/protobuffs_compile.erl, line 806)
in call from protobuffs_compile:filter_enum_to_int_clause/2 (src/protobuffs_compile.erl, line 558)
in call from protobuffs_compile:'-expand_enum_to_int_function/3-lc$^0/1-0-'/2 (src/protobuffs_compile.erl, line 554)
in call from protobuffs_compile:expand_enum_to_int_function/3 (src/protobuffs_compile.erl, line 554)
in call from protobuffs_compile:filter_forms/5 (src/protobuffs_compile.erl, line 271)
in call from protobuffs_compile:output_source/4 (src/protobuffs_compile.erl, line 170)
If I modify protobuffs_compile.erl like this, it works:
$ git diff
diff --git a/src/protobuffs_compile.erl b/src/protobuffs_compile.erl
index 9deb6aa..7ac2d49 100644
--- a/src/protobuffs_compile.erl
+++ b/src/protobuffs_compile.erl
@@ -800,9 +800,12 @@ generate_field_definitions([{Name, _, Default} | Tail], Acc) ->
generate_field_definitions(Tail, [Head | Acc]).
%% @hidden
+atomize([String]) ->
+ atomize(String);
atomize(String) ->
list_to_atom(string:to_lower(String)).
+
%% @hidden
replace_atom(Find, Find, Replace) -> Replace;
replace_atom(Tuple, Find, Replace) when is_tuple(Tuple) ->
I guess the right fix should be to remove the extra list further up the processing chain, but I haven't found the 'right' place to fix it yet.
If this is a bug at all. Can you confirm this?
Best regards,
Lars
I have 2 proto files:
package_import_target.proto:
package Exporter;
message ExportedMessage {
required int32 value = 1;
}
package_import.proto:
import "import_package_target.proto";
message ImporterMessage {
required Exporter.ExportedMessage internal = 1;
}
Running protoc on them yields no errors. However, when I try to run them through protobuffs_compile:scan_file("package_import.proto"), I get the following error:
protobuffs_compile:generate_source("import_package.proto").
** exception throw: ["Unknown Type ","Exporter.ExportedMessage"]
in function protobuffs_compile:'-resolve_types/4-fun-0-'/5 (src/protobuffs_compile.erl, line 693)
in call from lists:foldl/3 (lists.erl, line 1197)
in call from protobuffs_compile:resolve_types/4 (src/protobuffs_compile.erl, line 703)
in call from protobuffs_compile:generate_source/2 (src/protobuffs_compile.erl, line 102)
If I change package import to not use the package name, it works in erlang_protobuffs, but not with protoc:
import "import_package_target.proto";
message ImporterMessage {
required ExportedMessage internal = 1;
}
protoc:
protoc import_package.proto --python_out=.
import_package.proto:4:12: "ExportedMessage" is not defined.
decode_extensions(Types, [{Fnum, Bytes} | Tail], Acc) ->
NewAcc = case lists:keyfind(Fnum, 1, Types) of
{Fnum, Name, Type, Opts} ->
{Value1, Rest1} = case lists:member(is_record, Opts) of
true ->
{{FNum, V}, R} =
protobuffs:decode(Bytes, bytes),
RecVal = decode(Type, V),
{RecVal, R};
false ->
case lists:member(repeated_packed,
Opts)
of
true ->
{{FNum, V}, R} =
protobuffs:decode_packed(Bytes,
Type),
{V, R};
false ->
{{FNum, V}, R} =
protobuffs:decode(Bytes,
Type),
{unpack_value(V, Type), R}
end
end,
case lists:member(repeated, Opts) of
true ->
case lists:keytake(FNum, 1, Acc) of
{value, {FNum, Name, List}, Acc1} -> %% use opt 18, there is a warning on this line,use otp 17, there is no warning.
decode(Rest1, Types,
[{FNum, Name,
lists:reverse([int_to_enum(Type, Value1)
| lists:reverse(List)])}
| Acc1]);
false ->
decode(Rest1, Types,
[{FNum, Name, [int_to_enum(Type, Value1)]}
| Acc])
end;
false ->
[{Fnum,
{optional, int_to_enum(Type, Value1), Type, Opts}}
| Acc]
end;
false -> [{Fnum, Bytes} | Acc]
end,
decode_extensions(Types, Tail, NewAcc).what's your opinion?
There is import conflict when the same proto is imported more than once in the import chain.
message pbUser {
repeated pb_15.pbItem equip = 1;
}
it is ok to generate c++ code, but fail generate erlang code, it must remove pb_15. but when I remove pb_15,fail generate c++ code now.
bertd@beam-dev:~/erlang_protobuffs/ebin$ erl
Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.9.1 (abort with ^G)
1>
1> Path = "../test/erlang_protobuffs_SUITE_data/proto/simple.proto".
"../test/erlang_protobuffs_SUITE_data/proto/simple.proto"
2> protobuffs_compile:scan_file(Path).
=INFO REPORT==== 28-Jun-2012::17:20:55 ===
Writing header file to "simple_pb.hrl"
=INFO REPORT==== 28-Jun-2012::17:20:55 ===
Writing beam file to "simple_pb.beam"
ok
4> rr(".hrl").
[location,person]
5> simple_pb:encode_person({person, <<"Nick">>, <<"Mountain View">>, <<"+1 (000) 555-1234">>,25}).
* exception error: no function clause matching simple_pb:encode_person({person,<<"Nick">>,<<"Mountain View">>,
<<"+1 (000) 555-1234">>,25}) (src/simple_pb.erl, line 37)
Is there some way to see the generated source code ?
Is it possible to use binary instead of list in decode result?
i.e. decode into {person, <<"John Doe">>} instead of {person, "John Doe"}
The example in the documentation shows decode result of binary, but actually it decodes into lists.
When trying to build Riak from source on Debian Sid problems occured when compiling protobuffs.
$ make
==> Entering directory /home/avtobiff/src/riak/deps/protobuffs/deps/meck' ==> meck (get-deps) ==> Leaving directory/home/avtobiff/src/riak/deps/protobuffs/deps/meck'
==> protobuffs (get-deps)
==> Entering directory /home/avtobiff/src/riak/deps/protobuffs/deps/meck' ==> meck (compile) ==> Leaving directory/home/avtobiff/src/riak/deps/protobuffs/deps/meck'
==> protobuffs (compile)
make: *** [compile] Error 1
Installed erlang packages.
$ dpkg -l | grep erlang-
ii erlang-asn1 1:15.b.1-dfsg-3 amd64 Erlang/OTP modules for ASN.1 support
ii erlang-base 1:15.b.1-dfsg-3 amd64 Erlang/OTP virtual machine and base applications
ii erlang-crypto 1:15.b.1-dfsg-3 amd64 Erlang/OTP cryptographic modules
ii erlang-dev 1:15.b.1-dfsg-3 amd64 Erlang/OTP development libraries and headers
ii erlang-eunit 1:15.b.1-dfsg-3 amd64 Erlang/OTP module for unit testing
ii erlang-inets 1:15.b.1-dfsg-3 amd64 Erlang/OTP Internet clients and servers
ii erlang-mnesia 1:15.b.1-dfsg-3 amd64 Erlang/OTP distributed relational/object hybrid database
ii erlang-public-key 1:15.b.1-dfsg-3 amd64 Erlang/OTP public key infrastructure
ii erlang-runtime-tools 1:15.b.1-dfsg-3 amd64 Erlang/OTP runtime tracing/debugging tools
ii erlang-ssl 1:15.b.1-dfsg-3 amd64 Erlang/OTP implementation of SSL
ii erlang-syntax-tools 1:15.b.1-dfsg-3 amd64 Erlang/OTP modules for handling abstract Erlang syntax trees
ii erlang-tools 1:15.b.1-dfsg-3 amd64 Erlang/OTP various tools
ii erlang-webtool 1:15.b.1-dfsg-3 amd64 Erlang/OTP helper for web-based tools
For example, if I defined something like this in protocol:
messaage Enum {
enum Type {
Type1 = 1;
Type2 = 2;
}
required Type type = 1;
}
How do I use erlang code to get 'Type1' is 1 here?
below function is not working please give solution to me
protobuffs_compile:scan_file("simple.proto").
** exception exit: scanning_error
in function protobuffs_compile:parse_file/2 (protobuffs_compile.erl, line 193)
in call from protobuffs_compile:parse_file/1 (protobuffs_compile.erl, line 183)
in call from protobuffs_compile:scan_file/2 (protobuffs_compile.erl, line 59)
module 'protobuffs_proper'
module 'erlang_protobuffs_SUITE'
module 'protobuffs'
module 'protobuffs_tests'
protobuffs_tests: proper_specs_test...*failed*
in function protobuffs_tests:'-proper_specs_test/0-fun-0-'/1 (test/protobuffs_tests.erl, line 44)
**error:{assertEqual,[{module,protobuffs_tests},
{line,44},
{expression,"( proper : check_specs ( protobuffs , [ long_result ] ) )"},
{expected,[]},
{value,[{{protobuffs,read_field_num_and_wire_type,1},[[<<>>]]},
{{protobuffs,encode_packed,3},[[0,[-1],bool]]},
{{protobuffs,decode_value,3},[[<<>>,0,bool]]},
{{protobuffs,encode,3},[[0,2,bool]]},
{{protobuffs,next_field_num,1},[[<<6>>]]},
{{protobuffs,decode_packed,2},[[<<>>|...]]},
{{protobuffs,decode,...},[[...]]},
{{protobuffs,...},[...]},
{{...},...},
{...}]}]}
output:<<"Testing protobuffs:read_field_num_and_wire_type/1
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,read_field_num_and_wire_type,
[<<>>],
[{file,"src/protobuffs.erl"},{line,215}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<>>]
Shrinking (0 time(s))
[<<>>]
Testing protobuffs:encode_packed/3
.!
Failed: After 2 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,encode_internal,
[1,0.2586897728450852,sint32],
[{file,"src/protobuffs.erl"},{line,175}]},
{protobuffs,encode_packed_internal,3,
[{file,"src/protobuffs.erl"},{line,186}]},
{protobuffs,encode_packed,3,
[{file,"src/protobuffs.erl"},{line,87}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[1,[0.2586897728450852],sint32]
Shrinking ....(4 time(s))
[0,[-1],bool]
Testing protobuffs:decode_value/3
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,decode_varint,
[<<>>,11,7],
[{file,"src/protobuffs.erl"},{line,426}]},
{protobuffs,decode_value,3,
[{file,"src/protobuffs.erl"},{line,329}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<139>>,2,string]
Shrinking ...(3 time(s))
[<<>>,0,bool]
Testing protobuffs:encode/3
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,encode_internal,
[0,[],double],
[{file,"src/protobuffs.erl"},{line,175}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[0,[],double]
Shrinking ...(3 time(s))
[0,2,bool]
Testing protobuffs:next_field_num/1
...!
Failed: After 4 test(s).
An exception was raised: error:{throw,{error,[69,114,114,111,114,32,100,101,99,111,100,105,110,103,32,116,121,112,101]}}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,read_field_num_and_wire_type,1,
[{file,"src/protobuffs.erl"},{line,212}]},
{protobuffs,next_field_num,1,
[{file,"src/protobuffs.erl"},{line,251}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<103,194>>]
Shrinking ..(2 time(s))
[<<6>>]
Testing protobuffs:decode_packed/2
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,decode_varint,
[<<>>,6,7],
[{file,"src/protobuffs.erl"},{line,426}]},
{protobuffs,read_field_num_and_wire_type,1,
[{file,"src/protobuffs.erl"},{line,196}]},
{protobuffs,decode_packed,2,
[{file,"src/protobuffs.erl"},{line,235}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<134>>,int64]
Shrinking ..(2 time(s))
[<<>>,bool]
Testing protobuffs:decode/2
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,read_field_num_and_wire_type,
[<<>>],
[{file,"src/protobuffs.erl"},{line,215}]},
{protobuffs,decode,2,
[{file,"src/protobuffs.erl"},{line,224}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<>>,sint64]
Shrinking .(1 time(s))
[<<>>,bool]
Testing protobuffs:decode_varint/1
!
Failed: After 1 test(s).
An exception was raised: error:{error,badarg}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,decode_varint,
[<<>>,30,7],
[{file,"src/protobuffs.erl"},{line,426}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<158>>]
Shrinking .(1 time(s))
[<<>>]
Testing protobuffs:encode_varint/1
!
Failed: After 1 test(s).
[-1]
Shrinking (0 time(s))
[-1]
Testing protobuffs:skip_next_field/1
.!
Failed: After 2 test(s).
An exception was raised: error:{throw,{error,[69,114,114,111,114,32,100,101,99,111,100,105,110,103,32,116,121,112,101]}}.
Stacktrace: [{proper_typeserver,'-apply_spec_test/5-fun-0-',
[{protobuffs,read_field_num_and_wire_type,1,
[{file,"src/protobuffs.erl"},{line,212}]},
{protobuffs,skip_next_field,1,
[{file,"src/protobuffs.erl"},{line,261}]},
{proper_typeserver,'-apply_spec_test/5-fun-0-',6,
[{file,"src/proper_typeserver.erl"},{line,533}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}],
[{file,"src/proper_typeserver.erl"},{line,556}]},
{proper,apply_args,3,[{file,"src/proper.erl"},{line,1353}]},
{proper,child,3,[{file,"src/proper.erl"},{line,1392}]},
{proper,'-spawn_link_migrate/1-fun-1-',2,
[{file,"src/proper.erl"},{line,669}]}].
[<<23>>]
Shrinking .(1 time(s))
[<<6>>]
">>
There's a usr.proto:
message Msg {
required int32 id = 1;
extensions 100 to max;
}
message Usr {
extend Msg { optional Usr msg = 1000; }
required int64 id = 1;
}
I can encode Usr:
UsrBin = usr_pb:encode_usr(#usr{id = 1})
and decode Usr:
UsrMsg = usr_pb:decode_usr(list_to_binary(UsrBin))
Here's the problem:
Now I want to encode Msg, so I create a dict:
Extensions = dict:append(1000, UsrMsg, dict:new()),
then I encode Msg:
MsgBin = usr_pb:encode_msg(#msg{id = 1, '$extensions' = Extensions}),
and decode Msg:
MsgMsg = usr_pb:decode_msg(list_to_binary(MsgBin)),
However, I can't get the extensions:
usr_pb:get_extension(MsgMsg, 1000)(this returned "undefined")
What was wrong? I thought I was coding it the wrong way, plz help me out!
Create message with more that optional field. At encode process don't set optional field and you receive something like this:
exception error: {badrecord,person}
in function person_pb:iolist/2 (src/person_pb.erl, line 48)
in call from person_pb:encode/2 (src/person_pb.erl, line 40
Is it normal?
I fixed this and submitted a pull request ages ago, however, looks like it got re-introduced. If I have a chance I'll fix it again and attach a pull request again, but figured I'd put the issue out there.
In Java, ProtocolBuffers can not parse the message which have multiply extensions without calling "registerAllExtensions()" first. I wonder what's the equivalent method in doing the same thing in erlang_protobuffs. Thx!
When I was using utf8 with protobuffs, I noticed that protobuffs decodes string typed fields to lists, but the readme of erlang_protobuffs says "binary string" as:
1> protobuffs_compile:scan_file("simple.proto").
ok
2> simple_pb:decode_person(<<10,4,78,105,99,107,18,13,77,111,...>>).
{person,<<"Nick">>,<<"Mountain View">>, <<"+1 (000) 555-1234">>,25,undefined}
Why is that? How can I decode as binary string?
I got error when I firstly used erlang_protobuffs with my rebar project.
a part of rebar.config:
%%-------------------------------------------------
{deps_dir, "deps"}.
{deps, [
{protobuffs, ".*", {git, "https://github.com/basho/erlang_protobuffs.git"}}
]}.
{proto_opts, [
{compiler, protobuffs},
{src_dirs, ["proto"]}
]}.
%%-------------------------------------------------
rebar get-deps
rebar compile generate
rel/server/bin/server console
([email protected])1> protobuffs:encode(1, 1, uint32).
** exception error: undefined function protobuffs:encode/3
It would be highly useful to add the same return/return_warnings/return_errors/report/report_warnings/report_errors/warnings_as_errors options as available in compile:file and leex:file for erlang_protobuffs' own errors/warnings.
I create a copy of simple.proto and change type of last field to string(my simple2.proto):
package simple2;
message Person2 {
required string name = 1;
required string address = 2;
required string phone_number = 3;
required int32 age = 4;
optional string location = 5;
}
message Location2
{
required string region = 1;
required string country = 2;
}
(a@localhost)18> A = iolist_to_binary(simple2_pb:encode_person2({person2, <<"a">>, <<"b">>, <<"c">>, 5, undefined})).
<< 10, 1, 97, 18, 1, 98, 26, 1, 99, 32, 5 >>
And now try to decode this value as simple.proto message and as simple2.proto:
(a@localhost)19> simple_pb:decode_person(A).
{person,"a","b","c",5}
(a@localhost)20> simple2_pb:decode_person2(A).
age = 5,location = undefined}
Could you explain me why result is different?
As noted in this thread on the erlang-questions mailing list:
http://erlang.org/pipermail/erlang-questions/2012-February/064685.html
... the README's text about the code quality could use an update.
I have download the latest version, I compiled all the erl files in src folder, and copy simple.proto to src folder, when I excute "protobuffs_compile:scan_file("simple.proto")." in the erl console, it shows error as follows:
protobuffs_compile:scan_file("simple.proto").
** exception exit: scanning_error
why ? Howerver it runs ok if I user the pre-compiled beam files in the ebin folder.
another question : How to use nested messages in the .proto file?
Dear Bashoers, the proto's optional fields generates valid hrl record's fields with default values, but the required - not.
example
message Constants
{
optional string name0 = 1 [ default = "theName" ];
required string name1 = 2 [ default = "theName" ];
}
generates the follows code
-record(constants, {
name0 = "theName",
name1 = erlang:error({required, name1})
}).
in file protobuffs_compile.erl, function scan_string/3 at line 68 around, there is a bug for ImportPaths variate:
ImportPaths = ["./", "src/" | proplists:get_value(imports_dir, Options, [])],
because erlang language will treat string as a list, so list operation will flatten list, for example, the code following:
Options = [{imports_dir, "dir/"}],
ImportPaths = ["./", "src/" | proplists:get_value(imports_dir, Options, [])]
the ImportPaths is ["./","src/",100,105,114,47], while we expect the result is ["./","src/", "dir/"]
so, I think the right code maybe :
ImportPaths =
case proplists:get_value(imports_dir, Options) of
undefined -> ["./", "src/"];
ImportDir -> [ImportDir, "./", "src/"]
end,
is there anything i am wrong?
I've got this error:
3> protobuffs_compile:scan_file("realtime-bidding.proto").
** exception throw: ["Unknown Type ","Hyperlocal.Point"]
in function protobuffs_compile:'-resolve_types/4-fun-0-'/5 (src/protobuffs_compile.erl, line 753)
in call from lists:foldl/3 (lists.erl, line 1197)
in call from protobuffs_compile:resolve_types/4 (src/protobuffs_compile.erl, line 763)
in call from protobuffs_compile:scan_string/3 (src/protobuffs_compile.erl, line 73)
Meanwhile the same .proto file compiles ok into c++ lango using protoc compiler provided as a debian package here on my local system.
I have a little understanding of protobuf yet and can't answer why does this happen, is this expected behaviour, and how can I fix it or workaround it. Please help.
Hi, I'm newbie in erlang, and i have problem with protobuffs.
I'm build application release with "./rebar compile && ./rebar generate".
run application from "release/hamster/bin/hamster console".(bin/hamser generated by rebar)
in opened erl shell -
protobuffs_compile:scan_file(code:priv_dir(hamster) ++ "/hamster.proto").
and have error
=INFO REPORT==== 10-Dec-2012::19:05:22 ===
Writing header file to "hamster_pb.hrl"
** exception error: no match of right hand side value
{error,
[{[],
[{none,compile,
{crash,v3_life,
{{case_clause,
{'EXIT',
{undef,
[{v3_life,module,
[{k_mdef,[],hamster_pb,[{...}|...],[],...},
[binary,return_errors,return_warnings]],
[]},
{compile,'-select_passes/2-anonymous-2-',2,
[{file,"compile.erl"},{line,473}]},
{compile,'-internal_comp/4-anonymous-1-',2,
[{file,[...]},{line,...}]},
{compile,fold_comp,3,[{file,...},{...}]},
{compile,internal_comp,4,[{...}|...]},
{compile,'-do_compile/2-anonymous-0-',2,
[...]}]}}},
[{compile,'-select_passes/2-anonymous-2-',2,
[{file,"compile.erl"},{line,473}]},
{compile,'-internal_comp/4-anonymous-1-',2,
[{file,"compile.erl"},{line,273}]},
{compile,fold_comp,3,
[{file,"compile.erl"},{line,291}]},
{compile,internal_comp,4,
[{file,"compile.erl"},{line,275}]},
{compile,'-do_compile/2-anonymous-0-',2,
[{file,"compile.erl"},{line,152}]}]}}}]}],
[]}
in function protobuffs_compile:output/4 (src/protobuffs_compile.erl, line 147)
hamster_pb.hrl was created in release/hamster/, but .beam not created.
if I use protobuffs_compile:scan_file from normal erl shell(not in application), all is fine, hamster_pb.hrl and hamster_pb.beam created in /ehamster/deps/protobuffs/ebin
our project has a proto define as follows:
message A1 {
optional uint32 a = 1;
}
message A2 {
optional uint32 b = 1;
optional string c = 2;
}
message C {
optional A1 a1 = 1;
}
其中
record C 正确的情况下包含了一个A1， 但是一次程序误操作，将C里面的A1保存成了A2， 这两个在某些时候会互相转换。 然后在存db的时候，encode成功，但是下次启动服务器后，decode一直失败。
当然实际结构比这个复杂，查了很久，发现是存的地方写混淆了。 不过如果pb在encode的时候能做严格的检查 就会更早的发现这个问题了。
Is this intended and permanent? Here's a quick example:
$ cat test.proto
message testproto {
optional string id = 1;
}
$ protoc-erl test.proto
=INFO REPORT==== 13-Jun-2013::14:54:52 ===
Writing header file to "test_pb.hrl"
=INFO REPORT==== 13-Jun-2013::14:54:52 ===
Writing src file to "test_pb.erl"
Now in the erlang shell:
7> rr("test_pb.hrl").
[testproto]
8> c("test_pb").
{ok,test_pb}
9> Record1 = #testproto { id = "123" }.
10> Encoded1 = test_pb:encode_testproto(Record1).
[[["\n",[3],<<"123">>]]]
11> Decoded1 = test_pb:decode_testproto(Encoded1).
** exception error: no function clause matching test_pb:decode_testproto([[["\n",[3],<<"123">>]]]) (src/test_pb.erl, line 74)
12> Decoded1 = test_pb:decode_testproto(list_to_binary(Encoded1)).
current version was error during run example!
3> simple_pb:encode_person({person, <<"Nick">>, <<"Mountain View">>,
<<"+1 (000) 555-1234">>,25}).
[[["\n",[4],<<"Nick">>],
[[18],"\r",<<"Mountain View">>],
[[26],[17],<<"+1 (000) 555-1234">>],
[" ",[25]]]]
not
<<10,4,78,105,99,107,18,13,77,111,117,110,116,97,105,110,
32,86,105,101,119,26,17,43,49,32,40,48,48,...>>
We need the fix that includes the following line in rebar.config
{erl_opts, [debug_info]}.
It is on aptakhin's branch. Without the fix we get errors like this:
{"init terminating in do_boot",{{badmatch,{ok,{pokemon_pb,[{abstract_code,no_abstract_code}]}}},[{protobuffs_compile,output_source,4,[{file,"src/protobuffs_compile.erl"},{line,143}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
Compare this code to gpb.
If gpb has advantages over this library this project could be deprecated and replaced by gpb
Dear Bashoers,
We have noticed that there is a lot of dialyzer related work going on in your codebase at the moment. Cool.
To aid dialyzer we would very much like the type information that is present in .proto message definitions to transition over to the erlang world as erlang types.
I imagine the easiest place to start would be to make the types available in the protocol buffer record definitions.
Is this on your radar? We would rather avoid duplicating your work if this is something you are already working on. If it is not something you are working on, is this something you would like a pull request on?
Thanks!
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.