fraillt / bitsery Goto Github PK

View Code? Open in Web Editor NEW

1.0K 25.0 81.0 841 KB

Your binary serialization library

License: MIT License

CMake 1.44% C++ 98.12% Shell 0.09% C 0.35%

binary-serialization serialization cpp14 cpp11

bitsery's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger jaapsuter linecode ajiob djohansson areksredzki luli2020 tartanllama zhujie358 jpramosi viggeen amallia zhenqinshang q306023680 revtronix iomeone jh0x fcccode wiltonlazary ezhangle velocityra johngladp sunlive4ev bijoumd78 externalrepositories swoboderz marcos30004347 stoeckley indigendesigns geode-solutions timmitee10 net5f idfumg alexcasual lastexile-ch sszwfy binyfs alan0526 glitchedpolygons robbert-vdh wolfv michael-mueller-git museghost dr4ds raymond-sun josephtamhk ajunlonglive parleychou rooong victorstewart maketea-tech kebrahimpour qqnbhhgw classic130 songhtdo zone-organization crackercat l3nn0x gerhobbelt cloudbee7 sarvex softdrivefelix kuangtu yangjietadie yyf117 qwiff-dogg yssource saltworkermlu simonzhangsm masscollaborationlabs heitaoflower anton-kl anuragvohraec mensong fuxfork huebl oss-evaluation-repository eqk215 mezstd

bitsery's Issues

SegFault while deserializing

I am trying to upgrade my code to bitsery 5. No issue during the porting. However, I have a seg fault in my tests.

Here is the valgrind log of one of them:

valgrind ./test-vertex-identifier 
==26526== Memcheck, a memory error detector
==26526== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==26526== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==26526== Command: ./test-vertex-identifier
==26526== 
==26526== Invalid read of size 8
==26526==    at 0x5306FE2: void bitsery::ext::pointer_utils::PolyAllocWithTypeId::deallocate<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> >(std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>*, unsigned long, unsigned long) const (memory_resource.h:86)
==26526==    by 0x5306B42: bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> >::deallocate(__gnu_cxx::_Lock_policy*, unsigned long) (memory_resource.h:147)
==26526==    by 0x530680D: std::allocator_traits<bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> > >::deallocate(std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>&, __gnu_cxx::_Lock_policy*, unsigned long) (alloc_traits.h:328)
==26526==    by 0x53062AF: std::__allocated_ptr<bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> > >::~__allocated_ptr() (allocated_ptr.h:73)
==26526==    by 0x5308249: std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>::_M_destroy() (shared_ptr_base.h:476)
==26526==    by 0x723AB28: bitsery::ext::pointer_utils::StdPolyAlloc<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> > >::deallocate(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*, unsigned long) (memory_resource.h:147)
==26526==    by 0x72249B6: void bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>::addToMap<bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >(std::integral_constant<bool, false>)::{lambda(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*)#1}::operator()(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*) const (polymorphism_utils.h:150)
==26526==    by 0x72DE8B1: std::_Sp_counted_deleter<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*, void bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>::addToMap<bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >(std::integral_constant<bool, false>)::{lambda(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> > >, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:470)
==26526==    by 0x5277691: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:154)
==26526==    by 0x5277342: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:684)
==26526==    by 0x5298AA9: std::__shared_ptr<bitsery::ext::PolymorphicHandlerBase, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1123)
==26526==    by 0x5298AC5: std::shared_ptr<bitsery::ext::PolymorphicHandlerBase>::~shared_ptr() (shared_ptr.h:93)
==26526==  Address 0x8 is not stack'd, malloc'd or (recently) free'd
==26526== 
==26526== 
==26526== Process terminating with default action of signal 11 (SIGSEGV)
==26526==  Access not within mapped region at address 0x8
==26526==    at 0x5306FE2: void bitsery::ext::pointer_utils::PolyAllocWithTypeId::deallocate<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> >(std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>*, unsigned long, unsigned long) const (memory_resource.h:86)
==26526==    by 0x5306B42: bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> >::deallocate(__gnu_cxx::_Lock_policy*, unsigned long) (memory_resource.h:147)
==26526==    by 0x530680D: std::allocator_traits<bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> > >::deallocate(std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>&, __gnu_cxx::_Lock_policy*, unsigned long) (alloc_traits.h:328)
==26526==    by 0x53062AF: std::__allocated_ptr<bitsery::ext::pointer_utils::StdPolyAlloc<std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2> > >::~__allocated_ptr() (allocated_ptr.h:73)
==26526==    by 0x5308249: std::_Sp_counted_deleter<geode::VariableAttribute<unsigned int>*, bitsery::ext::smart_ptr_details::SmartPtrOwnerManager<std::shared_ptr<geode::VariableAttribute<unsigned int> > >::createSharedPolymorphic(bitsery::ext::smart_ptr_details::SharedPtrSharedState&, std::shared_ptr<geode::VariableAttribute<unsigned int> >&, bitsery::ext::MemResourceBase*, std::shared_ptr<bitsery::ext::PolymorphicHandlerBase> const&)::{lambda(geode::VariableAttribute<unsigned int>*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<geode::VariableAttribute<unsigned int> >, (__gnu_cxx::_Lock_policy)2>::_M_destroy() (shared_ptr_base.h:476)
==26526==    by 0x723AB28: bitsery::ext::pointer_utils::StdPolyAlloc<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> > >::deallocate(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*, unsigned long) (memory_resource.h:147)
==26526==    by 0x72249B6: void bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>::addToMap<bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >(std::integral_constant<bool, false>)::{lambda(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*)#1}::operator()(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*) const (polymorphism_utils.h:150)
==26526==    by 0x72DE8B1: std::_Sp_counted_deleter<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*, void bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>::addToMap<bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >(std::integral_constant<bool, false>)::{lambda(bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> >*)#1}, bitsery::ext::pointer_utils::StdPolyAlloc<bitsery::ext::PolymorphicHandler<bitsery::ext::StandardRTTI, bitsery::Deserializer<bitsery::BasicInputStreamAdapter<char, bitsery::DefaultConfig, std::char_traits<char> >, std::tuple<bitsery::ext::PolymorphicContext<bitsery::ext::StandardRTTI>, bitsery::ext::PointerLinkingContext, bitsery::ext::InheritanceContext> >, geode::VariableAttribute<unsigned int>, geode::VariableAttribute<unsigned int> > >, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:470)
==26526==    by 0x5277691: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:154)
==26526==    by 0x5277342: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:684)
==26526==    by 0x5298AA9: std::__shared_ptr<bitsery::ext::PolymorphicHandlerBase, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1123)
==26526==    by 0x5298AC5: std::shared_ptr<bitsery::ext::PolymorphicHandlerBase>::~shared_ptr() (shared_ptr.h:93)
==26526==  If you believe this happened as a result of a stack
==26526==  overflow in your program's main thread (unlikely but
==26526==  possible), you can try to increase the size of the
==26526==  main thread stack using the --main-stacksize= flag.
==26526==  The main thread stack size used in this run was 8388608.
==26526== 
==26526== HEAP SUMMARY:
==26526==     in use at exit: 23,006 bytes in 393 blocks
==26526==   total heap usage: 2,459 allocs, 2,066 frees, 199,049 bytes allocated
==26526== 
==26526== LEAK SUMMARY:
==26526==    definitely lost: 0 bytes in 0 blocks
==26526==    indirectly lost: 0 bytes in 0 blocks
==26526==      possibly lost: 0 bytes in 0 blocks
==26526==    still reachable: 23,006 bytes in 393 blocks
==26526==         suppressed: 0 bytes in 0 blocks
==26526== Rerun with --leak-check=full to see details of leaked memory
==26526== 
==26526== For counts of detected and suppressed errors, rerun with: -v
==26526== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Erreur de segmentation (core dumped)

Do you have an idea of the issue and its origin?

Make quickSerialization() usable for streams

That's because writtenBytesCount is not implemented for it, but can't it be implemented by using std::ostream::tellp? By storing the start in the constructor and then returning the difference, when writtenBytesCount() is called.

Is that possible or am I missing something?

isCompletedSuccesfully() for InputStreamAdaptor

This returns false even if all data is succesfully read

bitsery/include/bitsery/adapter/stream.h

Line 70 in d24dfe1

bool isCompletedSuccessfully() const {

My guess is because the last data read in was a container with trailing empty data that was not read in, and so the current char is not eof. I'd just have the check instead confirm that all remaining characters in the stream is blank data to the eof.

separate save/load

I am not an user yet, though the list of features is impressive and complete. I use currently boost serialization, and I think I'll be able to transition quite smoothly, except for one thing: I need in couple of places to have separate save/load in order to pre/post process some values. U mention this is possible through extensions, but I do not find neither the extension nor an example.
Can you help pls?

Serializing std::function and SG14 stdext::inplace_function objects?

According to the discussions in the comments on this answer,https://stackoverflow.com/a/22772214/5689597, it seems std::function objects should be serializable as long as they are deserialized on the same architecture!

Here's an implementation of it done: https://github.com/darabos/pinty/blob/master/pinty.h

And if so, then certainly the new SG14 function objects (https://github.com/WG21-SG14/SG14/blob/master/SG14/inplace_function.h) with explicitly defined memory buffer capacities must be able to be too?

Of course they have to be capture-less though!

I have time this week to immerse myself in the bitsery code base to suggest and draft implementations of these additions and the others I've mentioned. So if you want we can discuss and coordinate?

adapter/stream require rtti

BasicBufferedOutputStreamAdapter::flush() have dynamic_cast from basic_ios to basic_ostream.
What if BasicBufferedOutputStreamAdapter will work directly with basic_ostream?
I mean, constructor accept basic_ostream and store it as basic_ostream.

Are the serialized files cross-architecture (64/32 bits) and cross endianness? [using compact mode or flexible]

You did a fine job on this lib. It's really amazing, congratulations!

I would like to know if in flexible or compact mode, there is compatibility of serialized files between 32/64 bit architectures.

Let's say I serialize something using 64-bit architecture (compact mode), can I perform deserialization of this file in a 32-bit architecture (also using compact mode)?

Another issue is with regard to endianness, the serialized files in modes (flexible or compact) are cross endianness?

Thank you.

EDIT:

I found the information in the file containing the "motivation".

"Cross-platform compatible. if same code compiles on Android, PS3 console, and your PC either x64 or x86 architecture, you are 100% sure it works. To achieve this, bitsery specifically defines size of underlying data, hence syntax is value<2> (alias function value2b) instead or value, or container2b for element type of 16bits, eg int16_t. Bitsery also applies endianess transformation if nessesarry."

Serializing Strings and Vectors

both of these fail with these compiler errors. I guess these weren't imagined to be serialized on their own? I'm sure there's some quick tweak to make.

#include <bitsery/bitsery.h>
#include <bitsery/adapter/buffer.h>

#include <string>
#include <vector>

#include <bitsery/traits/vector.h>
#include <bitsery/traits/string.h>

using Buffer = std::vector<uint8_t>;

using InputAdapter = bitsery::InputBufferAdapter<Buffer>;
using OutputAdapter = bitsery::OutputBufferAdapter<Buffer>;

int main (int argc, char* argv[], char* env[])
{
   std::basic_string<char> token;
   Buffer tokenData;
   (void)bitsery::quickSerialization<OutputAdapter>(tokenData, token);
   (void)bitsery::quickDeserialization(InputAdapter(tokenData.begin(), tokenData.size()), token);

    
   std::vector<Buffer> buffers;
   Buffer buffersData;
   (void)bitsery::quickSerialization<OutputAdapter>(buffersData, buffers);
   (void)bitsery::quickDeserialization(InputAdapter(buffersData.begin(), buffersData.size()), buffers);
   return 1;
}

root@7342c4214c33 /root # g++ -std=gnu++17 -I/root/bitsery/include -L/usr/local/lib -lulib_g -o bitsery.test /root/test/bitsery.test.cpp
In file included from /root/bitsery/include/bitsery/serializer.h:27,
from /root/bitsery/include/bitsery/bitsery.h:40,
from /root/test/bitsery.test.cpp:1:
/root/bitsery/include/bitsery/details/serialization_common.h: In instantiation of ‘static void bitsery::details::SerializeFunction<S, T>::invoke(S&, T&) [with S = bitsery::BasicSerializer<bitsery::AdapterWriter<bitsery::OutputBufferAdapter<std::vector >, bitsery::DefaultConfig>, void>; T = std::__cxx11::basic_string]’:
/root/bitsery/include/bitsery/serializer.h:84:67: required from ‘void bitsery::BasicSerializer<TAdapterWriter, TContext>::object(const T&) [with T = std::__cxx11::basic_string; TAdapterWriter = bitsery::AdapterWriter<bitsery::OutputBufferAdapter<std::vector >, bitsery::DefaultConfig>; TContext = void]’
/root/bitsery/include/bitsery/serializer.h:427:9: required from ‘size_t bitsery::quickSerialization(Adapter, const T&) [with Adapter = bitsery::OutputBufferAdapter<std::vector >; T = std::__cxx11::basic_string; size_t = long unsigned int]’
/root/test/bitsery.test.cpp:19:69: required from here
/root/bitsery/include/bitsery/details/serialization_common.h:234:65: error: static assertion failed:
Please define 'serialize' function for your type (inside or outside of class):
template
void serialize(S& s)
{
...
}
             static_assert(HasSerializeFunction<S, T>::value || HasSerializeMethod<S, T>::value,
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/bitsery/include/bitsery/details/serialization_common.h: In instantiation of ‘static void bitsery::details::SerializeFunction<S, T>::invoke(S&, T&) [with S = bitsery::BasicDeserializer<bitsery::AdapterReader<bitsery::InputBufferAdapter<std::vector >, bitsery::DefaultConfig>, void>; T = std::__cxx11::basic_string&]’:
/root/bitsery/include/bitsery/deserializer.h:86:69: required from ‘void bitsery::BasicDeserializer<TAdapterReader, TContext>::object(T&&) [with T = std::__cxx11::basic_string&; TAdapterReader = bitsery::AdapterReader<bitsery::InputBufferAdapter<std::vector >, bitsery::DefaultConfig>; TContext = void]’
/root/bitsery/include/bitsery/deserializer.h:436:9: required from ‘std::pair<bitsery::ReaderError, bool> bitsery::quickDeserialization(Adapter, T&) [with Adapter = bitsery::InputBufferAdapter<std::vector >; T = std::__cxx11::basic_string]’
/root/test/bitsery.test.cpp:20:96: required from here
/root/bitsery/include/bitsery/details/serialization_common.h:234:65: error: static assertion failed:
Please define 'serialize' function for your type (inside or outside of class):
template
void serialize(S& s)
{
...
}

/root/bitsery/include/bitsery/details/serialization_common.h: In instantiation of ‘static void bitsery::details::SerializeFunction<S, T>::invoke(S&, T&) [with S = bitsery::BasicSerializer<bitsery::AdapterWriter<bitsery::OutputBufferAdapter<std::vector >, bitsery::DefaultConfig>, void>; T = std::vector<std::vector >]’:
/root/bitsery/include/bitsery/serializer.h:84:67: required from ‘void bitsery::BasicSerializer<TAdapterWriter, TContext>::object(const T&) [with T = std::vector<std::vector >; TAdapterWriter = bitsery::AdapterWriter<bitsery::OutputBufferAdapter<std::vector >, bitsery::DefaultConfig>; TContext = void]’
/root/bitsery/include/bitsery/serializer.h:427:9: required from ‘size_t bitsery::quickSerialization(Adapter, const T&) [with Adapter = bitsery::OutputBufferAdapter<std::vector >; T = std::vector<std::vector >; size_t = long unsigned int]’
/root/test/bitsery.test.cpp:25:73: required from here
/root/bitsery/include/bitsery/details/serialization_common.h:234:65: error: static assertion failed:
Please define 'serialize' function for your type (inside or outside of class):
template
void serialize(S& s)
{
...
}

/root/bitsery/include/bitsery/details/serialization_common.h: In instantiation of ‘static void bitsery::details::SerializeFunction<S, T>::invoke(S&, T&) [with S = bitsery::BasicDeserializer<bitsery::AdapterReader<bitsery::InputBufferAdapter<std::vector >, bitsery::DefaultConfig>, void>; T = std::vector<std::vector >&]’:
/root/bitsery/include/bitsery/deserializer.h:86:69: required from ‘void bitsery::BasicDeserializer<TAdapterReader, TContext>::object(T&&) [with T = std::vector<std::vector >&; TAdapterReader = bitsery::AdapterReader<bitsery::InputBufferAdapter<std::vector >, bitsery::DefaultConfig>; TContext = void]’
/root/bitsery/include/bitsery/deserializer.h:436:9: required from ‘std::pair<bitsery::ReaderError, bool> bitsery::quickDeserialization(Adapter, T&) [with Adapter = bitsery::InputBufferAdapter<std::vector >; T = std::vector<std::vector >]’
/root/test/bitsery.test.cpp:26:102: required from here
/root/bitsery/include/bitsery/details/serialization_common.h:234:65: error: static assertion failed:
Please define 'serialize' function for your type (inside or outside of class):
template
void serialize(S& s)
{
...
}

Compilation fails when using map。

�i just edit brief_syntax example to using std::map but compilation fails. code as following:

#include <bitsery/adapter/buffer.h>
#include <bitsery/bitsery.h>
//to use brief syntax always include this header
#include <bitsery/brief_syntax.h>
//we also need additional traits to work with container types,
//instead of including <bitsery/traits/vector.h> for vector traits, now we also need traits to work with brief_syntax types.
//so include everything from <bitsery/brief_syntax/...> instead of <bitsery/traits/...>
//otherwise we'll get static assert error, saying to define serialize function.
#include <bitsery/brief_syntax/map.h>
#include <bitsery/brief_syntax/vector.h>

enum class MyEnum : uint16_t { V1, V2, V3 };
struct MyStruct {
uint32_t i;
MyEnum e;
//std::vector fs;
std::map<std::string, int32_t> fm;

//define serialize function as usual
template <typename S>
void serialize(S& s) {
    //now we can use brief syntax with
    s(i, e, fm);
}

};

//some helper types
using Buffer = std::vector<uint8_t>;
using OutputAdapter = bitsery::OutputBufferAdapter;
using InputAdapter = bitsery::InputBufferAdapter;

int main() {
//set some random data
MyStruct data{8941, MyEnum::V2, {{"xxx", 1}, {"yyy", 2}}};
MyStruct res{};

//serialization, deserialization flow is unchanged as in basic usage
Buffer buffer;
auto writtenSize = bitsery::quickSerialization<OutputAdapter>(buffer, data);

auto state = bitsery::quickDeserialization<InputAdapter>({buffer.begin(), writtenSize}, res);

}

and the errors as following

In file included from /Users/bjarneyang/Workspace/incubator/lfe/tests/bitsery_test.cc:70:
In file included from /Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/buffer.h:26:
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../details/adapter_common.h:149:17: warning: unknown pragma ignored [-Wunknown-pragmas]
#pragma warning( disable : 4310 )
^
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../details/adapter_common.h:154:17: warning: unknown pragma ignored [-Wunknown-pragmas]
#pragma warning( default : 4310 )
^
In file included from /Users/bjarneyang/Workspace/incubator/lfe/tests/bitsery_test.cc:71:
In file included from /Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/bitsery.h:40:
In file included from /Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../details/../serializer.h:27:
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../traits/core/../../details/serialization_common.h:249:17: error: static_assert failed due to requirement
'HasSerializeFunction<bitsery::Serializer<bitsery::OutputBufferAdapter<std::__1::vector<unsigned char, std::__1::allocator >, bitsery::DefaultConfig>, void>, std::__1::basic_string, void>::value ||
HasSerializeMethod<bitsery::Serializer<bitsery::OutputBufferAdapter<std::__1::vector<unsigned char, std::__1::allocator >, bitsery::DefaultConfig>, void>, std::__1::basic_string, void>::value' "\nPlease
define 'serialize' function for your type (inside or outside of class):\n template\n void serialize(S& s)\n {\n ...\n }\n"
static_assert(HasSerializeFunction<S, T>::value || HasSerializeMethod<S, T>::value,
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../details/../serializer.h:178:56: note: in instantiation of member function
'bitsery::details::SerializeFunction<bitsery::Serializer<bitsery::OutputBufferAdapter<std::__1::vector<unsigned char, std::__1::allocator >, bitsery::DefaultConfig>, void>, std::__1::basic_string
>::invoke' requested here
details::SerializeFunction<Serializer, T>::invoke(*this, const_cast<T& >(obj));
^
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/brief_syntax/map.h:35:21: note: in instantiation of function template specialization
'bitsery::Serializer<bitsery::OutputBufferAdapter<std::__1::vector<unsigned char, std::__1::allocator >, bitsery::DefaultConfig>, void>::object<std::__1::basic_string >' requested here
s.object(key);
^
/Users/bjarneyang/Workspace/incubator/lfe/deps/bitsery/include/bitsery/adapter/../traits/core/../../details/serialization_common.h:276:17: note: in instantiation of function template specialization
'bitsery::serialize<bitsery::Serializer<bitsery::OutputBufferAdapter<std::__1::vector<unsigned char, std::__1::allocator >, bitsery::DefaultConfig>, void>, std::__1::basic_string, int,
std::__1::less<std::__1::basic_string >, std::__1::allocator<std::__1::pair<const std::__1::basic_string, int> > >' requested here
serialize(s, v);
^

std::chrono support

Specifically chrono::time_point and chrono::duration (see cereal).

const input buffer

Hey, cool Lib,
But I wanna ask, if there is a reason, why the bitsery::InputBufferAdapter only works with non const iterators?

No Heap Array support?

you have text1b(), which would be the closest thing to char array support, though generic heap arrays arent supported at all, not even an option like

s.ext(ObjectPtr, PointerOwner{PointerType::NotNull}, ObjectCount);

Serializing forward-declared types

I have a bunch of forward declares for types used in my classes and in order to use bitsery it looks like I need to include the headers for them (forward declaring doesn't work).

More of a detail:
I use forward declares because I hold raw pointers to my types, but I don't actually use bitsery's raw pointer features because I manage the memory/hierarchy myself.
Instead I serialize them dereferenced.

Anyway, is there a way to avoid having the includes? Using forward declares is better practice.

Thanks, great library :)

How to serializate and deserializate std::vector<std::any> or std::vector<std::variant> ?

[Question] What is the structure of the binary stream after serialize?

Thanks for this awesome project!

Forgive me that I haven't read the source code but documents. I have a question about the serialize result of bitsery. What's the structure of the binary stream? Could you explain it briefly?

Thank you very much! Hope you have a nice day.

Error on Centos7

I am trying to use Bitsery5.0.1 on a Centos7 witg gcc4.8.2. I got this error:

/usr/include/c++/4.8.2/bits/hashtable.h: In instantiation of 'class std::_Hashtable<long unsigned int, std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > >, bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >, std::__detail::_Select1st, std::equal_to<long unsigned int>, std::hash<long unsigned int>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >':
/usr/include/c++/4.8.2/bits/unordered_map.h:100:18:   required from 'class std::unordered_map<long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> >, std::hash<long unsigned int>, std::equal_to<long unsigned int>, bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > > >'
/__w/OpenGeode/OpenGeode/build/third_party/bitsery/install/include/bitsery/ext/utils/polymorphism_utils.h:182:19:   required from here
/usr/include/c++/4.8.2/bits/hashtable.h:194:47: error: no type named 'pointer' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
       typedef typename _Alloc::pointer        pointer;
                                               ^
/usr/include/c++/4.8.2/bits/hashtable.h:195:55: error: no type named 'const_pointer' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
       typedef typename _Alloc::const_pointer          const_pointer;
                                                       ^
/usr/include/c++/4.8.2/bits/hashtable.h:196:55: error: no type named 'reference' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
       typedef typename _Alloc::reference              reference;
                                                       ^
/usr/include/c++/4.8.2/bits/hashtable.h:197:55: error: no type named 'const_reference' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
       typedef typename _Alloc::const_reference        const_reference;
                                                       ^
/usr/include/c++/4.8.2/bits/hashtable.h:317:8: error: no class template named 'rebind' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
        _Node_allocator_type;
        ^
/usr/include/c++/4.8.2/bits/hashtable.h:319:8: error: no class template named 'rebind' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
        _Bucket_allocator_type;
        ^
/usr/include/c++/4.8.2/bits/hashtable.h:321:75: error: no class template named 'rebind' in 'class bitsery::ext::pointer_utils::StdPolyAlloc<std::pair<const long unsigned int, std::vector<long unsigned int, bitsery::ext::pointer_utils::StdPolyAlloc<long unsigned int> > > >'
       using __before_begin = __detail::_Before_begin<_Node_allocator_type>;
                                                                           ^

Do you have an idea of the origin of this error?

Stop writing on non-resizable buffer overflow

Hi, first off I wanna say I've been playing around with your library for a few days and I'm quite liking it. Keep up with the good work!

Now, I don't know if it's a bug or the behavior is intended, but serialization keeps writing on a non-resizable buffer even when the buffer limit is exceeded. I'm planning to use your library in a network application where the buffer size of network messages has a max size known at compile time but message data can vary and possibly overflow the buffer limit, in which case I'd like to both stop the serialization and know the serialization wasn't completed, which I think is how it should be for all cases of non-resizable buffers? Reading your code, I see you're simply assert()ing the distance between iterators in OutputBufferAdapter::writeInternalImpl() for non-resizable buffers. So I changed it to this:

void writeInternalImpl(const TValue *data, size_t size, std::false_type) {
    //optimization

    //assert(std::distance(_outIt + size, _end) >= 0);

    if (std::distance(_outIt + size, _end) < 0)
        throw 1; // couldn't serialize, reached non-resizable buffer limit
            
    memcpy(std::addressof(*_outIt), data, size);
    _outIt += size;
}

And it works as I expect. But it seems that you have designed your library so as to not throw exceptions, so maybe you could add an enum class WriterError (for symmetry with ReaderError) and let the user code call error() to check if serialization succeeded.

Read Serialized Buffers in Place

not sure the effort that would be required to implement this, but, given the memory layout of variables and their data in serialized buffers must (?) be deterministic, it is certainly possible to read buffers in place.

right now I use flatbuffers instead of bitsery to serialize/deserialize the information I exchange between my app and server solely because of the ability to read flatbuffers in place-- in my architecture philosophy of minimizing the usage of server resources and time by executing any work on the client possible. so even though the sum total work to serialize/deserialize with bitsery is much less than with flatbuffers, it's still the less optimal choice.

adding this ability would instantly make bitsery the absolutely optimal choice for networking!

Does it support C?

Is it has C port libraries? Thanks for any feedback!

Tonns of "unused parameter" in Release.

As title says.
Need tests in Release mode too.
PR is coming...

io_uring based stream adapter

not sure if you've heard about io_uring on Linux but it basically allows for syscall-less asynchronous operations and IO through the kernel. It's incredibly fast. I can show you a generic read/write file object i wrote to do DirectIO.

anyway this would obviously be the most performant way to stream to/from disk.

not sure how you'd feel about including a new stream adapter into the library that had a dependency on https://github.com/axboe/liburing + Linux.

even if not in the library, i assume if I wrote a new BasicBufferedOutputStreamAdapter say IOuringBufferedOutputStreamAdapter, and the same for input, and mimicked all the function calls etc, that it would "just work"?

I have a situation where I'm serializing very large (possible into the GBs) objects then writing them to disk, and don't want to blow up memory, thrash the page cache (hence DirectIO), nor waste precious seconds getting the data onto disk.

Passing a bitsery buffer to a C API

I want to pass some serialized state to a C API, but I don't really see a way of doing that without an additional copy.
By "additional copy" I mean, currently I am using a std::vector BufferWriter. The std::vector will be destroyed after my C API call returns because of RAII. I can't stop that. So I have to use malloc + copy that data somewhere else, which is wasteful.

To be clear, I am the developer of that C API. I want to expose "load" and "save" functions for some state.
So perhaps I could use an opaque pointer that holds a newed std::vector? But that sounds hacky.
Maybe there's a solution where OutputBufferAdapter can be implemented to use malloc + realloc.

Excuse my quick + maybe unclear problem description.
Also if this is not currently feasible, I understand.

Thanks.

iterator out of range

Well I didnt know that either but it seems thats iterrating an iterrator outside of the container range is UB.

An assert triggers in the readChecked function, on msvc 2017 in Debug mode.

Only verifying dynamic container size in debug.

Hey, this might be intended but I just wanted to double check.

In the dynamic container functions, assert() is being used to verify that the given container isn't too large. (Link)
This won't trigger in release builds with NDEBUG defined. I noticed it because I was getting an unused parameter warning on maxSize. Is this intended?

Deserializing a file with different PolymorphicContext

I have an issue while deserializing a file with different PolymorphicContext. I have a custom PolymorphicContext build using the registerSingleBaseBanch method, let's call it PC_A. I save a file using PC_A, say file.txt. Then, I add a new base branch into PC_A, so PC_B = PC_A + one branch. And then I try to deserialize file.txt using PC_B, I got this error:

include/bitsery/ext/utils/pointer_utils.h:95: void bitsery::ext::pointer_utils::PLCInfo::update(bitsery::ext::PointerOwnershipType): Assertion `ptrType == PointerOwnershipType::SharedOwner || ptrType == PointerOwnershipType::SharedObserver' failed.

The new branch added to the context is not used during this all process. I want to add a polymorphic class to the contexte for future usage.

Another test: saving and loading a file with the same context is working, both PC_A and PC_B.

Recreate Pointer Objects w/ Custom Allocator During Deserialization

title is pretty self explanatory :)

i have some objects i'm currently using new to allocate, but want to switch over to using a custom allocator. i serialize/deserialize these objects with bitsery, and so would need the custom allocator to be triggered upon deserialization.

maybe i missed it but i didn't see any hooks to allocate memory when deserializing a pointer.

Rename error() function

error is a really common name and if bitsery's user codebase defines it to a macro, bitsery won't compile.

Related to #48.

Feel free to close this if you think it's ridiculous :)

Dynamic polymorphic class registration

I would like to know how to dynamically extend bitsery polymorphism. Let's take your example and assume this code is in a library A:

template<>
struct PolymorphicBaseClass<Shape> : PolymorphicDerivedClasses<Circle, Rectangle> {
};

Now, in another library B, I would like to add another class Square derived from Shape.
Is it possible to register this new relationship between Square and Shape without modifying the library A which does not know Square?

buffer.h integer implicit conversion

bitsery/include/bitsery/adapter/buffer.h have "implicit conversion changes signedness" warnigns all over the place.

Mainly because you adding std::size_t to TIterator. While TIterator::operator+(diff_t).
Either make everything diff_t, either use explicit casts:

using diff_t = typename std::iterator_traits<TIterator>::difference_type;
diff_t _currOffset{0};
diff_t _bufferSize{0};
diff_t _biggestCurrentPos{0};

Also, what about run tests with:

add_compile_options(-Wall -Wextra -Wconversion -pedantic -Werror)

static_assert error in flexible.h with rvalue to serialize

In our application there were cases that requires different implementations of serial/deserialization, and my solution is to write separate “serialize” functions with the Serializer class (non-template). In the serialization function, I encountered the errors when using the flexible format and it is something like:
'
…
using StreamedSerializer =
bitsery::BasicSerializer<bitsery::AdapterWriter<OutputStreamAdapter, bitsery::DefaultConfig>, TContext>;
int get_test(const Test&t){return t.data_int;}

void serialize(StreamedSerializer &ar, const Test &t) {
ar.archive(t.data_int);//ok
ar.archive(static_cast(t.data_int));//fail
ar.archive(get_test(t));//fail
}
…
'
The failure cases is giving the following error:
“bitsery/include/bitsery/flexible.h:44:13: error: static_assert failed due to requirement 'std::is_base_of<ArchiveWrapperFnc, int>::value' "\nOnly archive behaviour modifying functions can be passed by rvalue to deserializer\n”

If I removed the assertion statement, things are actually WORKING fine. I guess the static assertion is to make sure the serialize function works for both serial/deserializing functions. However, in our code there were some complicated cases (including third party libraries) using rvalues for serializing, so I am wondering is there any better way (if not removing the assertion) to make this case work?
Thanks a lot!
Lu

std::string_view support

Is it possible to serialize/deserialize std::string_view?

Example of serializing into the middle of a buffer?

This might be obvious, but I'm having trouble figuring out how to write into the middle of a buffer. For instance, if I'm using a vector, is there a way to say "start writing at index 10" instead of starting at 0?

Thanks for the lib btw! It's been exactly what I was looking for.

Why does the buffer uses resize instead of reserve?

https://github.com/fraillt/bitsery/blob/master/include/bitsery/traits/core/std_defaults.h#L85

Wouldn't be the reserve method a better solution instead of resize?
Wasn't that possible?

Any plan for golang support?

Delta encoding

Tests show that using delta encoding on my serialized data before compressing reduces file size.

According to https://libraries.io/github/fraillt/bitsery delta encoding might have already been planned:

[bitsery] is still missing functions for delta changes and geometry compression

How would I best implement delta encoding in Bitsery?

EDIT: After separating serialize into save/load functions by not using the quickSerialization function it's now clear how to do this manually.

includes are inconsistent

Some of #include's in your library have relative path, some "absolute". Like:

#ifndef BITSERY_BRIEF_SYNTAX_TYPE_STD_ARRAY_H
#define BITSERY_BRIEF_SYNTAX_TYPE_STD_ARRAY_H

#include "../traits/array.h"
#include "bitsery/details/brief_syntax_common.h"

If relative paths would used everywhere, it would be possible to just drop in your library, like in my_library/third_party/bitsery. Without additional include paths for compiler/linker.
IOW another header only library, which uses bitsery, could just have it internally, without additional fuss for end-user.

Serializer value() size auto-deduction

Probably discussed somewhere before, but.

Serializer have

template<size_t VSIZE, typename T>
void value(const T &v)

why this is impossible:

template<typename T>
void value(const T &v)
{
    value<sizeof(T)>(v);
}

Why value size must be always specified?

Lambdas

any way to serialize lambdas? I can't really think of a way to lol.

Lots of unknown pragma warnings with GCC and Clang in bitsery 5.1.0

First of all, thanks for making bitsery! I was just about to upgrade my project to bitsery 5.1.0, but I noticed that commit 9cade41 adds a few MSVC-only pragmas. This results in a ton of unknown pragma warnings when compiling on GCC or Clang with -Wall. Would it be possible to gate those pragmas behind a compiler check?

MSVC C4310 error

msvc 17 gave C4310 error (truncating constant value) here:

        struct EndiannessTestData {
            static constexpr uint32_t _sample4Bytes = 0x01020304;
            static constexpr uint8_t _sample1stByte = (const uint8_t &) _sample4Bytes;
        };

Why you cast to reference? This will not have the same effect ?

            static constexpr uint8_t _sample1stByte = static_cast<uint8_t>(_sample4Bytes);

mishandles doubly linked lists?

struct Object {
	int32_t i = 25;
    
    Object* m_next;
    Object* m_prev;
	bool operator== (Object other){
		if(other.i == i){
			return true;
		}
		return false;
	}
};
template <typename S>
void serialize(S& s, Object& o){
    s.value4b(o.i);
    s.ext(o.m_next,PointerObserver{PointerType::Nullable});
    s.ext(o.m_prev,PointerObserver{PointerType::Nullable});
}
struct MyStruct {
  Object* ptr;
  int32_t object_count = 0;
};
template <typename S>
void serialize(S& s, MyStruct& o) {
    s.value4b(o.object_count);
    for(Object* start = o.ptr; start != nullptr; start = start->m_next){
        s.ext(start,PointerOwner{PointerType::NotNull});
    }
};

Assertion failed: (state.first == ReaderError::NoError && state.second), function main, file src/main.cpp, line 95.

std::mutex support

VS2019 compile errors

I'm using C++20 and almost the latest VS2019 version.
bitsery is the latest release (v5.0.3).

#include <bitsery/bitsery.h>

results in:

C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C3928: '->': trailing return type is not allowed after a parenthesized declarator
  C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(160): note: see reference to class template instantiation 'bitsery::details::InputAdapterBitPackingWrapper<TAdapter>' being compiled
C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C3613: missing return type after '->' ('int' assumed)
C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C3646: 'log': unknown override specifier
C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C2457: '__FUNCTION__': predefined macro cannot appear outside of a function body
C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C2096: '_logger': A data member cannot be initialized with a parenthesized initializer
C:\vcpkg\installed\x64-windows\include\bitsery\deserializer.h(114): error C2059: syntax error: 'const'

Several polymorphic hierarchies

Let's assume this code:

struct Shape {
    virtual ~Shape() = 0;
};
Shape::~Shape() = default;

struct Circle : Shape {
    int32_t radius{};
};

struct OtherShape {
    virtual ~OtherShape() = 0;
};
OtherShape::~OtherShape() = default;

struct OtherCircle : OtherShape {
    std::unique_ptr< Shape > circle{};
};

How would you serialize this?

Does it work in arduino?

serialization of 3rd party datastructures

Is it possible to serialize e.g. glm datastructures (glm::vec2, glm::vec3, ...)? The examples show only serialization of classes and struct defined by the user, and simply defining

#include <glm/vec3.hpp>

template<typename S>
void serialize(S& s, glm::vec3& vec) {
    s.value4b(vec.x);
    s.value4b(vec.y);
    s.value4b(vec.z);
}

somewhere where the serialization is done was apparently not the correct way.