Giter Club home page Giter Club logo

hoard's Introduction

Emery Berger

Professor, Computer Science, Univ. of Massachusetts Amherst. I am also an Amazon Scholar at Amazon Web Services. My students, collaborators, and I in the PLASMA lab build innovative and impactful systems, especially focused on performance, reliability, and security. Among other things, I am the creator/maintainer of https://CSrankings.org. For the academic year 2023/2024, I am on sabbatical.

Recent blog posts

Current PhD Students

Links

Selected software from me and my collaborators (for a full list, see PLASMA)

  • Scalene: A state-of-the-art CPU+GPU+memory profiler for Python, with AI-powered optimization suggestions
    PyPI Latest Release Anaconda-Server Badge Downloads Downloads GitHub stars
  • Coz: A causal profiler that tells you where to optimize your code (C/C++/Rust/Swift/Java)
    GitHub stars Rust crate downloads (just counting Rust crate downloads)
  • CSrankings: Ranks the world's CS departments
    GitHub stars
  • Slipcover: Nearly zero-overhead code coverage analysis for Python
    PyPI Latest Release Downloads GitHub stars
  • ChatDBG: AI-assisted debugging in lldb, gdb, and pdb. Uses AI to answer 'why'
    PyPI Latest Release Downloads Downloads GitHub stars
  • Commentator: Automatically comments Python code, adding docstrings and type annotations
    PyPI Latest Release Downloads Downloads GitHub stars
  • CWhy: ("See why") Explains and suggests fixes for C/C++/Rust compiler error messages.
    PyPI Latest Release Downloads Downloads GitHub stars
  • SQLwrite: SQLite3 with natural language to SQL, recommends indexes to optimize queries
    GitHub stars

Selected Talks

Performance Matters (Strange Loop 2019) views
The second most popular Strange Loop video of all time!
Performance Matters (Strange Loop 2019)

Python Performance Matters (Strange Loop 2022)
Python Performance Matters

Scalene (PyCon 2021) views
Scalene (PyCon 2021)

How to Get Your Research Adopted (PLDI keynote) views
PLDI 2022 keynote

Selected Papers

Bio

Emery Berger is a Professor in the Manning College of Information and Computer Sciences at the University of Massachusetts Amherst, the flagship campus of the UMass system; he is also an Amazon Scholar. He graduated with a Ph.D. in Computer Science from the University of Texas at Austin in 2002. Professor Berger has been a Visiting Scientist at Microsoft Research and at the Universitat Politècnica de Catalunya (UPC) / Barcelona Supercomputing Center (BSC).

Professor Berger’s research spans programming languages, runtime systems, and operating systems, with a particular focus on systems that transparently improve reliability, security, and performance. He and his collaborators have created a number of influential software systems including Hoard, a fast and scalable memory manager that accelerates multithreaded applications (used by companies including British Telecom, Cisco, Crédit Suisse, Reuters, Royal Bank of Canada, SAP, and Tata, and on which the Mac OS X memory manager is based); DieHard, an error-avoiding memory manager that directly influenced the design of the Windows 7 Fault-Tolerant Heap; and DieHarder, a secure memory manager that was an inspiration for hardening changes made to the Windows 8 heap (see this map of the landscape of memory management research for an overview). He also created and maintains the widely-used CSrankings website.

His honors include a Microsoft Research Fellowship, an NSF CAREER Award, a Lilly Teaching Fellowship, the Distinguished Artifact Award for PLDI 2014, the Most Influential Paper Award at OOPSLA 2012, the Most Influential Paper Award at PLDI 2016, the ASPLOS 2019 Influential Paper Award, five SIGPLAN Research Highlights, five CACM Research Highlights (1, 2, 3, 4, 5), a Google Research Award, a Microsoft SEIF Award, and Best Paper Awards at FAST, OOPSLA, and SOSP. Professor Berger served two terms (1, 2) as an elected member of the SIGPLAN Executive Committee; he served for a decade (2007-2017) as Associate Editor of the ACM Transactions on Programming Languages and Systems, was Program Chair for PLDI 2016, and served as co-Program Chair of ASPLOS 2021. He was named an ACM Fellow in 2019.

hoard's People

Contributors

alexmipego avatar barracuda156 avatar bertwesarg avatar bpowers avatar devnexen avatar emeryberger avatar francoisvn avatar gabrielganne avatar jserv avatar michalbiesek avatar tdaniely avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hoard's Issues

Compilation issues on Solaris SunOS 5.10

Hoard 3.12 is not compiling on SunOS 5.10 with compiler version CC: Sun C++ 5.8

Errors:

CC -dalign -xbuiltin=%all -fast -xO5 -DNDEBUG -mt -g -xildoff -xthreadvar=dynamic -L/usr/lib/lwp -R/usr/lib/lwp -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -G -PIC source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/wrapper.cpp Heap-Layers/wrappers/arch-specific/sparc-interchange.il -o libhoard_32.so -lthread -ldl -lCrun
source/libhoard.cpp:
"include/superblocks/tlab.h", line 90: Error: Cannot use HL::SLList::Entry* to initialize int_.
"source/libhoard.cpp", line 131: Where: While instantiating "Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>::malloc(unsigned)".
"source/libhoard.cpp", line 131: Where: Instantiated from non-template code.
"include/superblocks/tlab.h", line 102: Error: Cannot use void_ to initialize int_.
"source/libhoard.cpp", line 131: Where: While instantiating "Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>::malloc(unsigned)".
"source/libhoard.cpp", line 131: Where: Instantiated from non-template code.
2 Error(s) detected.
source/unixtls.cpp:
"source/unixtls.cpp", line 82: Error: "," expected instead of "attribute".
"source/unixtls.cpp", line 83: Error: "," expected instead of "attribute".
"source/unixtls.cpp", line 87: Error: "{" expected instead of "attribute".
"source/unixtls.cpp", line 87: Error: constructor is not defined.
"source/unixtls.cpp", line 89: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 99: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 100: Error: Cannot return bool from a function that should return Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536, Hoard::HoardHeapType>.
"source/unixtls.cpp", line 103: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 185: Error: Linkage specifications are allowed only at file level.
"source/unixtls.cpp", line 185: Error: Identifier expected instead of "{".
"source/unixtls.cpp", line 185: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 200: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 215: Error: Linkage specifications are allowed only at file level.
"source/unixtls.cpp", line 215: Error: Identifier expected instead of "{".
"source/unixtls.cpp", line 215: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 216: Error: "inline" is not allowed here.
"source/unixtls.cpp", line 216: Error: Use ";" to terminate declarations.
"source/unixtls.cpp", line 219: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 220: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 220: Error: a is not defined.
"source/unixtls.cpp", line 220: Error: Cannot use int to initialize int
.
"source/unixtls.cpp", line 222: Error: threadFunctionType is not defined.
"source/unixtls.cpp", line 222: Error: int is not a structure type.
"source/unixtls.cpp", line 223: Error: int is not a structure type.
"source/unixtls.cpp", line 226: Error: The operation "_ int" is illegal.
Compilation aborted, too many Error messages.
Heap-Layers/wrappers/wrapper.cpp:
make: *** [SunOS-sunw-sparc] Error 3

Compilation fails on Mac OS X 10.8.4

At commit 7e8647e, "make macos" fails with numerous errors. Some are in Heap-Layers, some not. Transcript below. You may well be aware of all this :)

I downloaded the 3.9 tarball and it built (with many warnings). It would be nice if there were Git tags for the releases, assuming you have the history in Git.

Thanks!

$ uname -a
Darwin silverbird.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
$ clang --version
Apple clang version 4.0 (tags/Apple/clang-421.0.60) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin12.4.0
Thread model: posix
$ git clone --recursive git://github.com/emeryberger/Hoard.git
$ cd Hoard/src
$ make macos
clang++ -arch i386 -arch x86_64 -pipe -g -O3 -Wall -DNDEBUG -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -compatibility_version 1 -current_version 1 -dynamiclib -D'CUSTOM_PREFIX(x)=xx##x' source/libhoard.cpp Heap-Layers/wrappers/macwrapper.cpp source/mactls.cpp -o libhoard.dylib -ldl -lpthread 
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:51:9: error: use of undeclared
      identifier 'assert'
        assert (SuperHeap::getSize(ptr) >= sizeof(dict));
        ^
Heap-Layers/heaps/combining/strictsegheap.h:102:50: note: in instantiation of
      member function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
        ptr = SuperHeap::myLittleHeap[sizeClass].malloc (realSize);
                                                 ^
include/hoard/thresholdsegheap.h:42:31: note: in instantiation of member
      function 'HL::StrictSegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
      void * ptr = SuperHeap::malloc (sz);
                              ^
Heap-Layers/heaps/threads/lockedheap.h:43:21: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::malloc' requested here
      return Super::malloc (sz);
                    ^
Heap-Layers/heaps/threads/threadheap.h:66:28: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::malloc' requested here
      return getHeap(tid)->malloc (sz);
                           ^
Heap-Layers/heaps/combining/hybridheap.h:82:17: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::malloc' requested here
      return bm.malloc (sz);
                ^
Heap-Layers/heaps/combining/hybridheap.h:57:15: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::slowPath' requested here
        ptr = slowPath (sz);
              ^
Heap-Layers/wrappers/ansiwrapper.h:52:31: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::malloc' requested here
      void * ptr = SuperHeap::malloc (sz);
                              ^
include/superblocks/tlab.h:102:33: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::malloc' requested here
      void * ptr = _parentHeap->malloc (sz);
                                ^
source/libhoard.cpp:116:21: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::malloc' requested here
    void * ptr = h->malloc (sz);
                    ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:59:9: error: use of undeclared
      identifier 'assert'
        assert (SuperHeap::getSize(ptr) >= sizeof(dict));
        ^
Heap-Layers/heaps/combining/strictsegheap.h:127:50: note: in instantiation of
      member function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
        SuperHeap::myLittleHeap[objectSizeClass].free (ptr);
                                                 ^
include/hoard/thresholdsegheap.h:55:18: note: in instantiation of member
      function 'HL::StrictSegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      SuperHeap::free (ptr);
                 ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:6:
In file included from Heap-Layers/heaps/combining/all.h:2:
Heap-Layers/heaps/combining/segheap.h:189:15: error: no member named 'clear' in
      'Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>'
      bigheap.clear();
      ~~~~~~~ ^
include/hoard/thresholdsegheap.h:63:15: note: in instantiation of member
      function 'HL::SegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
          SuperHeap::clear();
                     ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:103:
In file included from Heap-Layers/heaps/all.h:1:
In file included from Heap-Layers/heaps/buildingblock/all.h:1:
Heap-Layers/heaps/buildingblock/adaptheap.h:81:18: error: no member named
      'clear' in 'Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType,
      65536, Hoard::BigHeap>, 65536, Hoard::MmapSource>'
      SuperHeap::clear();
                 ^
Heap-Layers/heaps/combining/segheap.h:184:25: note: in instantiation of member
      function 'HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
        myLittleHeap[i].clear();
                        ^
include/hoard/thresholdsegheap.h:63:15: note: in instantiation of member
      function 'HL::SegHeap<80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::clear' requested here
          SuperHeap::clear();
                     ^
Heap-Layers/heaps/threads/lockedheap.h:48:14: note: in instantiation of member
      function 'Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource> >::free' requested here
      Super::free (ptr);
             ^
Heap-Layers/heaps/threads/threadheap.h:73:21: note: in instantiation of member
      function 'HL::LockedHeap<HL::MacLockType, Hoard::ThresholdSegHeap<20,
      65536, 80, size2class, class2size, HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>> >::free' requested here
      getHeap(tid)->free (ptr);
                    ^
Heap-Layers/heaps/combining/hybridheap.h:68:12: note: in instantiation of member
      function 'HL::ThreadHeap<64, HL::LockedHeap<HL::MacLockType,
      Hoard::ThresholdSegHeap<20, 65536, 80, size2class, class2size,
      HL::AdaptHeap<HL::DLList,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>,
      Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::MacLockType, 65536,
      Hoard::BigHeap>, 65536, Hoard::MmapSource>>> >::free' requested here
        bm.free (ptr);
           ^
include/superblocks/ignoreinvalidfree.h:49:13: note: in instantiation of member
      function 'HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128,
      Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::free' requested here
        SuperHeap::free (ptr);
                   ^
Heap-Layers/wrappers/ansiwrapper.h:59:19: note: in instantiation of member
      function 'Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap> >::free' requested here
        SuperHeap::free (ptr);
                   ^
include/superblocks/tlab.h:132:23: note: in instantiation of member function
      'HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192,
      Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>,
      Hoard::BigHeap>> >::free' requested here
          _parentHeap->free (ptr);
                       ^
source/libhoard.cpp:121:22: note: in instantiation of member function
      'Hoard::ThreadLocalAllocationBuffer<11, getSizeClass, getClassSize, 256,
      262144, Hoard::HoardSuperblock<HL::MacLockType, 65536, Hoard::SmallHeap>,
      65536, Hoard::HoardHeapType>::free' requested here
    getCustomHeap()->free (ptr);
                     ^
4 errors generated.
make: *** [macos] Error 1

Fix SIGSEGV

3.12 is just failing when LD_PRELOADed on Ubuntu Linux 16.04 LTS — on literally any executables' run.

Is Hoard alive at all? I understand that tcmalloc outperforms it a bit (and is under active development for sure) but it'd be great to know the status for sure as well.

compilation fails for linux-gcc-x86 or -64

Heap-Layers/utility/align.h:17:49: warning: unused variable ‘isPowerOfTwo’ [-Wunused-variable]
Heap-Layers/utility/align.h: In instantiation of ‘size_t HL::align(size_t) [with long unsigned int Alignment = 65536ul; size_t = long unsigned int]’:
include/util/alignedmmap.h:83:7: required from ‘void* Hoard::AlignedMmapInstance<Alignment_>::malloc(size_t) [with long unsigned int Alignment_ = 65536ul; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>; size_t = long unsigned int]’
include/util/exactlyoneheap.h:42:34: required from ‘void* Hoard::ExactlyOneHeap::malloc(size_t) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >; size_t = long unsigned int]’
include/superblocks/alignedsuperblockheap.h:54:70: required from ‘void* Hoard::SuperblockStore<SuperblockSize, TheLockType, MmapSource>::malloc(size_t) [with long unsigned int SuperblockSize = 65536ul; TheLockType = HL::SpinLockType; MmapSource = Hoard::MmapSource; size_t = long unsigned int]’
include/util/fixedrequestheap.h:43:44: required from ‘void* Hoard::FixedRequestHeap<RequestSize, SuperHeap>::malloc(size_t) [with long unsigned int RequestSize = 65536ul; SuperHeap = Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource>; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::FixedRequestHeap<65536ul, Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource> >; size_t = long unsigned int]’
include/hoard/hoardmanager.h:354:49: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::getAnotherSuperblock(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:298:4: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::slowPathMalloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:93:2: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::malloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardheap.h:171:31: required from here
Heap-Layers/utility/align.h:17:49: warning: unused variable ‘isPowerOfTwo’ [-Wunused-variable]
In file included from Heap-Layers/heaps/top/mmapheap.h:49:0,
from Heap-Layers/heaps/debug/sanitycheckheap.h:18,
from Heap-Layers/heaps/debug/all.h:4,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from include/hoard/hoardheap.h:31,
from include/hoard/hoardtlab.h:39,
from source/unixtls.cpp:56:
Heap-Layers/utility/myhashmap.h: In instantiation of ‘Value HL::MyHashMap<Key, Value, Allocator>::get(Key) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
include/util/alignedmmap.h:127:44: required from ‘void Hoard::AlignedMmapInstance<Alignment_>::free(void_) [with long unsigned int Alignment_ = 65536ul]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>]’
include/util/exactlyoneheap.h:45:7: required from ‘void Hoard::ExactlyOneHeap::free(void_) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >]’
include/superblocks/addheaderheap.h:85:7: required from ‘void Hoard::AddHeaderHeap<SuperblockType, SuperblockSize, SuperHeap>::free(void_) [with SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>; long unsigned int SuperblockSize = 65536ul; SuperHeap = Hoard::MmapSource]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>, 65536ul, Hoard::MmapSource>]’
Heap-Layers/heaps/combining/hybridheap.h:69:2: required from ‘void HL::HybridHeap<BigSize, SmallHeap, BigHeap>::free(void_) [with int BigSize = 8192; SmallHeap = Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>; BigHeap = Hoard::BigHeap]’
include/superblocks/ignoreinvalidfree.h:49:2: required from ‘void Hoard::IgnoreInvalidFree::free(void_) [with SuperHeap = HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>]’
Heap-Layers/wrappers/ansiwrapper.h:54:8: required from ‘void HL::ANSIWrapper::free(void_) [with SuperHeap = Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> >]’
include/superblocks/tlab.h:145:4: required from ‘void Hoard::ThreadLocalAllocationBuffer<NumBins, getSizeClass, getClassSize, LargestObject, LocalHeapThreshold, SuperblockType, SuperblockSize, ParentHeap>::clear() [with int NumBins = 11; unsigned int (_ getSizeClass)(size_t) = HL::bins<Header, 65536>::getSizeClass<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; size_t (* getClassSize)(unsigned int) = HL::bins<Header, 65536>::getClassSize<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; unsigned int LargestObject = 256u; unsigned int LocalHeapThreshold = 262144u; SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; unsigned int SuperblockSize = 65536u; ParentHeap = Hoard::HoardHeapType]’
source/unixtls.cpp:172:15: required from here
Heap-Layers/utility/myhashmap.h:68:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::erase(Key) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
include/util/alignedmmap.h:136:7: required from ‘void Hoard::AlignedMmapInstance<Alignment_>::free(void_) [with long unsigned int Alignment_ = 65536ul]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>]’
include/util/exactlyoneheap.h:45:7: required from ‘void Hoard::ExactlyOneHeap::free(void_) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >]’
include/superblocks/addheaderheap.h:85:7: required from ‘void Hoard::AddHeaderHeap<SuperblockType, SuperblockSize, SuperHeap>::free(void_) [with SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>; long unsigned int SuperblockSize = 65536ul; SuperHeap = Hoard::MmapSource]’
Heap-Layers/heaps/threads/lockedheap.h:47:7: required from ‘void HL::LockedHeap<LockType, Super>::free(void_) [with LockType = HL::SpinLockType; Super = Hoard::AddHeaderHeap<Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::BigHeap>, 65536ul, Hoard::MmapSource>]’
Heap-Layers/heaps/combining/hybridheap.h:69:2: required from ‘void HL::HybridHeap<BigSize, SmallHeap, BigHeap>::free(void_) [with int BigSize = 8192; SmallHeap = Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>; BigHeap = Hoard::BigHeap]’
include/superblocks/ignoreinvalidfree.h:49:2: required from ‘void Hoard::IgnoreInvalidFree::free(void_) [with SuperHeap = HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>]’
Heap-Layers/wrappers/ansiwrapper.h:54:8: required from ‘void HL::ANSIWrapper::free(void_) [with SuperHeap = Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> >]’
include/superblocks/tlab.h:145:4: required from ‘void Hoard::ThreadLocalAllocationBuffer<NumBins, getSizeClass, getClassSize, LargestObject, LocalHeapThreshold, SuperblockType, SuperblockSize, ParentHeap>::clear() [with int NumBins = 11; unsigned int (_ getSizeClass)(size_t) = HL::bins<Header, 65536>::getSizeClass<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; size_t (* getClassSize)(unsigned int) = HL::bins<Header, 65536>::getClassSize<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap> >; unsigned int LargestObject = 256u; unsigned int LocalHeapThreshold = 262144u; SuperblockType = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; unsigned int SuperblockSize = 65536u; ParentHeap = Hoard::HoardHeapType]’
source/unixtls.cpp:172:15: required from here
Heap-Layers/utility/myhashmap.h:81:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::insert(Key, Value) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’:
Heap-Layers/utility/myhashmap.h:64:7: required from ‘void HL::MyHashMap<Key, Value, Allocator>::set(Key, Value) [with Key = void_; Value = long unsigned int; Allocator = Hoard::AlignedMmapInstance<65536ul>::SourceHeap]’
include/util/alignedmmap.h:73:2: required from ‘void* Hoard::AlignedMmapInstance<Alignment_>::malloc(size_t) [with long unsigned int Alignment_ = 65536ul; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::AlignedMmapInstance<65536ul>; size_t = long unsigned int]’
include/util/exactlyoneheap.h:42:34: required from ‘void* Hoard::ExactlyOneHeap::malloc(size_t) [with Heap = HL::LockedHeap<HL::SpinLockType, Hoard::AlignedMmapInstance<65536ul> >; size_t = long unsigned int]’
include/superblocks/alignedsuperblockheap.h:54:70: required from ‘void* Hoard::SuperblockStore<SuperblockSize, TheLockType, MmapSource>::malloc(size_t) [with long unsigned int SuperblockSize = 65536ul; TheLockType = HL::SpinLockType; MmapSource = Hoard::MmapSource; size_t = long unsigned int]’
include/util/fixedrequestheap.h:43:44: required from ‘void* Hoard::FixedRequestHeap<RequestSize, SuperHeap>::malloc(size_t) [with long unsigned int RequestSize = 65536ul; SuperHeap = Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource>; size_t = long unsigned int]’
Heap-Layers/heaps/threads/lockedheap.h:42:31: required from ‘void* HL::LockedHeap<LockType, Super>::malloc(size_t) [with LockType = HL::SpinLockType; Super = Hoard::FixedRequestHeap<65536ul, Hoard::SuperblockStore<65536ul, HL::SpinLockType, Hoard::MmapSource> >; size_t = long unsigned int]’
include/hoard/hoardmanager.h:354:49: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::getAnotherSuperblock(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:298:4: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::slowPathMalloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardmanager.h:93:2: required from ‘void* Hoard::HoardManager<SourceHeap, ParentHeap, SuperblockType_, EmptinessClasses, LockType, thresholdFunctionClass, HeapType>::malloc(size_t) [with SourceHeap = Hoard::AlignedSuperblockHeap<HL::SpinLockType, 65536ul, Hoard::MmapSource>; ParentHeap = Hoard::GlobalHeap<65536ul, 8, Hoard::MmapSource, HL::SpinLockType>; SuperblockType_ = Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>; int EmptinessClasses = 8; LockType = HL::SpinLockType; thresholdFunctionClass = Hoard::hoardThresholdFunctionClass; HeapType = Hoard::SmallHeap; size_t = long unsigned int]’
include/hoard/hoardheap.h:171:31: required from here
Heap-Layers/utility/myhashmap.h:107:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/wrappers/gnuwrapper.cpp: In function ‘void my_init_hook()’:
Heap-Layers/wrappers/gnuwrapper.cpp:79:23: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:80:21: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:81:24: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:82:25: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:85:5: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:86:5: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:87:5: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:88:5: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
make: *** [linux-gcc-x86-64] Error 1

Compilation fails for win32

Tried compiling commit 5996bcf (current master) for Windows (VC2008) and it failed on hoardsuperblockheader.h line 287 with an error about it not being legal to use "this" there.

Fix is to assign sizeof(*this) to a local const size_t variable; this can then be used in the sassert template without errors.

Use a specific commit of Heap-Layers

Looks like Hoard's Makefile and GNUmakefile clone master of Heap-Layers. Unless you never change Heap-Layers, it seems likely that at some point you will introduce a change to Heap-Layers that will require a corresponding change to Hoard. When that happens, old versions of Hoard (which might be present in package management systems) will no longer be able to build. This problem might not always be noticed right away, leaving your software in broken state in that package management system for an unbounded period of time.

For this reason, you should download a specific version of Heap-Layers that is known to work with this version of Hoard.

I think git already behaved that way before when you were using submodules, so you could get this behavior back by reverting 5275686 and also making the same change to GNUmakefile.

Linking in windows does not seem to work

Program: Executable / with /MD flag / x64 / Release / VC10
Code: Compiled from source / x64 / Release. / VC10

In the binary project I've updated the linker options to link to the appropriate usewinhoard.obj and winhoard.lib. I've placed the winhoard.dll next to the executable. I've validated that it is required during runtime.
The runtime on my program is not improving and I know that there is alot of room for improvement as I have verified this using other allocators. The dll does get hooked, I have checked this by std::cout-ing some indicators.
Do I need to explicitly call hoard_malloc? As I read it Hoard should hijack any new, malloc calls. If that is the case how does it work in the presence of another pair of allocation hooks (this is what my code has at the moment). Do I need to take those out for hoard's hooks to kick in?

Thanks.

Runtime issues with Visual Studio 2015

I tried to build libhoard on windows with VS2015 using nmake. It built successfully. But when I try to run a sample program using hoard, It crashes in the following call stack :

ntdll.dll!RtlReportCriticalFailure()    Unknown
ntdll.dll!RtlpReportHeapFailure()   Unknown
ntdll.dll!RtlpHeapHandleError() Unknown
ntdll.dll!RtlpLogHeapFailure()  Unknown
ntdll.dll!NtdllpFreeStringRoutine() Unknown
ntdll.dll!RtlFreeUnicodeString()    Unknown
ntdll.dll!LdrpSnapThunk()   Unknown
ntdll.dll!LdrGetProcedureAddressEx()    Unknown
ntdll.dll!LdrGetProcedureAddress()  Unknown
KernelBase.dll!GetProcAddress() Unknown
libhoard.dll!PatchMe() Line 594 C++
libhoard.dll!DllMain(void * hinstDLL, unsigned long fdwReason, void * lpreserved) Line 122

It's a simple program to allocate memory :

int main()
{
    char *str = NULL;
    int rc = ERROR;

    printf("Allocating a buffer...\n");
    str = (char*)malloc(MAX_BUF_SIZE*sizeof(char));

    if (str == NULL)
    {
        printf("Allocation of buffer failed !!! Exiting... \n");
        return ERROR;
    }

    strncpy(str, "Hello", 6);
    printf("Entered string is : %s\n", str);
    return OK;
}

It's a visual studio project where I added source/uselibhoard.cpp as additional source file and libhoard.lib in the linker options. Please check the attached screenshots.

1
2
3
error
errorcode

Nmake error

\Hoard\src>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

GNUmakefile Heap-Layers include Makefile README.md source test test.sh
This Makefile is for Windows only. For other systems, use gmake.
GNUmakefile Heap-Layers include Makefile README.md source test test.sh
        git submodule update --init --checkout --recursive
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE" /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console /OUT:libhoard.dll
libhoard.cpp
\Hoard\src\Heap-Layers\heaplayers.h(41): fatal error C1083: Cannot open include file: 'assert.h': No such file or directory
winwrapper.cpp
Heap-Layers\wrappers\winwrapper.cpp(38): fatal error C1083: Cannot open include file: 'windows.h': No such file or directory
wintls.cpp
source\wintls.cpp(37): fatal error C1083: Cannot open include file: 'new': No such file or directory
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.EXE"' : return code '0x2'
Stop.

Not very sure where to go with this. Any help would be appreciated.
Another question while I'm at it how effective is hoard with games?

Allow macOS architectures to be configured

GNUmakefile hardcodes the architectures for which to compile on macOS in the MACOS_COMPILE and MACOS_COMPILE_DEBUG variables. You should make this configurable via another makefile variable. Currently you're hardcoding the architectures i386 and x86_64, but Apple is phasing out 32-bit support and you can't compile for i386 anymore as of the macOS 10.14 SDK.

hoard revers result

I could not compile hoard from source in fedora 17 x64 ( #5 I compiled it with compat gcc 34 and produced libhoard.so. After that tried to compile and test cache-scratch with and without hoard.so the result was weird as below.could you tell me where is the problem?.I also download libhoard.so for 64bit llinux from <a href=http://plasma.cs.umass.edu/emery/index.php?page=download-hoard>Emery but the result was the same.

g++ /home/src/Hoard/benchmarks/cache-scratch/cache-scratch.cpp -I../common/ /home/javad/src/Hoard/libhoard.so -o cache-scratch-hoard -lpthread

./cache-scratch 1 1000 1 1000000
Time elapsed = 2.383032 seconds.
./cache-scratch-hoard 1 1000 1 1000000
Time elapsed = 10.085448 seconds.

FreeBSD 10.1 loaded issue

Hello,

I just install hoard on FreeBSD 10.1 when I set libhoard. Any idea what I'm doing wrong?

root@localhost:~ # ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x800824000)
libncurses.so.8 => /lib/libncurses.so.8 (0x800a36000)
libc.so.7 => /lib/libc.so.7 (0x800c83000)
root@localhost:~ # setenv LD_PRELOAD /usr/local/lib/libhoard.so.1
root@localhost:~ # ldd /bin/ls
/bin/ls:
libutil.so.9 => /lib/libutil.so.9 (0x800a83000)
libncurses.so.8 => /lib/libncurses.so.8 (0x800c95000)
libc.so.7 => /lib/libc.so.7 (0x800ee2000)
libc++.so.1 => /usr/lib/libc++.so.1 (0x80128b000)
libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x80154b000)
libm.so.5 => /lib/libm.so.5 (0x801767000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x80198f000)
libthr.so.3 => /lib/libthr.so.3 (0x801b9d000)
root@localhost:~ #

Poor performance with large objects in benchmark

It's possible that this is a quirk or unrealism of the particular benchmark design rather than a failing in hoard as such, but I tried running it through the t-test1.c test case from the ptmalloc family and the results were not good. (Tests run on Windows 7 high-end quad-core machine; each size change was run as a separate process.)

It performed reasonably well (better than the OS allocator and a ptmalloc-derivative allocator) with allocation sizes up to 8192, but when using size parameters of 16384 and above it performed abysmally. (eg. 0.3s at 8192 size, 22s at 16384 size; note that the system allocator scored 1.1s for hoard's 22s, and this difference only got worse as the size increased.)

There was also significantly higher CPU usage (to the point of pegging all processors to 100%, making the system unusable) when running those particular test values.

errors on ArchLinux and python3.9

Hello

I am trying to use scalene on some code.

I cannot post the whole code, but here is what it does, at least:

  • creating threads, processes, using the 'forkserver' start method, on linux, and multiprocessing.get_context()
  • overriding threading.Thread.run
  • creating multiprocessing,queues

I tried to create a simple reproducer code, but I couldn't find what causes the error.

When my code creates new processes, I get these errors:

Exception ignored in: <function _after_fork at 0x559984ec9880>
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 1496, in _after_fork
    thread._reset_internal_locks(False)
  File "/usr/lib/python3.9/threading.py", line 827, in _reset_internal_locks
    self._started._at_fork_reinit()
  File "/usr/lib/python3.9/threading.py", line 527, in _at_fork_reinit
    self._cond._at_fork_reinit()
  File "/usr/lib/python3.9/threading.py", line 253, in _at_fork_reinit
    self._lock._at_fork_reinit()
AttributeError: 'ReplacementLock' object has no attribute '_at_fork_reinit'

Is there a way to get more details about this exception?

Thank you

Crashes with Large Superblock Sizes

Hoard crashes when SUPERBLOCK_SIZE is #define'd with values larger than 256M.

We think it's during program initialization. Here's the backtrace:

Hoard::ThreadLocalAllocationBuffer<15, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288ul>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288ul>::getClassSize, 256ul, 2097152ul, Hoard::HoardSuperblock<HL::SpinLockType, 524288, Hoard::SmallHeap, Hoard::HoardSuperblockHeader>, 524288u, Hoard::HoardHeapType>::free(Hoard::ThreadLocalAllocationBuffer<15, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 524288, Hoard::SmallHeap>, 524288>::getClassSize, 256, 2097152, Hoard::HoardSuperblock<HL::SpinLockType, 524288, Hoard::SmallHeap, HoardSuperblockHeader>, 524288, Hoard::HoardHeapType> * this, void * ptr) (/home/.../include/superblocks/tlab.h:114)
__GI__IO_setb(_IO_FILE * f, char * b, char * eb, int a) (/build/glibc-yWQXbR/glibc-2.24/libio/genops.c:382)
_IO_new_file_close_it(_IO_FILE * fp) (/build/glibc-yWQXbR/glibc-2.24/libio/fileops.c:193)
_IO_new_fclose(_IO_FILE * fp) (/build/glibc-yWQXbR/glibc-2.24/libio/iofclose.c:58)
[Unknown/Just-In-Time compiled code] (Unknown Source:0)
ld-linux-x86-64.so.2!call_init(struct link_map * l, int argc, char ** argv, char ** env) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:72)
ld-linux-x86-64.so.2!call_init(char ** env, char ** argv, int argc, struct link_map * l) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:30)
ld-linux-x86-64.so.2!_dl_init(struct link_map * main_map, int argc, char ** argv, char ** env) (/build/glibc-yWQXbR/glibc-2.24/elf/dl-init.c:120)
ld-linux-x86-64.so.2!_dl_start_user (Unknown Source:0)
[Unknown/Just-In-Time compiled code] (Unknown Source:0)

Thanks in advance!

Support DESTDIR

You currently install your library with e.g.

cp libhoard.dylib $(PREFIX)

This assumes $(PREFIX) is writable. It might not be in the context of a package management system that wants to stage the files in a temporary location first.

You should support the standard $(DESTDIR) variable for this, by changing these copy commands to e.g.

cp libhoard.dylib $(DESTDIR)$(PREFIX)

Crashes everything on ubuntu 18.04

I did the following, not sure why but this seems to break everything on Ubuntu 18.04

git clone https://github.com/emeryberger/Hoard
make
echo "export LD_PRELOAD=$PWD/libhoard.so" >> ~/.bashrc

Opening up a new terminal and trying to do anything like ls, vim, chmod, etc. causes a seg fault

Sys Info Below

uname -a
Linux 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-27ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04) 

Last Commit on master
de06b8b

This is the same as #49 but I could not re-open that issue

Issues replacing malloc() on Windows

I'm evaluating the possibility to replace malloc() for LLVM.
I hit some problems I'd like to point out here:
The README is wrong:

c:\Users\fluttershy\work\Hoard\src>nmake windows

Microsoft (R) Program Maintenance Utility Version 14.00.23918.0
Copyright (C) Microsoft Corporation.  All rights reserved.

NMAKE : fatal error U1073: don't know how to make 'windows'
Stop.

The correct command seems to be just nmake:

c:\Users\fluttershy\work\Hoard\src>nmake

Microsoft (R) Program Maintenance Utility Version 14.00.23918.0
Copyright (C) Microsoft Corporation.  All rights reserved.

*****
This Makefile is for Windows only. For other systems, use gmake.
*****
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /I
Heap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE"
 /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /
Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layer
s\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console
 /OUT:libhoard.dll
libhoard.cpp
winwrapper.cpp
wintls.cpp
   Creating library libhoard.lib and object libhoard.exp
Generating code
Finished generating code
        cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /I
Heap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE"
 /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /
Zc:wchar_t /Zc:forScope /Gd /errorReport:queue /c "source\uselibhoard.cpp"
uselibhoard.cpp

The malloc() replacement example doesn't quite work, in basic cases:

$ cat patatino.cpp
#include <stdlib.h>

int main (void) {
        malloc(16);
        return (0);
}

If I try to run the way it's described on the website:

c:\Users\fluttershy\work\Hoard\src>cl /Ox /MD patatino.cpp source\libhoard.cpp libhoard.lib
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23918 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

patatino.cpp
libhoard.cpp
source\libhoard.cpp(33): fatal error C1083: Cannot open include file: 'heaplayer
s.h': No such file or directory
Generating Code...

It seems the issue is that it's not able to locate the headers, something like this might fix:

diff --git a/src/source/libhoard.cpp b/src/source/libhoard.cpp
index 54d3713..cb39efc 100644
--- a/src/source/libhoard.cpp
+++ b/src/source/libhoard.cpp
@@ -30,7 +30,7 @@
  * @author Emery Berger <http://www.cs.umass.edu/~emery>
  */

-#include "heaplayers.h"
+#include "../Heap-Layers/heaplayers.h"
 using namespace HL;

 #include <new>

But immediately after I hit another problem:

c:\users\fluttershy\work\hoard\src\heap-layers\heaps\combining\hybridheap.h(32):
 fatal error C1083: Cannot open include file: 'heaplayers.h': No such file or di
rectory

Any ideas on how to fix? I think the instructions on the website need some love.

Can`t build Hoard library on Windows

I clone Horad by usage the link:

https://github.com/emeryberger/Hoard.git

Then as descibed in documentation I change directory to

C:\hoard\src

Then I try to build it:

C:\hoard\src> nmake

And receive error:

git clone https://github.com/emeryberger/Heap-Layers
fatal: destination path 'Heap-Layers' already exists and is not an empty directory.

Its bug - the Heap-Layers already exist when I clone Hoard repo. Ok, I delete Heap-Layers folder and try to build again:

C:\hoard\src> nmake

And receive followed error:

source\libhoard.cpp(34): fatal error C1083: stdalign.h: No such file or directory

So at now the sources can not be compiled on the Windows platform. Could you fix it?

Use -install_name on macOS

Your library is being built without the install_name being set, which means that it cannot be used unless the user manually fixes the install_name later using install_name_tool.

You have instructions about setting DYLD_INSERT_LIBRARIES to the path where the library is, which would be unnecessary if you set the install_name to the place where the library will be installed.

You could do this by adding -install_name $(PREFIX)/libhoard.dylib to MACOS_COMPILE and MACOS_COMPILE_DEBUG in GNUmakefile.

difficulties building from src, linux x86_64 gcc 4.6.3

hitting an error and some warnings attempting to build

  • libhoard-3.9.tar.gz
  • Ubuntu Linux 12.04 (x86_64)
  • g++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
  • make linux-gcc-x86-64
    Relevant tail of build output below. Suggestions?

...
heaplayers/wrappers/gnuwrapper.cpp:68:42: error: conflicting declaration 'void (* __malloc_initialize_hook)()'
/usr/include/malloc.h:170:38: error: '__malloc_initialize_hook' has a previous declaration as 'void (* volatile __malloc_initialize_hook)()'

heaplayers/wrappers/gnuwrapper.cpp: In function 'void my_init_hook()':
heaplayers/wrappers/gnuwrapper.cpp:72:23: warning: '__malloc_hook' is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:73:21: warning: '__free_hook' is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:74:24: warning: '__realloc_hook' is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:75:25: warning: '__memalign_hook' is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:78:5: warning: '__malloc_hook' is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:79:5: warning: '__free_hook' is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:80:5: warning: '__realloc_hook' is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
heaplayers/wrappers/gnuwrapper.cpp:81:5: warning: '__memalign_hook' is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]

Crashing on Windows while doing make_shared for a shared_ptr

Am using Hoard in my application to run on both Linux and Windows. While it runs perfectly fine on Linux on Windows after some allocations it crashes at a std::make_shared for an object. Its gets a "Exception thrown at 0x00007FF6E568FCDE in : Access violation writing location 0x00007FF6E5778078". Does it require any specific configuration for Windows? I have followed the exact instructions to link to libhoard.dll and also my application loads it too but after some random number of allocations it crashes the application. The application works fine without Hoard. Am using Visual Studio 2015 community version on Windows 10 host.

why performance decline?

when i compile this program, type "make" instruction, so i change "-std=c++14" to "-std=c++11".
then i export envorment instr,
OK. when i run the program, find the performance decline ? So How to solve this problem?

Turn off banner in STDERR

I don't really find it any useful or even common for allocators to report they're being put in use, what's the logical reasoning behind it?

Does not compile on VS-2010

I am in the process of reviewing this library for use in a multi-threaded utility that makes extensive use of the STL. When I attempt to compile the source using Visual Studio 2010 I get the following error from the command line:

cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /Zi /nologo /W1 /WX- /Ox /Oi /Oy- /GL /D "_WINRT_DLL" /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_UNICODE" /D "UNICODE" /D "WINAPI_FAMILY=2" /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /link /DLL /subsystem:console /OUT:libhoard.dll

libhoard.cpp
c:\work\hoard\src\heap-layers\heaps/special/zoneheap.h(53) : error C2247: 'Alignment' not accessible because 'HL::MmapHeap' uses 'private' to inherit from 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(79) : see declaration of 'Alignment'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(140) : see declaration of 'HL::MmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(73) : see declaration of 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/buildingblock/freelistheap.h(49) : see reference to class template instantiation 'HL::ZoneHeap<SuperHeap,ChunkSize>' being compiled
with
[
SuperHeap=HL::MmapHeap,
ChunkSize=16384
]
c:\work\hoard\src\heap-layers\wrappers/stlallocator.h(63) : see reference to class template instantiation 'HL::FreelistHeap' being compiled
with
[
SuperHeap=HL::ZoneHeapHL::MmapHeap,16384
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(26) : see reference to class template instantiation 'HL::STLAllocator<T,Super>' being compiled
with
[
T=HL::SanityCheckHeap::objType,
Super=HL::SanityCheckHeap::heapType
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(451) : see reference to class template instantiation 'std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,_Mfl>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator,
_Mfl=false
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(520) : see reference to class template instantiation 'std::_Tree_nod<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(659) : see reference to class template instantiation 'std::_Tree_val<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(81) : see reference to class template instantiation 'std::_Tree<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(110) : see reference to class template instantiation 'std::map<_Kty,_Ty,_Pr,_Alloc>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(111) : see reference to class template instantiation 'HL::SanityCheckHeap' being compiled
c:\work\hoard\src\heap-layers\heaps\threads\threadspecificheap.h(34) : fatal error C1189: #error : "This functionality currently is not implemented for Windows."

winwrapper.cpp
wintls.cpp
c:\work\hoard\src\heap-layers\heaps/special/zoneheap.h(53) : error C2247: 'Alignment' not accessible because 'HL::MmapHeap' uses 'private' to inherit from 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(79) : see declaration of 'Alignment'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(140) : see declaration of 'HL::MmapHeap'
c:\work\hoard\src\heap-layers\heaps/top/mmapheap.h(73) : see declaration of 'HL::PrivateMmapHeap'
c:\work\hoard\src\heap-layers\heaps/buildingblock/freelistheap.h(49) : see reference to class template instantiation 'HL::ZoneHeap<SuperHeap,ChunkSize>' being compiled
with
[
SuperHeap=HL::MmapHeap,
ChunkSize=16384
]
c:\work\hoard\src\heap-layers\wrappers/stlallocator.h(63) : see reference to class template instantiation 'HL::FreelistHeap' being compiled
with
[
SuperHeap=HL::ZoneHeapHL::MmapHeap,16384
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(26) : see reference to class template instantiation 'HL::STLAllocator<T,Super>' being compiled
with
[
T=HL::SanityCheckHeap::objType,
Super=HL::SanityCheckHeap::heapType
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(451) : see reference to class template instantiation 'std::_Tmap_traits<_Kty,_Ty,_Pr,_Alloc,_Mfl>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator,
_Mfl=false
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(520) : see reference to class template instantiation 'std::_Tree_nod<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\xtree(659) : see reference to class template instantiation 'std::_Tree_val<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\Program Files\Microsoft Visual Studio 10.0\VC\INCLUDE\map(81) : see reference to class template instantiation 'std::_Tree<_Traits>' being compiled
with
[
_Traits=std::_Tmap_traits<void *,size_t,HL::SanityCheckHeap::localComparator,HL::SanityCheckHeap::localAllocator,false>
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(110) : see reference to class template instantiation 'std::map<_Kty,_Ty,_Pr,_Alloc>' being compiled
with
[
_Kty=void *,
_Ty=size_t,
_Pr=HL::SanityCheckHeap::localComparator,
_Alloc=HL::SanityCheckHeap::localAllocator
]
c:\work\hoard\src\heap-layers\heaps\debug\sanitycheckheap.h(111) : see reference to class template instantiation 'HL::SanityCheckHeap' being compiled
c:\work\hoard\src\heap-layers\heaps\threads\threadspecificheap.h(34) : fatal error C1189: #error : "This functionality currently is not implemented for Windows."

I would like to directly compile the library into the program to eliminate the need to distribute another file if necessary, but I cannot even built the library as retrieved from the source. What am I missing? Thanks.

Add caching for large objects to reduce mmap/munmap pressure.

Sample code for timing from Phong Vo:

#include    <errno.h>
#include    <stdlib.h>
#include    <string.h>
#include    <pthread.h>

#define N_THREAD    256
#define N_ALLOC     1000000
static int      Nthread = N_THREAD;
static int      Nalloc = N_ALLOC;
static size_t       Minsize = 10;
static size_t       Maxsize = 1024;

int error(char* mesg)
{
    write(2, mesg, strlen(mesg));
    exit(1);
}

void* allocate(void* arg)
{
    int     k, p, q, c;
    size_t      sz, nalloc, len;
    char        **list;
    size_t      *size;
    int     thread = (int)((long)arg);

    unsigned int    rand = 0; /* use a local RNG so that threads work uniformly */
#define FNV_PRIME   ((1<<24) + (1<<8) + 0x93)
#define FNV_OFFSET  2166136261
#define RANDOM()    (rand = rand*FNV_PRIME + FNV_OFFSET)

    nalloc = Nalloc/Nthread; /* do the same amount of work regardless of #threads */

    if(!(list = (char**)malloc(nalloc*sizeof(char*))) )
        error("failed to allocate space for list of objects\n");
    if(!(size = (size_t*)malloc(nalloc*sizeof(size_t))) )
        error("failed to allocate space for list of sizes\n");
    memset(list, 0, nalloc*sizeof(char*));
    memset(size, 0, nalloc*sizeof(size_t));

    for(k = 0; k < nalloc; ++k)
    {   
        /* get a random size favoring smaller over larger */
        len = Maxsize-Minsize+1;
        for(;;)
        {   sz = RANDOM() % len; /* pick a random size in [0,len-1] */
            if((RANDOM()%100) >= (100*sz)/len) /* this favors a smaller size */
                break;
            len = sz; /* the gods want a smaller length, try again */
        }
        sz += Minsize;

        if(!(list[k] = malloc(sz)) )
            error("malloc failed\n");
        else
        {   size[k] = sz;
            for(c = 0; c < 10; ++c)
                list[k][c*sz/10] = 'm';
        }

        if(k < 1000)
            continue;

        /* get an interval to check for free and realloc */
        if((p = RANDOM()%k) > (q = RANDOM()%k) )
            { c = p; p = q; q = c; }

        for(; p <= q; ++p)
        {   if(list[p])
            {   if(RANDOM()%2 == 0 ) /* 50% chance of being freed */
                {   free(list[p]);
                    list[p] = 0;
                    size[p] = 0;
                }
                else if(RANDOM()%4 == 0 ) /* survived free, check realloc */
                {   sz = size[p] > Maxsize ? size[p]/4 : 2*size[p];
                    if(!(list[p] = realloc(list[p], sz)) )
                        error("realloc failed\n");
                    else
                    {   size[p] = sz;
                        for(c = 0; c < 10; ++c)
                            list[p][c*sz/10] = 'r';
                    }
                }
            }
        }
    }

    free(list);
    free(size);

    return (void*)0;
}

int main(int argc, char* argv[])
{
    int     i, rv;
    void        *status;
    pthread_t   th[N_THREAD];

    for(; argc > 1; --argc, ++argv)
    {   if(argv[1][0] != '-')
            continue;
        else if(argv[1][1] == 'a') /* # malloc calls */
            Nalloc = atoi(argv[1]+2);
        else if(argv[1][1] == 't') /* # threads */
            Nthread = atoi(argv[1]+2);
        else if(argv[1][1] == 'z') /* min block size */
            Minsize = atoi(argv[1]+2);
        else if(argv[1][1] == 'Z') /* max block size */
            Maxsize = atoi(argv[1]+2);
    }

    if(Nalloc <= 0 || Nalloc > N_ALLOC)
        Nalloc = N_ALLOC;
    if(Nthread <= 0 || Nthread > N_THREAD)
        Nthread = N_THREAD;
    if(Minsize <= 0)
        Minsize = 1;
    if(Maxsize < Minsize)
        Maxsize = Minsize;

    for(i = 0; i < Nthread; ++i)
    {   if((rv = pthread_create(&th[i], NULL, allocate, (void*)((long)i))) != 0 )
            error("Failed to create thread\n");
    }

    for(i = 0; i < Nthread; ++i)
    {   if((rv = pthread_join(th[i], &status)) != 0 )
            error("Failed waiting for thread\n");
    }

    return 0;
}

Unused pages are not released

Hello,

I am using your allocator in a memory intensive application. If the application allocates and deallocates many small objects, then not enough free space for a big memory block remains. It seems that the allocator doesn't free any unused page, but this should be part of the task of your blowup prevention.

Test system:
Windows 7 x64
Visual Studio 2005
32bit application
Hoard 3.8

test application:

#include "hoard-38/src/libhoard.cpp"

class CNode
{
public:
    double dummy;
    CNode *next;

    CNode(double v)
    {
        this->dummy = v;
        this->next = NULL;
    }
};

int main()
{
    // Init TLS, ...
    DllMain(NULL, DLL_PROCESS_ATTACH, NULL);

    int bigSize = 600000000;
    char *big;

    printf("allocate big memory block, use it, free it ... should be okay\n");
    big = new char[bigSize];
    memset(big, 0, bigSize);
    delete[] big;
    big = NULL;

    printf("allocate many small objects ... should be okay\n");
    // memory usage of the process increases
    int smallElementCount = 100000000;
    CNode head(0), *current;
    current = &head;

    for (int i = 1; i < smallElementCount; i++)
    {
        current->next = new CNode(i);
        current = current->next;
    }

    printf("free all small objects ... should be okay\n");
    // memory usage of the process does NOT decrease
    current = head.next;
    while(current)
    {
        CNode* t = current->next;
        delete current;
        current = t;
    }

    printf("allocate another big memory block, use it, free it ... should be CRASH (bad_alloc)!!!\n"); 
    big = new char[bigSize];
    memset(big, 0, bigSize);
    delete[] big;
    big = NULL;

    return 0;
}

Can you check this issue, please. This would be helpful.

Thanks
Bastian

can not cross compile in arm

when i cross compilt it, it is error:
{standard input}: Assembler messages:
{standard input}:1118: Error: misaligned branch destination
{standard input}:1128: Error: misaligned branch destination
{standard input}:1132: Error: misaligned branch destination
{standard input}: Assembler messages:
{standard input}:356: Error: misaligned branch destination
{standard input}:366: Error: misaligned branch destination
{standard input}:370: Error: misaligned branch destination

any idea?

shbench.cpp license apparently forbids cloning

The leader comment of benchmarks/shbench/shbench.cpp says:

 * This source code, or source code derived from it, may not be redistributed
 * without express written permission of the copyright owner.

Could you clarify whether we (interested devs on GitHub) have such permission? I don't mean here in this issue discussion so much as in the file itself.

This license declaration seems at odds with "Hoard is distributed under the GPL".

Thanks!

Simple hello world program crashes when linked to hoard

Hi,

I have compiled Hoard according to the instructions under Windows 10 with Compiler "Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24218.1 for x64".

I have then created a new dir containing

  • hello.cpp
  • make.bat
  • libhoard.lib
  • libhoard.dll
  • uselibhoard.cpp

I am attaching the text files as "hg_test.zip" to this post:
hg_test.zip

I have then called "make" which creates "hello.exe", with this output on the console:
``
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24218.1 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

hello.cpp
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale(341): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\exception(359): warning C4577: 'noexcept' used with no exception handling mode specified; termination on exception is not guaranteed. Specify /EHsc
uselibhoard.cpp
Generating Code...
Microsoft (R) Incremental Linker Version 14.00.24218.1
Copyright (C) Microsoft Corporation. All rights reserved.

/out:hello.exe
hello.obj
uselibhoard.obj
libhoard.lib
``

(OK - I have specified /EHsc in addition to get rid of those warnings, but this is irrelevant here as the crash still happens then.)

When I now execute "hello.exe", a popup window appears: "Appication Error - The application was unable to start correctly (0xc0000142). Click OK to close the application."

What am I doing wrong?

Best regards,
Hagen

Failed to allocate more than 2147483647 bytes of memory

I wanted to evaluate the Hoard for one of the project but i observed that Hoard fails to allocate more than 2GB of memory. I built the Hoard 3.12 with g++ version 4.9.0 on Linux. Is there any limitation with the Hoard for the memory allocation. If not what could be wrong?
The allocation for 2147483647 bytes succeeds and allocation for 2147483648 onward fails.

Below are the g++ and Linux details:

	g++ --version
	g++ (GCC) 4.9.0 20130520 (experimental)
	Copyright (C) 2013 Free Software Foundation, Inc.
	This is free software; see the source for copying conditions.  There is NO
	warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
	uname -a
	Linux xxx.xxx.com 2.6.32-279.el6.x86_64 #1 SMP Fri Jun 22 12:19:21 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Attached here AllocMem.zip the sample program. Below the sample program output for allocations:

With Default gcc allocator:

	./AllocMem 2147483648
	Trying to allocate 2147483648 bytes of memory
	Successfully allocated 2147483648 bytes of memory

With Hoard Allocator:

	export LD_PRELOAD=/data/local/MemoryAllocators/Hoard/libhoard.so
	./AllocMem 2147483648
	Trying to allocate 2147483648 bytes of memory
	Failed to allocate 2147483648 bytes of memory

New release?

Hi, I'm trying to update the out of date hoard port in MacPorts to the latest version 3.12 but I'm getting this build failure, on macOS 10.13.6 with Xcode 9.4.1:

In file included from source/libhoard.cpp:33:
In file included from Heap-Layers/heaplayers.h:106:
In file included from Heap-Layers/heaps/all.h:2:
In file included from Heap-Layers/heaps/./debug/all.h:4:
In file included from Heap-Layers/heaps/./debug/sanitycheckheap.h:13:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/map:818:5: error: static_assert failed "Allocator::value_type must be same type as value_type"
    static_assert((is_same<typename allocator_type::value_type, value_type>::value),
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Heap-Layers/heaps/./debug/sanitycheckheap.h:110:13: note: in instantiation of template class 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >' requested here
    mapType allocatedObjects;
            ^
Heap-Layers/heaps/./debug/sanitycheckheap.h:64:16: error: no type named 'iterator' in 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >'
      mapType::iterator i;
      ~~~~~~~~~^
Heap-Layers/heaps/./debug/sanitycheckheap.h:88:16: error: no type named 'iterator' in 'std::__1::map<void *, unsigned long, std::__1::less<void *>, HL::STLAllocator<std::__1::pair<const void *, unsigned long>, HL::FreelistHeap<HL::ZoneHeap<HL::MmapHeap, 16384> > > >'
      mapType::iterator i;
      ~~~~~~~~~^

I tried master (Hoard May 24 2018, Heap-Layers Oct 17 2018) and it builds, but I would rather not update the MacPorts port to some random point in time of your repositories. Is a new stable release planned at some point to which I could then update the port?

Would be great if, when you make a new release, you could provide a tarball containing the complete source code of Hoard and Heap-Layers and attach it to the GitHub release, like you did before for version 3.10, so we don't have to clone from git.

Build failed: error: there are no arguments to ‘getpid’ ...

I tried to build hoard from source on linux Fedora core 17 (64bit) gcc version 4.7.2 20120921 (Red Hat 4.7.2-2) (GCC) I did git clone --recursive but it give out below error. here is some part of compilation output which contained errors.

Thanks

g++ -g -W -Wconversion -Wall -m64 -I/usr/include/nptl -pipe -fPIC -O3 -finline-limit=20000 -finline-functions -DNDEBUG -I. -Iinclude -Iinclude/util -Iinclude/hoard -Iinclude/superblocks -IHeap-Layers -D_REENTRANT=1 -shared source/libhoard.cpp source/unixtls.cpp Heap-Layers/wrappers/gnuwrapper.cpp -Bsymbolic -o libhoard.so -ldl -lpthread
In file included from Heap-Layers/heaps/debug/all.h:3:0,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from source/libhoard.cpp:33:
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
Heap-Layers/heaps/debug/logheap.h:35:46: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
.
.
.
source/unixtls.cpp:47:6: warning: multi-line comment [-Wcomment]
In file included from Heap-Layers/heaps/debug/all.h:3:0,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from include/hoard/hoardheap.h:31,
from include/hoard/hoardtlab.h:39,
from source/unixtls.cpp:56:
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
.
.
.
Heap-Layers/heaps/debug/logheap.h: In constructor ‘HL::Log<Obj, MAX_ENTRIES>::Log()’:
Heap-Layers/heaps/debug/logheap.h:35:46: error: there are no arguments to ‘getpid’ that depend on a template parameter, so a declaration of ‘getpid’ must be available [-fpermissive]
.
.
.Heap-Layers/heaps/general/kingsleyheap.h: In function ‘int Kingsley::ceilLog2(size_t)’:
Heap-Layers/heaps/general/kingsleyheap.h:89:63: warning: conversion to ‘int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/wrappers/gnuwrapper.cpp: In function ‘void my_init_hook()’:
Heap-Layers/wrappers/gnuwrapper.cpp:85:25: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:86:23: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:87:26: warning: ‘__realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:88:27: warning: ‘__memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:91:7: warning: ‘__malloc_hook’ is deprecated (declared at /usr/include/malloc.h:176) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:92:7: warning: ‘__free_hook’ is deprecated (declared at /usr/include/malloc.h:173) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:93:7: warning: ‘realloc_hook’ is deprecated (declared at /usr/include/malloc.h:179) [-Wdeprecated-declarations]
Heap-Layers/wrappers/gnuwrapper.cpp:94:7: warning: ‘memalign_hook’ is deprecated (declared at /usr/include/malloc.h:183) [-Wdeprecated-declarations]
In file included from Heap-Layers/heaps/top/mmapheap.h:50:0,
from Heap-Layers/heaps/debug/sanitycheckheap.h:18,
from Heap-Layers/heaps/debug/all.h:4,
from Heap-Layers/heaps/all.h:2,
from Heap-Layers/heaplayers.h:105,
from Heap-Layers/wrappers/gnuwrapper.cpp:23:
Heap-Layers/utility/myhashmap.h: In instantiation of ‘Value HL::MyHashMap<Key, Value, Allocator>::get(Key) [with Key = void
; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/heaps/top/mmapheap.h:178:33: required from here
Heap-Layers/utility/myhashmap.h:69:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::erase(Key) [with Key = void
; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/heaps/top/mmapheap.h:195:23: required from here
Heap-Layers/utility/myhashmap.h:82:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
Heap-Layers/utility/myhashmap.h: In instantiation of ‘void HL::MyHashMap<Key, Value, Allocator>::insert(Key, Value) [with Key = void
; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’:
Heap-Layers/utility/myhashmap.h:65:7: required from ‘void HL::MyHashMap<Key, Value, Allocator>::set(Key, Value) [with Key = void
; Value = long unsigned int; Allocator = HL::MmapHeap::MyHeap]’
Heap-Layers/heaps/top/mmapheap.h:170:25: required from here
Heap-Layers/utility/myhashmap.h:108:67: warning: conversion to ‘unsigned int’ from ‘long unsigned int’ may alter its value [-Wconversion]
make: *** [linux-gcc-x86-64] Error 1

Build issue on Visual Studio 2017

I downloaded the latest code from github and ran nmake from a VS 2017 command line. I got the following error:

    cl /I. /Iinclude /Iinclude/util /Iinclude/hoard /Iinclude/superblocks /IHeap-Layers /D "NDEBUG" /D "_WINDOWS" /D "_WINDLL" /D "_WINRT_DLL" /D "_UNICODE" /D "UNICODE" /Zi /Ox /MD /nologo /W1 /WX- /Ox /Oi /Oy- /Gm- /EHsc /MD /GS /Gy /Zc:wchar_t /Zc:forScope /Gd /errorReport:queue "source\libhoard.cpp" "Heap-Layers\wrappers\winwrapper.cpp" "source\wintls.cpp" /GL /link /DLL /subsystem:console /OUT:libhoard.dll

libhoard.cpp
c:\users\jeff\desktop\hoard-master\src\heap-layers\utility\bins.h(35): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
winwrapper.cpp
wintls.cpp
c:\users\jeff\desktop\hoard-master\src\heap-layers\utility\bins.h(35): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.15.26726\bin\HostX64\x64\cl.EXE"' : return code '0x2'
Stop.

"Assertion `c2s(left) >= sz' failed." on large array allocation.

Hi!

I tried the git master version of Hoard and I am pretty amazed by the performance. My first test where all right. But my tests with larger problem sizes failed. Attached I send you the output of gdb after a assertion raised.

include/hoard/geometricsizeclass.h:56: static int Hoard::GeometricSizeClass<MaxOverhead, Alignment>::size2class(size_t) [with int MaxOverhead = 20; int Alignment = 16; size_t = long unsigned int]: Assertion `c2s(left) >= sz' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff6eae1c9 in raise () from /usr/lib/libc.so.6
(gdb) where
#0  0x00007ffff6eae1c9 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff6eaf5c8 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff6ea7356 in __assert_fail_base () from /usr/lib/libc.so.6
#3  0x00007ffff6ea7402 in __assert_fail () from /usr/lib/libc.so.6
#4  0x00007ffff7b6f1d9 in Hoard::GeometricSizeClass<20, 16>::size2class (sz=314956304)
    at include/hoard/geometricsizeclass.h:56
#5  0x00007ffff7b719e6 in Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource>::malloc (this=0x7ffff7dc57b0 <getMainHoardHeap()::thBuf+244688>, sz=314956304) at include/hoard/thresholdsegheap.h:37
#6  0x00007ffff7b71232 in HL::LockedHeap<HL::SpinLockType, Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource> >::malloc (this=0x7ffff7dc57b0 <getMainHoardHeap()::thBuf+244688>, sz=314956304)
    at Heap-Layers/heaps/threads/lockedheap.h:43
#7  0x00007ffff7b70cc4 in HL::ThreadHeap<64, HL::LockedHeap<HL::SpinLockType, Hoard::ThresholdSegHeap<25, 1048576, 80, &Hoard::GeometricSizeClass<20, 16>::size2class, &Hoard::GeometricSizeClass<20, 16>::class2size, HL::AdaptHeap<HL::DLList, Hoard::objectSource>, Hoard::objectSource> > >::malloc (this=0x7ffff7db0678 <getMainHoardHeap()::thBuf+158360>, 
    sz=314956304) at Heap-Layers/heaps/./threads/threadheap.h:66
#8  0x00007ffff7b7063e in HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::slowPath (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, sz=314956304)
    at Heap-Layers/heaps/./combining/hybridheap.h:82
#9  0x00007ffff7b700a2 in HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap>::malloc (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, sz=314956304)
    at Heap-Layers/heaps/./combining/hybridheap.h:57
#10 0x00007ffff7b6f9f9 in HL::ANSIWrapper<Hoard::IgnoreInvalidFree<HL::HybridHeap<8192, Hoard::ThreadPoolHeap<2048, 128, Hoard::PerThreadHoardHeap>, Hoard::BigHeap> > >::malloc (this=0x7ffff7d89be0 <getMainHoardHeap()::thBuf>, 
    sz=314956304) at Heap-Layers/wrappers/ansiwrapper.h:52
#11 0x00007ffff7b6f4fb in Hoard::ThreadLocalAllocationBuffer<11, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getSizeClass, &HL::bins<Hoard::HoardSuperblockHeader<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536>::getClassSize, 256u, 262144u, Hoard::HoardSuperblock<HL::SpinLockType, 65536, Hoard::SmallHeap>, 65536u, Hoard::HoardHeapType>::malloc (this=0x7ffff7fd0770, sz=314956304) at include/superblocks/tlab.h:102
#12 0x00007ffff7b6e7ec in xxmalloc (sz=314956304) at source/libhoard.cpp:116
#13 0x00007ffff7b77909 in malloc (sz=314956304) at Heap-Layers/wrappers/gnuwrapper.cpp:265
#14 0x000000000040680a in utils_read_metis_edges (mg=0x7ffff7f90070, 
    file_name=0x7fffffffe9c9 "../../Graph-Daten/pokec/nodes-0.7-1/pokec-clean-0.7-1.metis") at utils.c:47
#15 0x000000000040c072 in main (argc=2, argv=0x7fffffffe698) at community-find.c:26

The problematic malloc at my side leads to the allocation of 314956304 bytes. Malloc of smaler arrays went fine.

How may I use Hoard for large memory allocs?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.