Write a compiler wrapper that intercepts the compilation process, and transforms the compiler invocation to apply TypeART.
Targets are multi-file Makefile projects and CMake.

Usage example

Instead of using the Clang compiler directly, replace it with our wrappers:
cmake -DCMAKE_C_COMPILER=typeart-clang -DCMAKE_CXX_COMPILER=typeart-clang++ ...

TODO

• Make scripts work in MPI context 303724e
• Find required executables through CMake: clang/clang++/opt/llc 3a4c1d6
• Disable script with (environment) flag during CMake config time eab5701
• Test against typeart-bench (CI) codes
• lit tests
• wrapper need to respect sanitizer flag, if TypeART was build with them

Issues

CMake

• mpicc-wrapper fails to set variables with CMakes find_config(MPI).

  message(STATUS "LIBS ${MPI_C_LIBRARIES}")
  message(STATUS ${MPI_C_COMPILE_OPTIONS})
  message(STATUS ${MPI_C_COMPILE_OPTIONS})
  message(STATUS ${MPI_C_LINK_FLAGS})
  message(STATUS ${MPIEXEC})  # this is set
  

  • This works instead: target_link_libraries(${target} PUBLIC MPI::MPI_C)

Assuming a target application does not use any user-defined types (or TypeART filters all related allocations) no types.yaml file is ever created.
However, the runtime lib requires a valid file path to types.yaml and otherwise exits.

Potential fix

Only exit if the user specified the environment variable TA_TYPE_FILE for the file location and it does not exist.
If defaultTypeFileName is not found (types.yaml), print a warning.

TODO

In turn, the runtime needs to handle the case of a callback for a user-defined type gracefully.

This is likely backward compatibiliy breaking change.

TypeInterface

The type id numbering should be reworked (enum typeart_builtin_type_t ) to handle new types (see also #62) and be robust w.r.t. code changes.

Where to put a new type?

Currently: TA_PTR is 10 and TA_NUM_VALID_IDS is 11.
Potentially:

• TA_PTR is 254, TA_UNKNOWN_TYPE is 255 and User-def. types start at 256 (as before).
• TA_NUM_VALID could be "concrete LLVM type" + 1 (e.g., TA_PPC_FP128 + 1 or TA_NEW_TYPE + 1)

Needs (some) backward compat. in case a new type is added.

TypeDB & TypeManager

Both dependent on TypeInterface refactoring.

TypeDB

• BuiltinNames/Sizes should be generated (or checked) dependent on TypeInterface.
• Query functions should be checked after refactoring.
• What about isUserDefinedType w.r.t. TA_USER_DEF flag etc.?

TypeManager

• Adapt getOrRegisterType, see also reserveNextId

The current workflows (CI, CI-ext) are sequential. This could be changed by using multi-job workflows, saving some execution time.

Pre-requisites

To avoid too much code/steps duplication, recent composite actions can be used depending on implementation state, see 1. Next Steps for Fully Functioning Composite Actions and 2. composite-run-steps-actions

• Action for setup of Clang/LLVM/OpenMPI package
  • Can this include the environment setup for the LLVM executables?
  • Can we set GITHUB_ENV values in a composite action?

Side effects

Artifacts are generated per job, e.g., 2 files are generated for the CI workflow: 1. coverage html report and 2. lulesh build/run data.

CI

Parallel Jobs:

1. Build for coverage/testing

OMP Extension

Build matrix to test different thread-safety levels for the TypeART runtime libary.

• Combine coverage of all levels (safeptr, mutex, unsafe)

CI-ext

Parallel Jobs with and without thread-safety enabled:

1. test-bench

• Dividing the subprojects probably too much overhead

2. AD test-bench
3. OMP test-bench

The segfault happens, if the Runtime is instantiated (and then discarded) by calling only the API that does not mutate the AccessCounter during execution, e.g.,

int main(int argc, char** argv) {
  const void* ret_check=NULL;
  const void* addr = 1;
  typeart_get_return_address(addr, &ret_check);
  return 0;
}

Defect

CounterStats::create -> vals is empty, the following code causes the segfault:

    Counter min  = *std::min_element(vals.begin(), vals.end());
    Counter max  = *std::max_element(vals.begin(), vals.end());

See https://llvm.org/docs/LangRef.html

Change pointer argument from ReadOnly to ReadNone(?):

readonly
On an argument, this attribute indicates that the function does not write through this pointer argument, even though it may write to the memory that the pointer points to.
If a readonly function writes memory visible to the program, or has other side-effects, the behavior is undefined. If a function writes to a readonly pointer argument, the behavior is undefined.

readnone
On an argument, this attribute indicates that the function does not dereference that pointer argument, even though it may read or write the memory that the pointer points to if accessed through other pointers.
If a readnone function reads or writes memory visible to the program, or has other side-effects, the behavior is undefined. If a function reads from or writes to a readnone pointer argument, the behavior is undefined.

Mark functions as (and also readnone?):

nofree
This function attribute indicates that the function does not, directly or indirectly, call a memory-deallocation function (free, for example). As a result, uncaptured pointers that are known to be dereferenceable prior to a call to a function with the nofree attribute are still known to be dereferenceable after the call (the capturing condition is necessary in environments where the function might communicate the pointer to another thread which then deallocates the memory).

Querying type info has an overflow of an unsigned type in getTypeInfo when resolving internal types of structs.

Trigger example

struct Datastruct {
  int start;
  double middle;
  float end;
};  
__typeart_alloc((const void*)&data, 257, 1);
  typeart_status status = typeart_get_type((const void*)&data.middle, &id_result, &count_check);

Applies to thread-safe codes with ENABLE_SAFEPTR=ON.

Lulesh appears to finish execution. Crashes when the TypeART softcounters are (almost?) printed, e.g.:

2021-04-05T18:39:18.1763435Z 3: [fv-az196-541:05429] Signal: Segmentation fault (11)
2021-04-05T18:39:18.1764428Z 3: [fv-az196-541:05429] Signal code: Address not mapped (1)
2021-04-05T18:39:18.1765347Z 3: [fv-az196-541:05429] Failing at address: 0x8
2021-04-05T18:39:18.1766476Z 3: [fv-az196-541:05429] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7fbf69ce33c0]
2021-04-05T18:39:18.1768594Z 3: [fv-az196-541:05429] [ 1] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(_ZN2sf28contention_free_shared_mutexILj36ELb0EE4lockEv+0x4f)[0x7fbf6a1e934f]
2021-04-05T18:39:18.1771641Z 3: [fv-az196-541:05429] [ 2] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(_ZN7typeart17AllocationTracker10doFreeHeapEPKvS2_+0x22)[0x7fbf6a1e6932]
2021-04-05T18:39:18.1774331Z 3: [fv-az196-541:05429] [ 3] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(__typeart_free+0x96)[0x7fbf6a1e76c6]
2021-04-05T18:39:18.1775733Z 3: [fv-az196-541:05429] [ 4] lulesh2.0[0x438fb3]
2021-04-05T18:39:18.1776592Z 3: [fv-az196-541:05429] [ 5] lulesh2.0[0x426fed]
2021-04-05T18:39:18.1777631Z 3: [fv-az196-541:05429] [ 6] /lib/x86_64-linux-gnu/libc.so.6(+0x49a27)[0x7fbf69b25a27]
2021-04-05T18:39:18.1778850Z 3: [fv-az196-541:05429] [ 7] /lib/x86_64-linux-gnu/libc.so.6(on_exit+0x0)[0x7fbf69b25be0]
2021-04-05T18:39:18.1780136Z 3: [fv-az196-541:05429] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfa)[0x7fbf69b030ba]
2021-04-05T18:39:18.1781189Z 3: [fv-az196-541:05429] [ 9] lulesh2.0[0x404b9e]
2021-04-05T18:39:18.1782073Z 3: [fv-az196-541:05429] *** End of error message ***
2021-04-05T18:39:18.1782994Z 3: [fv-az196-541:05432] *** Process received signal ***
2021-04-05T18:39:18.1783956Z 3: [fv-az196-541:05432] Signal: Segmentation fault (11)
2021-04-05T18:39:18.1784937Z 3: [fv-az196-541:05432] Signal code: Address not mapped (1)
2021-04-05T18:39:18.1785865Z 3: [fv-az196-541:05432] Failing at address: 0x70
2021-04-05T18:39:18.1786986Z 3: [fv-az196-541:05432] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7fb936acd3c0]
2021-04-05T18:39:18.1789170Z 3: [fv-az196-541:05432] [ 1] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(_ZN2sf28contention_free_shared_mutexILj36ELb0EE4lockEv+0x4c)[0x7fb936fd334c]
2021-04-05T18:39:18.1792198Z 3: [fv-az196-541:05432] [ 2] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(_ZN7typeart17AllocationTracker10doFreeHeapEPKvS2_+0x22)[0x7fb936fd0932]
2021-04-05T18:39:18.1794941Z 3: [fv-az196-541:05432] [ 3] /home/runner/work/tudasc-typeart/tudasc-typeart/install/typeart/lib/libtypeart-rt.so(__typeart_free+0x96)[0x7fb936fd16c6]
2021-04-05T18:39:18.1796325Z 3: [fv-az196-541:05432] [ 4] lulesh2.0[0x438fb3]
2021-04-05T18:39:18.1797200Z 3: [fv-az196-541:05432] [ 5] lulesh2.0[0x426fed]
2021-04-05T18:39:18.1798238Z 3: [fv-az196-541:05432] [ 6] /lib/x86_64-linux-gnu/libc.so.6(+0x49a27)[0x7fb93690fa27]
2021-04-05T18:39:18.1799427Z 3: [fv-az196-541:05432] [ 7] /lib/x86_64-linux-gnu/libc.so.6(on_exit+0x0)[0x7fb93690fbe0]
2021-04-05T18:39:18.1800702Z 3: [fv-az196-541:05432] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfa)[0x7fb9368ed0ba]
2021-04-05T18:39:18.1801755Z 3: [fv-az196-541:05432] [ 9] lulesh2.0[0x404b9e]
2021-04-05T18:39:18.1802624Z 3: [fv-az196-541:05432] *** End of error message ***

LLVM pass uses i32 for the stack allocation counter.
However, CallbackInterface.h uses size_t: void __typeart_leave_scope(size_t alloca_count);

Defect

This is closely related to #40

#include <stddef.h>
void __typeart_leave_scope(size_t alloca_count);
int main(void) {
  __typeart_leave_scope(0);
  return 0;
}

Applying TypeART to this code (at no optim. settings), the pass crashes:

declare dso_local void @__typeart_leave_scope(i64) #1

Call parameter type does not match function signature!
  %__ta_counter_load = load i32, i32* %__ta_alloca_counter
 i64  call void @__typeart_leave_scope(i32 %__ta_counter_load)
in function main

The MemOpVisitor currently only supports CallInst for (heap) allocations.

In some codes, InvokeInst may be used by the compiler for heap allocations, see InvokeInst LangRef

InvokeInst and CallInst have the common base class CallBase, which can be used to unify the collected list of MallocData.

CI and CI-ext are currently broken at step Build TypeART.

Potential issue llvm-9 is picked up by CMake, instead of installed v10.0.1, see file CI Step 8_Build TypeART.txt

TypeART should be configurable for user-defined allocation (and deallocation) functions

Consider the AMG multi grid solver.

AMG uses macros for memory allocation, e.g., a calloc-like implementation:

#define hypre_CTAlloc(type, count) \
( (type *)hypre_CAlloc((unsigned int)(count), (unsigned int)sizeof(type)) )

In turn the called function looks approximately like this:

char* hypre_CAlloc(int count, int elt_size) {
  char* ptr;
  int size = count * elt_size;

  ptr = calloc(count, elt_size);

  return ptr;
}

The defect:

Assume a call like:

 double* a = hypre_CTAlloc(double, num_variables);

1. Our static pass is not aware of this macro, it instead finds the C calloc.
2. Thus, it deduces the char* type for that calloc, which is put into our runtime.
3. The subsequent cast of the char* returned by hypre_CAlloc to double is not found by our current implementation.

This is a tracking issue for the support of array cookies in TypeART.

Steps

• Research when and how array cookies are added
• Add test cases for cases where array cookies are expected to be added -> 65684d7, 22d8b4f
• Implementation
  • Finding array cookies -> d40d567
  • Add array cookie data to existing data structures -> 7781f78
  • Consider the padding of array cookies when calculating array element counts -> 0817a3a
  • In case an array cookie is present, instrument the actual starting address of the array instead of the originally allocated address -> 650b1a3
  • Add instrumentation for array cookies

Unresolved Questions

• Should array cookies be instrumented in any way and if so, how? -> for now, only to correctly calculate array sizes

Array Cookie research

Array cookies are a size_t value saving the allocated length of an array. These cookies are allocated under certain conditions when operator new is used to allocate an array.

=> When are array cookies created?

According to [1], an array cookie is not allocated if either of

• "the element type T has a trivial destructor […] and the usual (array) deallocation function […] does not take two arguments" or
• "the new operator being used is ::operator new [](size_t, void*)"

In Clang this is implemented in CGCXXABI::requiresArrayCookie using:

• CXXNewExpr::doesUsualArrayDeleteWantSize and CXXDeleteExpr::doesUsualArrayDeleteWantSize to check whether the delet function takes two arguments.
• CXXNewExpr::getAllocatedType as expr->getAllocatedType().isDestructedType() to check whether the array element type has a non-trivial destructor.

=> How are array cookies created?

According to [1]:

• array cookies always have size sizeof(size_t)
• if align is the maximum alignment of size_t and an element of the array and padding is the maximum of sizeof(size_t} and align bytes:
  • "The space allocated for the will be the space required by the array itself plus padding bytes"
  • "The cookie will be stored in the sizeof(size_t) bytes immediately preceding the array
    In Clang this is implemented in ItaniumCXXABI::InitializeArrayCookie.

[1] https://itanium-cxx-abi.github.io/cxx-abi/abi.html#array-cookies

Currently only one allocation filter strategy is supported

The filter should be outlined to a base class, and different strategies inhereit from a common interface (see feat/sc20):

• Base class, for a common interface
• Runtime selectable with argument
• Whole-program call-graph filter

When a high log level (tested with 3) is chosen, the amg2013 integration test fails to link during the build phase.
In contrast, with level 0 everything works as expected.

The current implementation uses lambda functions for each memory operation

Each lambda (i) collects information for the callback (arguments), and (ii) adds the instrumentation callback at the correct position in the code.

In reference to #4 w.r.t. InvokeInst handling, this should be extracted and divided into two phases for easier maintainability:

1. Per BasicBlock, the information for the callback is collected (and stored appropriately in a list)
2. A Strategy is used to execute the instrumentation, per BB, using the collected information of phase 1

The default strategy is then to add the callback right after the memory operation (as is the case now).

Happens with MPI_LOGGER=OFF.

Fix

• Change:

  llvm::errs() << (LEVEL) << " " << LOG_BASENAME_FILE << ":" << __func__ << ":" << __LINE__ << ": " << MSG << "\n";

• To:

  std::string s;
  llvm::raw_string_ostream rso(s);
  rso << (LEVEL) << LOG_BASENAME_FILE << ":" << __func__ << ":" << __LINE__ << ":" << MSG << "\n";
  // log rso.str()...

Currently, if a test is unsupported in all configurations that are tested on the CI, it succeeds without notifying developers about the tests that were not executed.

If, say, an integer for the size of an array is passed to a malloc call, the filter will keep the value, as malloc is a decl call.
decl calls are unknown to the filter, hence, it keeps the aforementioned integer.

Example

extern double* a;
void foo() {
  int n = ...; // n is not filtered
  a = (double*)malloc(sizeof(double) * n);
}

Solution

Need to whitelist all known allocation and deallocation calls.
Also applies to the TypeART callback functions.

The different, collected memory instructions should all be wrapped with some struct, cf. MallocData, AllocaData.

Currently, the recorder does not track distinct types or scalar vs. array alloc

Consider the Recorder from feat/sc20 (Counter). It tracks distinct types and corresponding counts as well as distinction between scalar vs. array allocations.

Potential improvements:

• Output format for easier parsing
• Better TypeART runtime memory overhead estimation (especially w.r.t. non std::map implementations of the pointer map)

The commit message payload [ci skip] does not lead to skipping the CI extended test runner.

Fix

Change || to &&: "!contains(..., '[ci skip]') || !contains(..., '[ci ext skip]')"
to "!contains(..., '[ci skip]') && !contains(..., '[ci ext skip]')"

The MPI interceptor library does not currently check the type and element count when checking a buffer used as an argument to a MPI call.

Steps

• Research what MPI types exist and how they can be inspected
• Implementation
  • Add type checking for builtin MPI types -> b8295b9
  • Add the ability to run tests using mpic++ as a compiler and mpirun and FileCheck for execution and verification -> 82c6a64, 2408cac
  • Add type checking for custom MPI types
  • Add a README explaining whether and how any given MPI function is typechecked

Unresolved Questions

Research

All MPI types that may be encountered can generally be divided into two categories: builtin MPI types and custom MPI types. Both are represented using a common C++ type MPI_Datatype. A list of builtin MPI types for C/C++ can be found here.
For checking a custom MPI type, the functions MPI_Type_get_envelope and MPI_Type_get_contents may be used. MPI_Type_get_envelope allows to retrieve the function which was used to create a given type and the number of arguments used while MPI_Type_get_contents allows to retrieve the actual arguments passed to the function.

Currently, TypeART uses __builtin_return_address(0) to find the address location of where the allocation occurred

Alternatively, (i) during compilation each allocation is tagged with a unique ID, (ii) the id's are passed as arguments to the callbacks, and (iii) the ID and other useful (debug) information are serialised for look-ups in the runtime

The serialisation should happen as json/yaml, see preliminary implementation in feat/sc20

TypeART currently breaks when building with libc++.

Issues

• Missing header unordered_map in IRPath.h and TypeARTFunctions.h
• std::map internal type reference, see issue #58

If onAllocStack is called with a nullptr (should not happen), doAlloc will not create an entry in the memory map.
However, the stack vector will, which is errorneous.

doAlloc should be restructured to return an enum return code, which the onAlloc functions (heap, stack, global) can use for corner case handling.

Currently, the MPI interceptor is only useful to verify proper tracking of the buffer passed to MPI calls in a (TypeART) target code.

Simple MPI type checking w.r.t. basic MPI datatypes should be implemented to have a more thorough test insight.
This can later be extended to derived datatypes.

For reference, see the demo tool

Currently, the collected stats are serialized and printed when the programs exits, during the destruction of the runtime global.
However, when RuntimeSystem::~RuntimeSystem is called, the LLVM streams llvm::outs()/errs() have already been destroyed.
The logging system depends on these functions. If it is used at this point, the program crashes.

This can lead to problems in certain scenarios. For example, consider the 24_threads_type_check.cpp test.
Both the AccessRecorder and the test itself use atomics. Consequently, some of the functions implemented in the <atomic> are instrumented, if not filtered out.
During the serialization of the softcounters, one of these instrumented functions is called, which causes __typeart_on_alloc_stack to be invoked.
This in turn prompts an entry into the log, crashing the program.

For now, I'm adding a check that will stop the callback functions from being triggered from within the runtime's destructor.
In the future, we might want to think about avoiding this issue altogether, by printing the softcounter stats at an earlier time.
I can think of two possible options:

1. Add an API call, such as typeart_finalize, which the application has to call explicitly.
2. Automatically insert this call at the end of main (of course this will no work if the program returns via exit(0) or similar).

We may also choose to leave the code as is and be mindful of potential future issues.

Currently, the code style is an arbitrary mix of camel case and snake case.

General

Apply clang-tidy to enforce proper naming conventions, see clang-tidy identifier naming.

Style

A good style seems to be Googles configuration (with some modifications), see Google cloud cpp clang-tidy:

  - { key: readability-identifier-naming.NamespaceCase,          value: lower_case }
  - { key: readability-identifier-naming.ClassCase,              value: CamelCase  }
  - { key: readability-identifier-naming.StructCase,             value: CamelCase  }
  - { key: readability-identifier-naming.TemplateParameterCase,  value: CamelCase  }
  - { key: readability-identifier-naming.FunctionCase,           value: aNy_CasE  }
  - { key: readability-identifier-naming.VariableCase,           value: lower_case }
  - { key: readability-identifier-naming.ClassMemberCase,        value: lower_case }
  - { key: readability-identifier-naming.ClassMemberSuffix,      value: _          }
  - { key: readability-identifier-naming.PrivateMemberSuffix,    value: _          }
  - { key: readability-identifier-naming.ProtectedMemberSuffix,  value: _          }
  - { key: readability-identifier-naming.EnumConstantCase,         value: CamelCase }
  - { key: readability-identifier-naming.EnumConstantPrefix,       value: k         }
  - { key: readability-identifier-naming.ConstexprVariableCase,    value: CamelCase }
  - { key: readability-identifier-naming.ConstexprVariablePrefix,  value: k         }
  - { key: readability-identifier-naming.GlobalConstantCase,       value: CamelCase }
  - { key: readability-identifier-naming.GlobalConstantPrefix,     value: k         }
  - { key: readability-identifier-naming.MemberConstantCase,       value: CamelCase }
  - { key: readability-identifier-naming.MemberConstantPrefix,     value: k         }
  - { key: readability-identifier-naming.StaticConstantCase,       value: CamelCase }
  - { key: readability-identifier-naming.StaticConstantPrefix,     value: k         }
  - { key: readability-implicit-bool-conversion.AllowIntegerConditions,  value: 1   }
  - { key: readability-implicit-bool-conversion.AllowPointerConditions,  value: 1   }

Modifications

• (Class) Member suffix _ -> m_
• Method case: camelCase

TODO

• Evaluate out if these modfications make sense.
• Extend/complete list of modifications as necessary

Currently, optimization flags are passed as third argument in run.sh and apply.sh, which requires passing a placeholder for $ta_more_args:

target=$1
ta_more_args=${2:-""}
optimize=${3:-" "}

Must be called with run.sh file.c ' ' -O3.

Fix

Use proper parsing code to detect if we pass an optim flag at any point -O1 ... -O3

readonly typeart_san_flags="@TYPEART_SAN_FLAGS@" is written to in line 88 and line 120.

If the runtime tracks a type with type id < 0 incorrect typen ame and static type size are returned.

Fix

TypeDB::isBuiltinType needs a check with lower and upper bound.

The desired workflow with the optimization flag is to apply TypeART using opt like TA heap > optim flag > TA stack.

However, in the last step stack and (again) heap are instrumented.

Fix

Add -typeart-no-heap flag to last step for both run.sh and apply.sh

In the following code, TypeART calculates:

• Element count: 10
• Type: 0 (aka void*, as there is not cast)

void f() {
  void* pvd = calloc(10, sizeof(double));
}

Fix

Calculate 10 * sizeof(double) in the absence of a (primary) detectable bitcast.

Going forward, the typeart prefix should be used for executables (our scripts) and eventually the LLVM passes and runtime library.

• rename scripts: run-typeart.sh and apply-typeart.sh
• rename passes: typeartpass and meminstfinderpass

To fix, see Environment files:

echo "{name}={value}" >> $GITHUB_ENV

Is:

      - name: Install libc++
        if: matrix.libcxx
        run: sudo apt-get install --no-install-recommends libc++-10-dev libc++abi-10-dev

should be:

      - name: Install libc++
        if: matrix.preset.libcxx
        run: sudo apt-get install --no-install-recommends libc++-10-dev libc++abi-10-dev

The TypeART demo is not properly maintained currently.

Current status

• Makefile broken w.r.t. compiling and deprecated MPI symbols
• CMakeLists untested/out of date?
• Out of date scripts

Issues for release 1.6

• Update Makefile to compile examples with TypeART
• Remove deprecated apply script
• Fix deprecations for OpenMPI 4 or higher

Issues (future)

• Update CMakeLists to build examples
• Add demo to CI system
• Add README.md for demo explanation

If our pass encounters a i1 type, the TypeManafer::getOrRegisterType function will return TA_UNKNOWN_TYPE.

See test omp test 29_threads_concurrent_rwx.cpp without optimizations.

Currently, all allocas, which are not filtered by TypeART, are instrumented

Add an additional condition that we need a direct source relation of the alloca (e.g., a name; line/col. number)

In master/devel branch runtime implementation, the counters are not incremented for the memory operations (heap, stack, global)

Filtering based on linkage, e.g., GlobalValue::ExternalLinkage, is too inclusive.

int i; // @i = dso_local global i32 0, align 4, !dbg !6 -> ExternalLinkage 
int ii = 0; // @ii = dso_local global i32 0, align 4, !dbg !0 -> ExternalLinkage
extern const int eci; //  @eci = external dso_local constant i32, align 4 -> ExternalLinkage
static int si; //  @si = internal global i32 0, align 4, !dbg !12

Building TypeART with ninja works but executing the (lcov) coverage targets fails (e.g., lcov-make).

Most likely this behavior comes from differences between make and ninja w.r.t. commands in add_custom_target, see https://gitlab.kitware.com/cmake/cmake/-/issues/21778

The respective targets have to be re-worked for ninja-compat.

Update

Ninja doesn't like the lcov-make target, in particular, both the fragments --no-extenal and -b ${CMAKE_SOURCE_DIR}.
What seems to work is:

add_custom_target(
  lcov-make
  COMMAND ${LCOV_COMMAND} ${GCOV_TOOL} ${GCOV_WORKAROUND} -c -d ${CMAKE_BINARY_DIR} -o typeart.coverage
  COMMAND ${LCOV_COMMAND} --remove typeart.coverage '${CMAKE_BINARY_DIR}/*' -o typeart.coverage
  # need to remove externals:
  COMMAND ${LCOV_COMMAND} --remove typeart.coverage '/usr/*' -o typeart.coverage
  COMMAND ${LCOV_COMMAND} --remove typeart.coverage '*/llvm/*' -o typeart.coverage
)

A crash during the compilation is caused by explicit/direct calls to the various callback functions:

#include <stddef.h>
void __typeart_leave_scope(int alloca_count);
int main(void) {
  __typeart_leave_scope(0);
  return 0;
}

Potential fix

In TypeARTFunctions.cpp, check if the function is already declared in the module.

Rough fix outline

TAFunctionDeclarator::make_function(...)  {
  ...
  const auto do_make = [&](auto& name, auto f_type) {
    const bool has_f = m.getFunction(name) != nullptr;
    auto fc          = m.getOrInsertFunction(name, f_type);
    if (has_f) {
      auto fcc = fc.getCallee();
      return dyn_cast<Function>(fc.getCallee()->stripPointerCasts());
    }
    ...

Compiling TypeART against libc++ fails. Is _Link_type libstdc++ specific?

Currently, maxHeapAlloc is only incremented when a free heap operation triggers in a target code.

Hence, in a code without free, the max heap is incorrectly printed:

int main(void) {
  for (int i = 1; i <= 6; ++i) {
    // 6 heap alloc, max heap (concurrently) 6
    double* d = (double*)malloc(sizeof(double));
  }
  return 0;
}

-> Prints: Max. Heap Allocs 0 but expeced is 6.

Fix

AccessRecorder calculates max. for every heap allocation "just in time", instead of waiting for a free.

Defect

The following C++ allocation is wrongly identified by TypeART as int64 (LLVM 10)

  struct S1 {
    int x;
    virtual ~S1() = default;
  };
  try {
    S1* ss = new S1[2];
  } catch (...) {
  }

LLVM/TypeART Generated Code

  %5 = invoke i8* @_Znam(i64 40) #4
          to label %6 unwind label %16

6:                                                ; preds = %0
  call void @__typeart_alloc(i8* %5, i32 3, i64 5)
  %7 = bitcast i8* %5 to i64*
  store i64 2, i64* %7, align 16
  %8 = getelementptr inbounds i8, i8* %5, i64 8
  %9 = bitcast i8* %8 to %struct.S1*
  %10 = getelementptr inbounds %struct.S1, %struct.S1* %9, i64 2
  br label %11

Two bitcasts are generated.
TypeART erroneously takes the first one as the candidate to extract the type (i32 3 -> i64).

To be able to handle multi-threaded, e.g., hybrid MPI+OpenMP, applications, the TypeART runtime needs to be made thread-safe.

Most likely also some careful inspection of the instrumentation function interfaces, and the LLVM pass implementation is needed here to account for thread-local storage variables, etc.

OpenMP

Some points apply to general thread-safety.

Runtime

• Pointer map is thread-safe:
  • Mutex (shared)
  • Safe ptr, see https://github.com/AlexeyAB/object_threadsafe
• (Current) Stack allocations are stored with thread-local vector
• omp specific callbacks

Passes

• Support basics for data-flow filter with omp microtask indirection
  • Add data-flow tracking w.r.t. omp tasks (e.g., task_alloc)

The script templates "scrips/*sh.in" should stay in the build directory.
Multi-build projects would each overwrite the respective generated script, leading to unexpected side-effects during execution (wrong TA build executed).

Side effect

Need to look at lit test suite to use the updated script path.

Currently, these scripts are put into the build folder with appropriate (build-folder related) paths settings to the plugins.

Improvement

These should also be added to the install target.

• Set install path (instead of build related)
• Refactor script to put placeholder vars (e.g., @foo@) at central location (e.g., at top)