Giter Club home page Giter Club logo

dpp's Introduction

d++ - #include C and C++ headers in D files

CI Coverage Open on run.dlang.io

Goal

To directly #include C and C++ headers in D files and have the same semantics and ease-of-use as if the file had been #included from C or C++ themselves. Warts and all, meaning that C enum declarations will pollute the global namespace, just as it does "back home".

This work was supported by Symmetry Investments.

Example

// c.h
#ifndef C_H
#define C_H

#define FOO_ID(x) (x*3)

int twice(int i);

#endif
// c.c
int twice(int i) { return i * 2; }
// foo.dpp
#include "c.h"
void main() {
    import std.stdio;
    writeln(twice(FOO_ID(5)));  // yes, it's using a C macro here!
}

At the shell:

$ gcc -c c.c
$ d++ foo.dpp c.o
$ ./foo
$ 30

Open on run.dlang.io

C++ support

C++ support is currently limited. Including any header from the C++ standard library is unlikely to work. Simpler headers might, the probability rising with how similar the C++ dialect used is to C. Despite that, dpp currently does try to translate classes, templates and operator overloading. It's unlikely to work on production headers without judicious use of the --ignore-cursor and --ignore-namespace command-line options. When using these, the user can then define their own versions of problematic declarations such as std::vector.

Limitations

  • Only known to work on Linux with libclang versions 6 and up. It might work in different conditions.
  • When used on multiple files, there might be problems with duplicate definitions depending on imports. It is recommended to put all #includes in one .dpp file and import the resulting D module.
  • Not currently able to translate Linux kernel headers.

Success stories

Known project headers whose translations produce D code that compiles:

  • nanomsg/nn.h, nanomsg/pubsub.h
  • curl/curl.h
  • stdio.h, stdlib.h
  • pthread.h
  • julia.h
  • xlsxwriter.h
  • libvirt/libvirt.h, libvirt/virterror.h
  • libzfs
  • openssl/ssl.h
  • imapfilter.h
  • libetpan/libetpan.h
  • Python.h

Compilation however doesn't guarantee they work as expected and YMMV. Please consult the examples.

Command-line arguments

It is likely that the header or headers need -I flags to indicate paths to be searched, both by this executable and by libclang itself. The --include-path option can be used for that, once for each such path.

Use -h or --help to learn more.

Details

d++ is an executable that wraps a D compiler such as dmd (the default) so that D files with #include directives can be compiled.

It takes a .dpp file and outputs a valid D file that can be compiled. The original can't since D has no preprocessor, so the .dpp file is "quasi-D", or "D with #include directives". The only supported C preprocessor directive is #include.

The input .dpp file may also use C preprocessor macros defined in the file(s) it #includes, just as a C/C++ program would (see the example above). It may not, however, define macros of its own.

d++ goes through the input file line-by-line, and upon encountering an #include directive, parses the file to be included with libclang, loops over the definitions of data structures and functions therein and expands in-place the relevant D translations. e.g. if a header contains:

uint16_t foo(uint32_t a);

The output file will contain:

ushort foo(uint a);

d++ will also enclose each one of these original #include directives with either extern(C) {} or extern(C++) {} depending on the header file name and/or command-line options.

As part of expanding the #include, and as well as translating declarations, d++ will also insert text to define macros originally defined in the #included translation unit so that these macros can be used by the D program. The reason for this is that nearly every non-trivial C API requires the preprocessor to use properly. It is possible to mimic this usage in D with enums and CTFE, but the result is not guaranteed to be the same. The only way to use a C or C++ API as it was intended is by leveraging the preprocessor.

Trivial literal macros however(e.g. #define THE_ANSWER 42) are translated as D enums.

As a final pass before writing the output D file, d++ will run the C preprocessor (currently the cpp binary installed on the system) on the intermediary result of expanding all the #include directives so that any used macros are expanded, and the result is a D file that can be compiled.

In this fashion a user can write code that's not-quite-D-but-nearly that can "natively" call into a C/C++ API by #includeing the appropriate header(s).

Translation notes

enum

For convenience, this declaration:

enum Enum { foo, bar, baz }

Will generate this translation:

enum Enum { foo, bar, baz }
enum foo = Enum.foo;
enum bar = Enum.bar;
enum baz = Enum.baz;

This is to mimic C semantics with regards to the global namespace whilst also allowing one to, say, reflect on the enum type.

Renaming enums

There is the ability to rename C enums. With the following C definition:

enum FancyWidget { Widget_foo,  Widget_bar }

Then adding this to your .dpp file after the #include directive:

mixin dpp.EnumD!("Widget",      // the name of the new D enum
                 FancyWidget,   // the name of the original C enum
                 "Widget_");    // the prefix to cut out

will yield this translation:

enum Widget { foo, bar }

Names of structs, enums and unions

C has a different namespace for the aforementioned user-defined types. As such, this is legal C:

struct foo { int i; };
extern int foo;

The D translations just use the short name for these aggregates, and if there is a name collision with a variable or function, the latter two get renamed and have a pragma(mangle) added to avoid linker failures:

struct foo { int i; }
pragma(mangle, "foo") extern export __gshared int foo_;

Functions or variables with a name that is a D keyword

Similary to name collisions with aggregates, they get an underscore appended and a pragma(mangle) added so they link:

void debug(const char* msg);

Becomes:

pragma(mangle, "debug")
void debug_(const(char)*);

Build Instructions

dub install dpp

After the instructions for your OS (see below), you can use this commands to run dpp:

dub run dpp -- yoursourcefilenamehere.dpp

Note: for a reproducible and cross-platform build environment, you can run setup-cpp with --llvm=11.0.0. This will set up LLVM 11.0.0 and the proper environment variables.

Windows

Install LLVM into C:\Program Files\LLVM\, making sure to tick the "Add LLVM to the system PATH for all users" option.

If libclang.lib was not found, put the lib folder of the llvm directory on the PATH.

Linux

If libclang is not installed, install libclang-10-dev with apt: sudo apt-get install -y -qq libclang-10-dev

If libclang.so was not found, link it using the following command (adjust the installation path and the llvm version):

sudo ln -s path_to_llvm/lib/libclang-12.so.1 /lib/x86_64-linux-gnu/libclang.so

MacOS

If using an external LLVM installation, add these to your ~/.bash_profile

LLVM_PATH="/usr/local/opt/llvm/" # or any other path
LLVM_VERSION="11.0.0"
export PATH="$LLVM_PATH:$PATH"
export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
export LD_LIBRARY_PATH="$LLVM_PATH/lib/:$LD_LIBRARY_PATH"
export DYLD_LIBRARY_PATH="$LLVM_PATH/lib/:$DYLD_LIBRARY_PATH"
export CPATH="$LLVM_PATH/lib/clang/$LLVM_VERSION/include/"
export LDFLAGS="-L$LLVM_PATH/lib"
export CPPFLAGS="-I$LLVM_PATH/include"
export CC="$LLVM_PATH/bin/clang"
export CXX="$LLVM_PATH/bin/clang++"

(adjust the clang version and the external llvm installation path.)

Then run source ~/.bash_profile

If libclang.dylib was not found, link it using the following command (adjust the installation path):

ln -s path_to_llvm/lib/libclang.dylib /usr/local/opt/llvm/lib/libclang.dylib

dpp's People

Contributors

adamdruppe avatar alexandrumc avatar aminya avatar atilaneves avatar burner avatar crazypython avatar dkgroot avatar john-colvin avatar jonathangerlach avatar kinke avatar lempiji avatar nametoolong avatar puffi avatar tom-tan avatar trikko avatar uplinkcoder avatar veelo avatar webfreak001 avatar wilzbach avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dpp's Issues

[Request] Enable the wiki on this repo

Enabling the wiki on this repo would allow users to post information/examples about C libraries they have gotten to compile. It's rarely the case that you don't have to make modifications somewhere to work with a big C library. In any event, it would be good to have a list of libraries that are known to compile.

Error Building with Dub

//Win 10 x64
dmd --version // DMD32 D Compiler v2.079.0
dub --version // DUB version 1.8.0, built on Mar  2 2018 // and with  1.8.1-beta

source\dpp\cursor\macro_.d(38,31): Error: @safe function 'dpp.cursor.macro_.translateMacro' cannot call @system function 'std.stdio.File.rawRead!char.rawRead'

invalid - parameters are ignored

d++ --print-cursors --include-path . --include-path support -a julia.dpp
There is no -a but it doesn't complain. This is more of a problem if you guess -i will be --include-path and it isn't.

dub build dpp does not work

$ dub build dpp
Building package dpp in /home/office/.dub/packages/dpp-0.0.1/dpp/
WARNING: A deprecated branch based version specification is used for the dependency libclang. Please use numbered versions instead. Also note that you can still use the dub.selections.json file to override a certain dependency to use a branch instead.
Performing "debug" build using /usr/bin/dmd for x86_64.
libclang ~master: target for configuration "library" is up to date.
dpp 0.0.1: building configuration "executable"...
.dub/packages/dpp-0.0.1/dpp/source/dpp/runtime/app.d(106,16): Warning: C preprocessor directive #define is not supported
.dub/packages/dpp-0.0.1/dpp/source/dpp/cursor/typedef_.d(89,9): Error: @nogc function dpp.cursor.typedef_.isSomeFunction cannot call non-@nogc function clang.Type.pointee
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(176,8): Error: incompatible types for (type.pointee()) is (null): Type and typeof(null)
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(177,12): Error: incompatible types for (type.pointee()) !is (null): Type and typeof(null)
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(184,40): Error: can only * a pointer, not a Type
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(192,21): Error: can only * a pointer, not a Type
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(202,22): Error: cannot implicitly convert expression (*ptr).pointee() of type Type to const(Type)*
.dub/packages/dpp-0.0.1/dpp/source/dpp/type/package.d(246,31): Error: can only * a pointer, not a Type
/usr/bin/dmd failed with exit code 1.

volatile size_t

#include <stddef.h>

struct my_struct {
    volatile long test1;
    volatile size_t test2;
    volatile int test3;
};

This struct is translated like this:

    struct my_struct
    {
        c_long test1;
        volatile size_t test2;  // Volatile is kept in this case, why?
        int test3;
    }

And this doesn't like to dmd

Not all macros inside `signal.h` are expanded

On my Ubuntu system, sigaction is defined as:

/* Structure describing the action to be taken when a signal arrives.  */                                             
struct sigaction                                                                                                      
  {                                                                                                                   
    /* Signal handler.  */                                                                                            
#ifdef __USE_POSIX199309                                                                                              
    union                                                                                                             
      {                                                                                                               
   /* Used if SA_SIGINFO is not set.  */                                                                             
   __sighandler_t sa_handler;                                                                                        
   /* Used if SA_SIGINFO is set.  */                                                                                 
   void (*sa_sigaction) (int, siginfo_t *, void *);                                                                  
      }                                                                                                               
    __sigaction_handler;                                                                                              
#define sa_handler __sigaction_handler.sa_handler                                                                    
#define sa_sigaction   __sigaction_handler.sa_sigaction                                                              
#else                                                                                                                 
    __sighandler_t sa_handler;                                                                                        
#endif                                                                                                                
                                                                                                                      
    /* Additional set of signals to be blocked.  */                                                                   
    __sigset_t sa_mask;                                                                                               
                                                                                                                      
    /* Special flags.  */                                                                                             
    int sa_flags;                                                                                                     
                                                                                                                      
    /* Restore handler.  */                                                                                           
    void (*sa_restorer) (void);                                                                                       
  };                           

but it ends up as:

    struct sigaction                                                                                                  
    {                                                                                                                 
        union _Anonymous_1                                                                                            
        {                                                                                                             
            __sighandler_t sa_handler;                                                                                
            void function(int, siginfo_t*, void*) sa_sigaction;                                                       
        }                                                                                                             
        _Anonymous_1 __sigaction_handler;                                                                             
        __sigset_t sa_mask;                                                                                           
        int sa_flags;                                                                                                 
        void function() sa_restorer;                                                                                  
    }                                                                                                                 

So, __USE_POSIX199309 ifdef is expanded (there's __sigaction__handler generated), but the inner defines are not expanded into aliases.

Run example tests on Travis CI

The examples don't run on Travis right now because they fail - the older version of libclang there, even when installing the most recent one from apt, doesn't work the same way as libclang 6.0. The CI script should manually download the right libclang version, just like dstep does.

c standard library doesn't work on macOS

Everything complains about not being able to find the stdint types, e.g. Error parsing '/usr/local/include/nanomsg/nn.h': error: unknown type name 'uint64_t'

even a simple file that just does #include <stdlib.h> doesn't work: Error: object.Exception@../../.dub/packages/libclang-master/libclang/source/clang/package.d(61): Error parsing '/usr/include/stdlib.h': error: unknown type name 'uint8_t'

Anonymous enums with no typedef are ignored

I was translating one of the GSL examples for multidimensional root finding to D. The full .dpp program is here.

The solution is at the top of that file. I had to manually copy in the values from the header file.

extern(C) {
  int GSL_SUCCESS = 0;
  int GSL_CONTINUE = -2;
}

The enum definition that is ignored is found in gsl/gsl_errno.h:

enum { 
  GSL_SUCCESS  = 0, 
  GSL_FAILURE  = -1,
  GSL_CONTINUE = -2,  /* iteration has not converged */
  GSL_EDOM     = 1,   /* input domain error, e.g sqrt(-1) */
  GSL_ERANGE   = 2,   /* output range error, e.g. exp(1e100) */
  GSL_EFAULT   = 3,   /* invalid pointer */
  GSL_EINVAL   = 4,   /* invalid argument supplied by user */
  GSL_EFAILED  = 5,   /* generic failure */
  GSL_EFACTOR  = 6,   /* factorization failed */
  GSL_ESANITY  = 7,   /* sanity check failed - shouldn't happen */
  GSL_ENOMEM   = 8,   /* malloc failed */
  GSL_EBADFUNC = 9,   /* problem with user-supplied function */
  GSL_ERUNAWAY = 10,  /* iterative process is out of control */
  GSL_EMAXITER = 11,  /* exceeded max number of iterations */
  GSL_EZERODIV = 12,  /* tried to divide by zero */
  GSL_EBADTOL  = 13,  /* user specified an invalid tolerance */
  GSL_ETOL     = 14,  /* failed to reach the specified tolerance */
  GSL_EUNDRFLW = 15,  /* underflow */
  GSL_EOVRFLW  = 16,  /* overflow  */
  GSL_ELOSS    = 17,  /* loss of accuracy */
  GSL_EROUND   = 18,  /* failed because of roundoff error */
  GSL_EBADLEN  = 19,  /* matrix, vector lengths are not conformant */
  GSL_ENOTSQR  = 20,  /* matrix not square */
  GSL_ESING    = 21,  /* apparent singularity detected */
  GSL_EDIVERGE = 22,  /* integral or series is divergent */
  GSL_EUNSUP   = 23,  /* requested feature is not supported by the hardware */
  GSL_EUNIMPL  = 24,  /* requested feature not (yet) implemented */
  GSL_ECACHE   = 25,  /* cache limit exceeded */
  GSL_ETABLE   = 26,  /* table limit exceeded */
  GSL_ENOPROG  = 27,  /* iteration is not making progress towards solution */
  GSL_ENOPROGJ = 28,  /* jacobian evaluations are not improving the solution */
  GSL_ETOLF    = 29,  /* cannot reach the specified tolerance in F */
  GSL_ETOLX    = 30,  /* cannot reach the specified tolerance in X */
  GSL_ETOLG    = 31,  /* cannot reach the specified tolerance in gradient */
  GSL_EOF      = 32   /* end of file */
} ;

Note that it doesn't help to explicitly include gsl_errno.h in the .dpp file.

Fails on 'struct macro'

A C header that contains a struct called macro cannot be included, since macro is a (reserved) keyword in D.

dub support

I started adding dub support, see https://github.com/John-Colvin/dub/tree/dpp_support and read the stuff at the top of the README. All it does is just treat .dpp files the same as .d if dppSupport true is set in dub.sdl, but that should be enough to make it work.

Missing pieces in dpp to make it work (bringing it closer to a drop-in for the compilers itโ€™s wrapping) are:

  1. support for 0 and >1 dpp files. Per file: output placed in same location as input (so imports work). 0 is needed so dub can get build information
  2. support for .rsp files like dmd.

Can't build with reggae

../../.dub/packages/reggae-0.5.24/reggae/bin/reggae -b make
[Reggae]        +0s  Writing reggae source files
[Reggae]     +0.02s  Writing reggae configuration
[Reggae]     +0.02s  Writing dub configuration
[Reggae]     +0.02s  Calling `dub --annotate build --compiler=dmd --print-configs --build=docs --nodeps --skip-registry=all --arch=x86_64`
[Reggae]    +0.216s  Calling `dub describe -c executable --nodeps --skip-registry=all --arch=x86_64`
[Reggae]    +0.406s  Calling `dub describe -c unittest --nodeps --skip-registry=all --arch=x86_64`
[Reggae]    +0.608s  Compiling metabuild binary build.o
Couldn't execute ./dcompile --objFile=build.o --depFile=reggaefile.dep dmd -I/Users/john/Git/dpp -Isrc -g -debug /Users/john/Git/dpp/reggaefile.d src/reggae/config.d src/reggae/options.d src/reggae/buildgen_main.d src/reggae/buildgen.d src/reggae/build.d src/reggae/backend/package.d src/reggae/backend/binary.d src/reggae/package.d src/reggae/range.d src/reggae/reflect.d src/reggae/dependencies.d src/reggae/types.d src/reggae/ctaa.d src/reggae/sorting.d src/reggae/file.d src/reggae/rules/package.d src/reggae/rules/common.d src/reggae/rules/d.d src/reggae/rules/c_and_cpp.d src/reggae/core/package.d src/reggae/core/rules/package.d src/reggae/backend/ninja.d src/reggae/backend/make.d src/reggae/backend/tup.d src/reggae/dub/info.d src/reggae/rules/dub.d src/reggae/path.d
in /Users/john/Git/dpp/.reggae:
Could not compile with args:
dmd -I/Users/john/Git/dpp -Isrc -g -debug /Users/john/Git/dpp/reggaefile.d src/reggae/config.d src/reggae/options.d src/reggae/buildgen_main.d src/reggae/buildgen.d src/reggae/build.d src/reggae/backend/package.d src/reggae/backend/binary.d src/reggae/package.d src/reggae/range.d src/reggae/reflect.d src/reggae/dependencies.d src/reggae/types.d src/reggae/ctaa.d src/reggae/sorting.d src/reggae/file.d src/reggae/rules/package.d src/reggae/rules/common.d src/reggae/rules/d.d src/reggae/rules/c_and_cpp.d src/reggae/core/package.d src/reggae/core/rules/package.d src/reggae/backend/ninja.d src/reggae/backend/make.d src/reggae/backend/tup.d src/reggae/dub/info.d src/reggae/rules/dub.d src/reggae/path.d -ofbuild.o -c
/Users/john/Git/dpp/reggaefile.d(8): Error: undefined identifier `CompilationMode.package_`
/Users/john/Git/dpp/reggaefile.d(10): Error: template instance `dubTarget!(TargetName("utl_per_package"), Configuration("unittest"), CompilerFlags("-w -g -debug -version=unitThreadedLight"))` does not match template declaration `dubTarget(alias objsFunction = ()
{
Target[] t;
return t;
}
)(in TargetName targetName, in DubInfo dubInfo, in string compilerFlags, in string[] linkerFlags = [], in Flag!"main" includeMain = Yes.main, in Flag!"allTogether" allTogether = No.allTogether)`
/Users/john/Git/dpp/reggaefile.d(21): Error: template instance `dubObjects!(Configuration("default"), CompilerFlags(debugFlags), No.main, CompilationMode.package_)` template `dubObjects` is not defined
/Users/john/Git/dpp/reggaefile.d(29): Error: template instance `dlangObjectsPerModule!(Sources!"tests", CompilerFlags(debugFlags ~ ["-unittest", "-version=unitThreadedLight"]), dubImportPaths!(Configuration("unittest")))` template `dlangObjectsPerModule` is not defined
/Users/john/Git/dpp/reggaefile.d(39): Error: template instance `dubPackageObjects!(DubPackageName("unit-threaded"), Configuration("unittest"), CompilerFlags(["-unittest", "-version=unitThreadedLight"]))` template `dubPackageObjects` is not defined
/Users/john/Git/dpp/reggaefile.d(46): Error: template instance `dubLink!(TargetName("utl"), Configuration("unittest"), targetConcat!(prodObjs, testObjs, unitThreaded), LinkerFlags("-main"))` template `dubLink` is not defined


bin.name: build.o, bin.cmd: ./dcompile --objFile=build.o --depFile=reggaefile.dep dmd -I/Users/john/Git/dpp -Isrc -g -debug /Users/john/Git/dpp/reggaefile.d src/reggae/config.d src/reggae/options.d src/reggae/buildgen_main.d src/reggae/buildgen.d src/reggae/build.d src/reggae/backend/package.d src/reggae/backend/binary.d src/reggae/package.d src/reggae/range.d src/reggae/reflect.d src/reggae/dependencies.d src/reggae/types.d src/reggae/ctaa.d src/reggae/sorting.d src/reggae/file.d src/reggae/rules/package.d src/reggae/rules/common.d src/reggae/rules/d.d src/reggae/rules/c_and_cpp.d src/reggae/core/package.d src/reggae/core/rules/package.d src/reggae/backend/ninja.d src/reggae/backend/make.d src/reggae/backend/tup.d src/reggae/dub/info.d src/reggae/rules/dub.d src/reggae/path.d

C11 Anonymous structs and unions lead to wrong D code

Given the following files:

test.h

typedef struct {
    union {
        struct {
            double x;
            double y;
            double z;
        };
        double raw[3];
    };
} vec3d_t;

test.dpp

#include "test.h"

unittest {
    vec3d_t v;
    v.raw[1] = 3.0;
    assert(v.y == 3.0);
}

Running:

d++ -unittest test.dpp

We expect: no error.

We get: the following error:

Error: Could not execute `dmd -unittest test.d -oftest`:
test.d(27): Error: no property `raw` for type `vec3d_t`
test.d(28): Error: no property `y` for type `vec3d_t`

Workaround:

Not sure. Reusing the anonymous name would only lead to messy code. Renaming
the internal struct seems best but requires postprocessing.

Version: commit be7ecf3

build under macOS fails

should this be buildable under macOS ?
it worked on archlinux, however on mac with dmd v2.079.0 i get:

source/dpp/cursor/aggregate.d(57,19): Error: no property isDefinition for type Cursor
source/dpp/cursor/aggregate.d(101,15): Error: no property isDefinition for type const(Cursor)
source/dpp/cursor/aggregate.d(109,23): Error: template dpp.cursor.aggregate.translateAggregate.any!((a) => a.isBitField).any cannot deduce function from argument types !()(const(Cursor)[]), candidates are:
/usr/local/opt/dmd/include/dlang/dmd/std/algorithm/searching.d(172,10):        dpp.cursor.aggregate.translateAggregate.any!((a) => a.isBitField).any(Range)(Range range) if (isInputRange!Range && is(typeof(unaryFun!pred(range.front))))
source/dpp/cursor/aggregate.d(134,18): Error: no property isBitField for type const(Cursor)
source/dpp/cursor/aggregate.d(137,19): Error: no property isBitField for type const(Cursor)
source/dpp/cursor/aggregate.d(139,19): Error: no property isDefinition for type const(Cursor)
source/dpp/cursor/aggregate.d(142,39): Error: no property isBitField for type const(Cursor)
source/dpp/cursor/aggregate.d(143,18): Error: no property isBitField for type const(Cursor)
source/dpp/cursor/aggregate.d(143,54): Error: no property bitWidth for type const(Cursor)
source/dpp/cursor/aggregate.d(186,17): Error: no property isBitField for type const(Cursor)
source/dpp/cursor/aggregate.d(187,79): Error: no property bitWidth for type const(Cursor)
source/dpp/cursor/aggregate.d(199,29): Error: no property pointee for type const(Type)
source/dpp/cursor/aggregate.d(219,55): Error: no property pointee for type const(Type)
/usr/local/opt/dmd/include/dlang/dmd/std/algorithm/iteration.d(1108,16):        instantiated from here: FilterResult!(__lambda3, const(Type)[])
source/dpp/cursor/aggregate.d(219,9):        instantiated from here: filter!(const(Type)[])
source/dpp/cursor/aggregate.d(205,29):        instantiated from here: maybeRememberStructs!(const(Type)[])
source/dpp/cursor/dlang.d(20,62): Error: no property mangling for type const(Cursor)
source/dpp/cursor/function_.d(15,12): Error: module `clang` import Language not found
source/dpp/cursor/function_.d(24,67): Error: no property returnType for type const(Type)
source/dpp/cursor/aggregate.d(219,55): Error: no property pointee for type const(Type)
/usr/local/opt/dmd/include/dlang/dmd/std/algorithm/iteration.d(1108,16):        instantiated from here: FilterResult!(__lambda3, MapResult!(__lambda3, FilterResult!(__lambda2, const(Cursor)[])))
source/dpp/cursor/aggregate.d(219,9):        instantiated from here: filter!(MapResult!(__lambda3, FilterResult!(__lambda2, const(Cursor)[])))
source/dpp/cursor/function_.d(29,25):        instantiated from here: maybeRememberStructs!(MapResult!(__lambda3, FilterResult!(__lambda2, const(Cursor)[])))
source/dpp/cursor/translation.d(99,30): Error: no property isDefinition for type const(Cursor)
source/dpp/cursor/translation.d(138,16): Error: module `clang` import AccessSpecifier not found
source/dpp/cursor/translation.d(140,28): Error: no property accessSpecifier for type const(Cursor)

packed struct

The following test would currently fail:

@("packed struct")
@safe unittest {
    shouldCompile(
        C(
            q{
                struct NotPacked {
                    char x;
                    short y;
                    int z;
                };

                struct Packed {
                    char x;
                    short y;
                    int z;
                } __attribute__((__packed__));
            }
        ),
        D(
            q{
                static assert(NotPacked.sizeof == 8, "NotPacked should be 8 bytes");
                static assert(Packed.sizeof == 7, "Packed should be 7 bytes");
            }
        ),
    );
}

Gets fooled by casting

[2] laeeth@sheldrake> ./build.sh                                                            /tank2/develop/caml/example
Error: Could not execute `dmd -c modwrap.d -ofmodwrap`:
modwrap.d(1440): Error: C style cast illegal, use `cast(uintnat)n`
modwrap.d(1440): Error: C style cast illegal, use `cast(intnat)(cast(uintnat)n << 1)`
modwrap.d(1448): Error: C style cast illegal, use `cast(uintnat)n`
modwrap.d(1448): Error: C style cast illegal, use `cast(intnat)(cast(uintnat)n << 1)`
modwrap.d(1448): Error: C style cast illegal, use `cast(char*)caml_callback(*format_result_closure, cast(intnat)(cast(uintnat)n << 1) + 1)`
modwrap.d(1448): Error: C style cast illegal, use `cast(char*)cast(char*)caml_callback(*format_result_closure, cast(intnat)(cast(uintnat)n << 1) + 1)`

source:

#include <stdio.h>
#include <string.h>
#include "caml/mlvalues.h"
#include "caml/misc.h"
#include "caml/memory.h"
#include "caml/alloc.h"
#include "caml/fail.h"
#include "caml/osdeps.h"
#include "caml/callback.h"
#include "caml/custom.h"

/* File modwrap.c -- wrappers around the OCaml functions */

extern(C):
int fib(int n)
{
  static value* fib_closure = null;
  if (fib_closure is null) fib_closure = caml_named_value("fib");
  return Int_val(caml_callback(*fib_closure, Val_int(n)));
}

char * format_result(int n)
{
  static value* format_result_closure = null;
  if (format_result_closure is null)
    format_result_closure = caml_named_value("format_result");
  return strdup(String_val(caml_callback(*format_result_closure, Val_int(n))));
  /* We copy the C string returned by String_val to the C heap
     so that it remains valid after garbage collection. */
}

lines:

extern(C):
int fib(int n)
{
  static value* fib_closure = null;
  if (fib_closure is null) fib_closure = caml_named_value("fib");
  return (cast(int) ((caml_callback(*fib_closure, ((intnat) (((uintnat)(n) << 1)) + 1))) >> 1));
}

char * format_result(int n)
{
  static value* format_result_closure = null;
  if (format_result_closure is null)
    format_result_closure = caml_named_value("format_result");
  return strdup(((char *) ((char *) (caml_callback(*format_result_closure, ((intnat) (((uintnat)(n) << 1)) + 1))))));
}

Error on ZFS's /usr/include/libspl/sys/frame.h

@Laeeth wrote:

converting libzfs, it chokes on /usr/include/libspl/sys/frame.h (to find the problem I had to grep for the offending typedef).
typedefs should only have 1 member, not 2
Cursor(TypedefDecl, "gregset_t", Type(Typedef, "gregset_t"))
[Cursor(TypeRef, "greg_t", Type(Typedef, "greg_t")), Cursor(IntegerLiteral, "", Type(Int, "int"))]

Here is the offending code:

#if defined(_LP64) || defined(_I32LPx)
typedef long greg_t;
#else
typedef int greg_t;
#endif

Bizarrely, it chokes even if I hack the source by doing:

#define _LP64 1

just before the code.

$ include -I /usr/include/libspl libzfs.h /tank2/develop/include/libzfs/libzfs.di

typedefs should only have 1 member, not 2
Cursor(TypedefDecl, "gregset_t", Type(Typedef, "gregset_t"))
[Cursor(TypeRef, "greg_t", Type(Typedef, "greg_t")), Cursor(IntegerLiteral, "", Type(Int, "int"))]

If I hack the include D code by doing the following then it continues on its merry way and appears to finish. The resulting D source doesn't compile yet, but that's probably for other reasons.

You might want to have a treat errors as warnings command-line switch, because in the general case it's more than occasionally much better to do most of the job than just fail. Maybe you could add // FIXME: message to the resulting source.

source/include/translation/typedef_.d

if(!(children.length == 1 ||
       (children.length == 0 && typedef_.type.kind == Type.Kind.Typedef)))
{
        import std.stdio:writeln;
       writeln(text("typedefs should only have 1 member, not ", children.length, "\n", typedef_, "\n", children));
}

Compilation fails due to `alias bool = int;`

I created a mex file to be loaded into Octave just fine using Octave 4.2.2. On an older machine, which has Octave 4.0.0, dpp generated

alias bool = int;

And that won't compile. I was translating one of the examples in the Octave manual to D. The full program is here.

Anonymous struct in anonymous union problem

From libzfs:

 struct _Anonymous_67
    {
        int si_signo;
        int si_errno;
        int si_code;
        int __pad0;
        union _Anonymous_68
        {
            int[28] _pad;
            struct _Anonymous_69
            {
                __pid_t _sifields._kill.si_pid; // can't have a member name with dots in
                __uid_t _sifields._kill.si_uid; // ditto
            }

errors on coreclrhost.h (dotnet core)

[1] laeeth@sheldrake> cat coreclrhost.dpp                                                     /tank2/develop/dotnetcore
#include "coreclrhost.h"

 // For each hosting API, we define a function prototype and a function pointer
// The prototype is useful for implicit linking against the dynamic coreclr
// library and the pointer for explicit dynamic loading (dlopen, LoadLibrary)
//
#define CORECLR_HOSTING_API(function, ...) extern "C" int function(__VA_ARGS__); typedef int (*function##_ptr)(__VA_ARGS__)
    
CORECLR_HOSTING_API(coreclr_initialize, const char* exePath, const char* appDomainFriendlyName, int propertyCount, const char** propertyKeys, const char** propertyValues, void** hostHandle, unsigned int* domainId);
CORECLR_HOSTING_API(coreclr_shutdown,
            void* hostHandle,
            unsigned int domainId);

CORECLR_HOSTING_API(coreclr_shutdown_2,
            void* hostHandle,
            unsigned int domainId,
            int* latchedExitCode);

CORECLR_HOSTING_API(coreclr_create_delegate,
            void* hostHandle,
            unsigned int domainId,
            const char* entryPointAssemblyName,
            const char* entryPointTypeName,
            const char* entryPointMethodName,
            void** delegate);

CORECLR_HOSTING_API(coreclr_execute_assembly,
            void* hostHandle,
            unsigned int domainId,
            int argc,
            const char** argv,
            const char* managedAssemblyPath,
        unsigned int* exitCode);

laeeth@sheldrake> g++ coreclrhost.h                                                           /tank2/develop/dotnetcore
laeeth@sheldrake> d++ --keep-d-files --print-cursors coreclrhost.dpp                          /tank2/develop/dotnetcore
Error: Error parsing 'coreclrhost.h':
error: expected identifier or '('
error: expected identifier or '('
error: expected identifier or '('
error: expected identifier or '('
error: expected identifier or '('

Add ability to rename anonymous enums

Eg:

    enum _Anonymous_268
    {
        VIR_CONNECT_LIST_DOMAINS_ACTIVE = 1,
        VIR_CONNECT_LIST_DOMAINS_INACTIVE = 2,
        VIR_CONNECT_LIST_DOMAINS_PERSISTENT = 4,
        VIR_CONNECT_LIST_DOMAINS_TRANSIENT = 8,
        VIR_CONNECT_LIST_DOMAINS_RUNNING = 16,
        VIR_CONNECT_LIST_DOMAINS_PAUSED = 32,
        VIR_CONNECT_LIST_DOMAINS_SHUTOFF = 64,
        VIR_CONNECT_LIST_DOMAINS_OTHER = 128,
        VIR_CONNECT_LIST_DOMAINS_MANAGEDSAVE = 256,
        VIR_CONNECT_LIST_DOMAINS_NO_MANAGEDSAVE = 512,
        VIR_CONNECT_LIST_DOMAINS_AUTOSTART = 1024,
        VIR_CONNECT_LIST_DOMAINS_NO_AUTOSTART = 2048,
        VIR_CONNECT_LIST_DOMAINS_HAS_SNAPSHOT = 4096,
        VIR_CONNECT_LIST_DOMAINS_NO_SNAPSHOT = 8192,
    }

#include <libvirt>
string enumMapper(string name)
{
  if(name.startsWith("VIR_CONNECT_LIST_DOMAINS"))
    return "VirConnectListDomain";
  else return name;
}

#includepragma(translator,[enumMapper]);

would produce instead of the anonymous enum:

enum VirConnectListDomain
{
  active =1,
  inactive = 2,
...
}

Function parameter of function type not translated correctly if wrapped with parentheses

The following works fine

int binOp(int f(int x, int y), int a, int b);

//becomes

extern(C)
{
    int binOp(int function(int, int), int, int) @nogc nothrow;
}

However if you wrap the function's name that's being passed as an argument in parentheses, no "function" keyword is output

int binOp(int (f)(int x, int y), int a, int b);

//becomes

extern(C)
{
    int binOp(int (int, int), int, int) @nogc nothrow;
}

fails on linux postgres headers from master postgres

note postgres.h is used for server-side extensions to postgres. like this insane poc:
'ssh over postgres'
https://github.com/cybertec-postgresql/pg_remote_exec

postgres2.c

#include "postgres.h"
#include "funcapi.h"
#include "access/hash.h"
#include "utils/builtins.h"
gcc -c -I. postgres2.c   
(success)

cat postgres.dpp
#include "postgres.h"
//#include "funcapi.h"
//#include "access/hash.h"
//#include "utils/builtins.h"

#ifdef PG_MODULE_MAGIC
PG_MODULE_MAGIC;
#endif
[130] laeeth@sheldrake> d++ --include-path . ./postgres2.dpp

Error: Error parsing 'funcapi.h':
error: unknown type name 'Datum'
error: function cannot return function type 'int (FunctionCallInfo)' (aka 'int (struct FunctionCallInfoData *)')
error: unknown type name 'PGFunction'
error: unknown type name 'Oid'
error: unknown type name 'bool'
error: unknown type name 'bool'
error: unknown type name 'MemoryContext'
error: unknown type name 'Oid'
error: unknown type name 'bool'
error: use of undeclared identifier 'FUNC_MAX_ARGS'
error: unknown type name 'bool'
error: use of undeclared identifier 'FUNC_MAX_ARGS'
error: unknown type name 'Oid'
error: unknown type name 'Oid'
error: unknown type name 'MemoryContext'
error: unknown type name 'MemoryContext'
error: unknown type name 'Oid'
error: unknown type name 'int32'
error: unknown type name 'int32'
fatal error: too many errors emitted, stopping now

Anonymous enum declarations clash

3850 in my edited .di file:

    enum
    {
        ILL_ILLOPC = 1,
        ILL_ILLOPN = 2,
        ILL_ILLADR = 3,
        ILL_ILLTRP = 4,
        ILL_PRVOPC = 5,
        ILL_PRVREG = 6,
        ILL_COPROC = 7,
        ILL_BADSTK = 8,
    }
    struct_mntent* getmntent(FILE*, );

line 818 in my edited .di file:

    enum
    {
        ILL_ILLOPC = 1,
        ILL_ILLOPN = 2,
        ILL_ILLADR = 3,
        ILL_ILLTRP = 4,
        ILL_PRVOPC = 5,
        ILL_PRVREG = 6,
        ILL_COPROC = 7,
        ILL_BADSTK = 8,
    }

    alias clock32_t = int32_t;

Unnamed bitfields cause error

I didn't even know this was a thing, but I was testing compiling "more" from util-linux with dpp and got the following line a bunch of times before it threw an error.

Cursor(FieldDecl, "", Type(Int, "int")) CAN DEF @ SourceRange("/usr/include/bits/timex.h", 51:3, 51:11)
...
Fatal error: core.exception.AssertError@/home/chris/Sources/dpp/source/dpp/cursor/aggregate.d(194): Assertion failure

caused by the following struct

struct timex
{
  unsigned int modes;	/* mode selector */
  long int offset;	/* time offset (usec) */
  long int freq;	/* frequency offset (scaled ppm) */
  long int maxerror;	/* maximum error (usec) */
  long int esterror;	/* estimated error (usec) */
  int status;		/* clock command/status */
  long int constant;	/* pll time constant */
  long int precision;	/* clock precision (usec) (read only) */
  long int tolerance;	/* clock frequency tolerance (ppm) (read only) */
  struct timeval time;	/* (read only) */
  long int tick;	/* (modified) usecs between clock ticks */

  long int ppsfreq;	/* pps frequency (scaled ppm) (ro) */
  long int jitter;	/* pps jitter (us) (ro) */
  int shift;		/* interval duration (s) (shift) (ro) */
  long int stabil;	/* pps stability (scaled ppm) (ro) */
  long int jitcnt;	/* jitter limit exceeded (ro) */
  long int calcnt;	/* calibration intervals (ro) */
  long int errcnt;	/* calibration errors (ro) */
  long int stbcnt;	/* stability limit exceeded (ro) */

  int tai;		/* TAI offset (ro) */

  /* ??? */
  int  :32; int  :32; int  :32; int  :32;
  int  :32; int  :32; int  :32; int  :32;
  int  :32; int  :32; int  :32;
};

Apparently this is valid C, I'm guessing used for padding?

convert C arrays used as a return type to ptr

laeeth@sheldrake> cat test.h
char*[] ret(char** arg);
laeeth@sheldrake> bin/include test.h test.di             /tank2/develop/include
laeeth@sheldrake> cat test.di                            /tank2/develop/include
import core.stdc.config;
import core.stdc.stdarg: va_list;
struct struct___locale_data { int dummy; }

char*[] ret(char** arg);

typedef struct should be pointer not value

    struct struct_nvlist_prtctl;
    alias nvlist_prtctl_t = struct_nvlist_prtctl;

should be alias nvlist_prtctl_t=struct_nvlist_prtctl*;

(I think this is your code, not my hack, but not absolutely certain)

need to report which file failed; weird typedef problem

  1. currently when it fails on an assert it doesn't tell you which source file failed. it would be nice to know.

  2. converting libzfs, it chokes on /usr/include/libspl/sys/frame.h (to find the problem I had to grep for the offending typedef).
    typedefs should only have 1 member, not 2
    Cursor(TypedefDecl, "gregset_t", Type(Typedef, "gregset_t"))
    [Cursor(TypeRef, "greg_t", Type(Typedef, "greg_t")), Cursor(IntegerLiteral, "", Type(Int, "int"))]

Here is the offending code:
#if defined(_LP64) || defined(_I32LPx)
typedef long greg_t;
#else
typedef int greg_t;
#endif

Bizarrely, it chokes even if I hack the source by doing:

FIXME

#define _LP64 1

just before the code.

`
laeeth@sheldrake> C_INCLUDE_PATH=/usr/include/libspl:/usr/include/libzfs /tank2/develop/include/bin/include libzfs.h /tank2/develop/include/libzfs/libzfs.di

typedefs should only have 1 member, not 2
Cursor(TypedefDecl, "gregset_t", Type(Typedef, "gregset_t"))
[Cursor(TypeRef, "greg_t", Type(Typedef, "greg_t")), Cursor(IntegerLiteral, "", Type(Int, "int"))]
`

If I hack the include D code by doing the following then it continues on its merry way and appears to finish. The resulting D source doesn't compile yet, but that's probably for other reasons.

You might want to have a treat errors as warnings command-line switch, because in the general case it's more than occasionally much better to do most of the job than just fail. Maybe you could add // FIXME: message to the resulting source.

source/include/translation/typedef_.d

`

if(!(children.length == 1 ||
       (children.length == 0 && typedef_.type.kind == Type.Kind.Typedef)))
{
        import std.stdio:writeln;
       writeln(text("typedefs should only have 1 member, not ", children.length, "\n", typedef_, "\n", children));
}

`

Function pointers throw an error

Example:

// test.h
void (*f)();

//test.dpp
#include "test.h"
#include <stdio.h>

void printHello()
{
    printf("Hello\n");
}

void main()
{
    f = &printHello;
    f();
}

gives

Could not translate cursor Cursor(VarDecl, "f", Type(Pointer, "void (*)(), Type(Unexposed, "void ()")")) sourceRange: SourceRange("test.h", 1:1, 1:12) children: []
Error: Don't know how to translate type Type(FunctionNoProto, "void ()")

The equivalent C code works however

readme typo

under Details

uint16_t foo(uin32_t a);

The output file will contain:

ushort foo(ushort a);

should be
The output file will contain:

ushort foo(uint a);

typedef enum and typedef struct leaks through

Okay - I'm dopey today. I'm using include wrong. I should run it on the pseudo-D source.

Maybe because of my typedef hack to turn the assert into a warning, but probably not.

typedef enum zfs_error {
 EZFS_SUCCESS = 0,
 EZFS_NOMEM = 2000,
};
typedef struct zfs_perm_node {
 avl_node_t z_node;
 char z_pname[4096];
} zfs_perm_node_t;

typedef struct libzfs_handle libzfs_handle_t;

and a few more - strangely only at the last 1k lines of an 11k line file.

Use D array syntax

extern char *zpool_default_import_path[9];
libzfs/libzfs.di(10158): Deprecation: instead of C-style syntax, use D-style syntax char*[9] zpool_default_import_path

Conditional includes maybe messing with compilation

Unfortunately I'm not too sure where the issue with this one is coming from. I'm (still) attempting to compile more from util-linux, but this code sneaks is somehow sneaking in from term.h

typedef struct termtype {
    char *term_names;
    char *str_table;
    char *Booleans;
    short *Numbers;
    char **Strings;


    char *ext_str_table;
    char **ext_Names;

    unsigned short num_Booleans;
    unsigned short num_Numbers;
    unsigned short num_Strings;

    unsigned short ext_Booleans;
    unsigned short ext_Numbers;
    unsigned short ext_Strings;


} TERMTYPE;
typedef struct term TERMINAL;
extern TERMINAL * cur_term;
extern const char * const boolnames[];
extern const char * const boolcodes[];
extern const char * const boolfnames[];
extern const char * const numnames[];
extern const char * const numcodes[];
extern const char * const numfnames[];
extern const char * const strnames[];
extern const char * const strcodes[];
extern const char * const strfnames[];
extern const TERMTYPE * _nc_fallback (const char *);
extern int _nc_read_entry (const char * const, char * const, TERMTYPE *const);

Error: Could not execute `dmd -ofmore more.d`:
more.d(3081): Error: no identifier for declarator `typedef`
more.d(3092): Error: no identifier for declarator `unsigned`
more.d(3093): Error: no identifier for declarator `unsigned`
more.d(3094): Error: no identifier for declarator `unsigned`
more.d(3096): Error: no identifier for declarator `unsigned`
more.d(3097): Error: no identifier for declarator `unsigned`
more.d(3098): Error: no identifier for declarator `unsigned`
more.d(3101): Error: no identifier for declarator `TERMTYPE`
more.d(3102): Error: no identifier for declarator `typedef`
more.d(3102): Error: { } expected following `struct` declaration
more.d(3102): Error: no identifier for declarator `TERMINAL`
more.d(3104): Error: no identifier for declarator `char*`
more.d(3104): Error: no identifier for declarator `boolnames[]`
more.d(3105): Error: no identifier for declarator `char*`
more.d(3105): Error: no identifier for declarator `boolcodes[]`
more.d(3106): Error: no identifier for declarator `char*`
more.d(3106): Error: no identifier for declarator `boolfnames[]`
more.d(3107): Error: no identifier for declarator `char*`
more.d(3107): Error: no identifier for declarator `numnames[]`
more.d(3108): Error: no identifier for declarator `char*`

My only guess is that it has to do with the headers being conditionally included (HAVE_NCURSESW_TERM_H in my case)

#if defined(HAVE_NCURSESW_TERM_H)
# include <ncursesw/term.h>
#elif defined(HAVE_NCURSES_TERM_H)
# include <ncurses/term.h>
#elif defined(HAVE_TERM_H)
# include <term.h>
#endif

Probably not a lot to go off of, let me know if I can provide any more info.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.