dlang / tools Goto Github PK

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18042

User @timotheecour reported (2017-Dec-07 07:20:59):

adding -deps inside rdmd command below will print all dependencies inside bar.foo.fun; it shouldn't because we passed --exclude=bar

./setup.sh:

build_lib(){
        dmd -of=libfoo.a -lib bar/foo.d
}

build_main(){
        exe=./main

        rdmd --force -v --build-only -of$exe -L-lfoo -L-L. --exclude=bar main.d
        $exe
}

./main.d:

import bar.foo;
void main(){ fun(); }

./bar/foo.d:

module bar.foo;

void fun(){
        import std.stdio;
        writeln("ok1");
}

User @marler8997 responded (2018-Jan-10 03:07:11):

Just wanted to make a note that we should retest this bug once this PR is integrated.

#271

User @marler8997 responded (2018-Jan-16 17:37:57):

Fixed with: #271

User @marler8997 responded (2018-Jan-20 02:23:24):

#271 was reverted

Transferred from: https://issues.dlang.org/show_bug.cgi?id=11405

User @MartinNowak reported (2013-Oct-31 16:28:42):

Old discussion but still a valid issue.
http://forum.dlang.org/thread/[email protected]
Rdmd needs a reasonable policy to evict old cached folders.
A simple solution would be to delete an existing folder with the same
first 2-3 bytes of the digest. This means we'd have to lock the folders
to synchronize different rdmd processes.

User @AndrejMitrovic responded (2014-Apr-26 19:31:14):

(In reply to Martin Nowak from comment #0)

A simple solution would be to delete an existing folder with the same
first 2-3 bytes of the digest. This means we'd have to lock the folders
to synchronize different rdmd processes.

I'd hate to make RDMD more complicated, but one idea is to make RDMD read its own config file just like DMD reads its .conf file, and then make this stuff configurable.

User @bearophile responded (2014-Apr-26 19:43:05):

Renamed issue it's => its, hopefully for the better.

User @CyberShadow responded (2017-Jul-08 01:53:45):

With /tmp being a tmpfs on many distros today, and Windows' disk cleanup utility knowing about and deleting the %TEMP% directory on request or when the disk gets full, this may not be as much of an issue nowadays, though still may be worth fixing.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18433

User @marler8997 reported (2018-Feb-13 16:41:59):

rdmd currently ignores DFLAGS. These flags should be treated like all the other compiler flags.

User @wilzbach responded (2018-Mar-27 22:48:04):

Hmm I think the problem is more in DMD not respecting everything from DFLAGS:
However, this has been in master:

$ cat > foo.d << EOF
void main(){
    import std.stdio;
    version(Foo)
    {
        "foo".writeln;
    }
    else
    {
        "bar".writeln;
    }
}
EOF

$ dmd -version=Foo foo.d && ./foo
foo
$ DFLAGS="-version=Foo" dmd foo.d && ./foo
bar
$ DFLAGS="-version=Foo" rdmd --force foo.d
bar

(with ~master this correctly yields foo for all commands)

But it looks like rdmd's caching doesn't include a hash of DFLAGS.

$ DFLAGS="-version=Foo" rdmddev foo.d
foo
$ DFLAGS="-version=Bar" rdmddev foo.d
foo

(so I'm renaming this)

User @wilzbach responded (2018-Mar-27 23:16:22):

Argh, apparently this hasn't been fixed in master and DFLAGS is only looked at when -conf= is set:

$ DFLAGS="-version=Foo" ../dmd/generated/linux/release/64/dmd -v foo.d && ./foo
predefs   DigitalMars Posix linux ELFv1 LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 CRuntime_Glibc D_LP64 D_PIC assert D_HardFloat
binary    ../dmd/generated/linux/release/64/dmd
version   v2.079.0-284-g23b2e2e0d
config    ../dmd/generated/linux/release/64/dmd.conf
DFLAGS    -I../dmd/generated/linux/release/64/../../../../../druntime/import -I../dmd/generated/linux/release/64/../../../../../phobos -L-L../dmd/generated/linux/release/64/../../../../../phobos/generated/linux/release/64 -L--export-dynamic -fPIC

$ DFLAGS="-version=Foo" dmd -conf= -I~/dlang/phobos -I~/dlang/druntime/import -c -v foo.d
predefs   Foo DigitalMars Posix linux ELFv1 LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 CRuntime_Glibc D_LP64 D_PIC assert D_HardFloat
binary    /home/seb/dlang/dmd/generated/linux/release/64/dmd
version   v2.079.0-284-g23b2e2e0d
config
DFLAGS    -version=Foo

User @wilzbach responded (2018-Mar-27 23:31:42):

#343

Transferred from: https://issues.dlang.org/show_bug.cgi?id=2620

User snake.scaly reported (2009-Jan-26 18:00:00):

coffimplib, when run with only one library as an argument, usually complains about not being able to overwrite the library. Nevertheless it will silently overwrite the library if its extension isn't all-lowercase. This happens quite often to me because most of libraries in Microsoft Platform SDK are capitalized this way.

$ coffimplib bufferoverflowu.lib
output file 'bufferoverflowu.lib' would overwrite input file, use -f to force it
$ coffimplib Certadm.lib
output file 'Certadm.lib' would overwrite input file, use -f to force it
$ coffimplib ComDlg32.Lib
(nothing)

CC @WalterBright : You're the only one that can handle such bugs as they are on DigitalMars tools.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18040

User @timotheecour reported (2017-Dec-07 06:07:41):

I don't see any reason why --exclude= --include= should be only allowing packages, it should also allow modules, eg:
rdmd --exclude=bar main.d
=> will exclude package bar or module bar whichever is defined

Eg, there's no way currently to exclude a single module. This will allow this as well as other use cases.

User @marler8997 responded (2018-Jan-10 03:07:35):

I think this will be fixed with: #271

User @marler8997 responded (2018-Jan-16 17:39:40):

#271

User @marler8997 responded (2018-Jan-20 02:22:53):

#271 was reverted

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18421

User @WebDrake reported (2018-02-11 15:21:59 CET):

rdmd_test includes a test-case to ensure that rdmd works for programs that use pragma(lib) to load libraries (see https://issues.dlang.org/show_bug.cgi?id=16978 for the motivation). The correctness sought here is not in any direct handling of pragma(lib), but in ensuring that the way rdmd invokes the D compiler will allow the pragma(lib) call to succeed. Since GDC does not support pragma(lib), the test case will fail when gdmd is used as the D compiler invoked by rdmd. As this is a fault in the compiler rather than rdmd, the test case can simply be disabled when testing with gdmd (since from rdmd's point of view, it suffices to demonstrate that it works with D compilers that support pragma(lib)). However, this is merely a workaround, not a fix. Long-term, the test-case should be re-enabled as soon as GDC support for pragma(lib) is available. This issue has been created in order to ensure that this task does not get forgotten ;-)

Marking as enhancement since we don't do that level of testing at the moment.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=16431

User @wilzbach reported (2016-Aug-25 20:50:41):

$ python -m timeit -v -n 10 -s 'import os' 'os.system("dub run --single --force foo.d")'
10 loops, best of 3: 578 msec per loop

$ python -m timeit -v -n 10 -s 'import os' 'os.system("rdmd --force foo.d")'
10 loops, best of 3: 730 msec per loop

I used an example from the DLang Tour (see below or http://tour.dlang.io/tour/en/gems/template-meta-programming) and tested with more and the overhead seems to stay there:

== Associative arrays ==

http://tour.dlang.io/tour/en/basics/associative-arrays

rdmd 5 loops, best of 3: 637 msec per loop
dub: 5 loops, best of 3: 584 msec per loop

== Loops ==

http://tour.dlang.io/tour/en/basics/loops

rdmd: 5 loops, best of 3: 488 msec per loop
dub: 5 loops, best of 3: 373 msec per loop

#!/usr/bin/env rdmd
/+ dub.sdl:
name "foo"
+/

import std.traits: isFloatingPoint;
import std.uni: toUpper;
import std.string: format;
import std.stdio: writeln;

/// A Vector that just works for
/// numbers, integers or floating points.
struct Vector3(T)
  if (is(T: real))
{
private:
    T x,y,z;

    /// Generator for getter and setter because
    /// we really hate boiler plate!
    ///
    /// var -> T getVAR() and void setVAR(T)
    mixin template GetterSetter(string var) {
        // Use mixin to construct function
        // names
        mixin("T get%s() const { return %s; }"
          .format(var.toUpper, var));

        mixin("void set%s(T v) { %s = v; }"
          .format(var.toUpper, var));
    }

    // Easily generate our getX, setX etc.
    // functions with a mixin template.
    mixin GetterSetter!"x";
    mixin GetterSetter!"y";
    mixin GetterSetter!"z";

public:
    // We don't allow the dot function
    // for anything but floating points
    static if (isFloatingPoint!T) {
        T dot(Vector3!T rhs) {
            return x*rhs.x + y*rhs.y +
                z*rhs.z;
        }
    }
}

void main()
{
    auto vec = Vector3!double(3,3,3);
    // That doesn't work because of the template
    // constraint!
    // Vector3!string illegal;

    auto vec2 = Vector3!double(4,4,4);
    writeln("vec dot vec2 = ", vec.dot(vec2));

    auto vecInt = Vector3!int(1,2,3);
    // doesn't have the function dot because
    // we statically enabled it only for float's
    // vecInt.dot(Vector3!int(0,0,0));

    // generated getter and setters!
    vecInt.setX(3);
    vecInt.setZ(1);
    writeln(vecInt.getX, ",",
      vecInt.getY, ",", vecInt.getZ);
}

User @aG0aep6G responded (2016-Aug-25 21:26:33):

I don't think this is much of a surprise. rdmd detects dependencies. dub doesn't seem to do that.

User @wilzbach responded (2016-Aug-26 02:00:56):

I don't think this is much of a surprise. rdmd detects dependencies. dub doesn't seem to do that.

Well, it still seems unnecessary to me to run the full-blown CTFE compiler twice on a file. I gave it a quick try with two hacks:

#191

User @aG0aep6G responded (2016-Aug-26 13:08:02):

(In reply to greensunny12 from comment 2)

Well, it still seems unnecessary to me to run the full-blown CTFE compiler
twice on a file.

Sure. If you make rdmd faster, that's great. I just don't think that dub has anything to do with it.

User @AndrejMitrovic responded (2016-Aug-31 21:30:26):

Fixed right? Closing.

User @wilzbach responded (2017-Jul-20 20:07:38):

PR was reverted :/

User @marler8997 responded (2017-Dec-14 16:37:33):

Potential fix: #271
"Use "-i" to prevent rdmd from having to invoke compiler twice."

Depends on this first: dlang/dmd#7099
"Enable automatic compiling of imports"

User @marler8997 responded (2018-Jan-17 02:39:18):

#271

User @marler8997 responded (2018-Jan-20 02:21:14):

#271 was reverted

Transferred from: https://issues.dlang.org/show_bug.cgi?id=17838

User @WebDrake reported (2017-Sep-18 20:02:36):

rdmd uses spawnProcess to launch the D compiler of choice, and passes a custom File instance to the stdout parameter.

This means that the D compiler has to inherit a file-descriptor from rdmd. For traditional Linux packages, this is not a problem. However, where snap packages are concerned, this creates an issue: when the D compiler is itself a snap package, its confinement rules prevent it from inheriting the file descriptor, and rdmd will exit with an error message:

$ rdmd --compiler=ldc2 hello.d
Failed to flush stdout: Permission denied
Failed: ["ldc2", "-v", "-o-", "hello.d", "-I."]

Some extended discussion of this issue can be found in this thread on the snapcraft forums:
https://forum.snapcraft.io/t/failed-to-flush-stdout-permission-denied/485

This is considered at least partly a bug in snapd, hence I've held off on reporting it as an rdmd issue, but it looks like this is going to be long-lived enough for me to think it might be worth addressing in rdmd itself if possible.

A recent patch to rdmd (f7e6f4e) partially addresses this issue, as it ensures that when rdmd is invoking a D compiler within the same snap package as itself, the confinement rules do not come into play (because the compiler executable is launched directly, instead of indirectly via the wrapper script that the snap package exposes to the host system).

However, this still leaves unaddressed the broader issue of rdmd invoking a compiler provided by a separate snap package (e.g. the ldc2 snap package).

So: can rdmd be either patched or invoked in order to avoid the D compiler it invokes needing to inherit a file descriptor from it? :-)

User @wilzbach responded (2018-Mar-24 07:32:32):

So: can rdmd be either patched or invoked in order to avoid the D compiler it invokes needing to inherit a file descriptor from it? :-)

Yes it can (in theory). With the improvements in 2.079, DMD now supports JSON output via stdout (see e.g. https://run.dlang.io/is/DiSyCl).
That's what #292 was supposed to do.
With the JSON output, no temporary .deps file would be needed and the stdout could be parsed directly, but I think 292 would just instruct dmd to save the JSON output to the temp folder, s.t. no further complications are needed.

So: can rdmd be either patched

Sure, have you tried patching spawnProcess out with
a) execute and saving the output manually
b) a custom implementation that uses real files instead of file pointers?

User @WebDrake responded (2018-Mar-24 13:25:53):

With the improvements in 2.079, DMD now supports JSON output via stdout

That's hardly a viable solution. The problem only arises when one attempts to specify a different D compiler with the --compiler flag, and that compiler is itself snap-packaged.

That would usually mean one is trying to use a different compiler than dmd -- i.e. ldmd2 or gdmd -- neither of which uses the 2.079 frontend yet. And even allowing for that, it's desirable to find a solution that does NOT work only for future compiler releases.

We have to get away from this notion that a release of rdmd only has to work with the dmd it's bundled with, and that it can ignore the question of working properly with other D compilers on the same system, or D compilers using an older frontend version.

This is ultimately an upstream bug that causes problems with the --compiler flag in a narrow context. We shouldn't work around it by causing problems with the --compiler flag in almost EVERY context.

Sure, have you tried patching spawnProcess out with
a) execute and saving the output manually

Yes, but with limited success, probably because I didn't find a way that worked with spawnProcess. Probably the way forward here is to use spawnShell instead, and to request that stdout be piped to a specified file name. That might also address other issues that people have had (e.g. one person mentioned problems on Windows using a dmd.bat script that wrapped the true dmd compiler).

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18079

User @marler8997 reported (2017-Dec-14 16:25:16):

Related to (https://issues.dlang.org/show_bug.cgi?id=8858). That bug was created because dmd would not discover all the imported modules when compiling a single file. However, this was closed as a "non issue" because the compiler is not "supposed" to discover all dependencies when compiling a single module. If an application wants to discover all imported modules, then they can use the "-deps" flag to force the compiler to do so.

rdmd currently uses this "import discovery" to obtain a list of all the modules it needs to compile. However, rdmd does not use the "-deps" flag when discovering dependencies which means that in some cases it will be missing modules to compile, i.e.

main.d

import foo;
void main() { printx(); }

foo.d

module foo;
void printx() { import std.stdio; import bar : x; writefln("x is %s", x); }

bar.d

module bar;
int x = 42;

rdmd main.d

this could be fixed by modifying rdmd to use the "-deps" flag, however, this has a significant performance cost. Another solution proposed is to integrate the "compile imports" feature (dlang/dmd#7099) and modify rdmd to use that feature instead (#271).

User @marler8997 responded (2018-Jan-10 03:06:04):

This bug should be fixed with this PR.

#271

User @marler8997 responded (2018-Jan-16 17:39:51):

#271

User @marler8997 responded (2018-Jan-20 02:04:42):

#271 was reverted

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18122

User @wilzbach reported (2017-Dec-24 22:22:50):

Would be very interesting for people who want to know the details. In most cases the changelog every is added as part of the PR, so it should be possible to query this directly from the git log in the tools/changed.d script.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18400

User @wilzbach reported (2018-02-08 21:02:54 CET):

See also: https://github.com/dlang/dmd/tree/master/docs and https://github.com/dlang/tools/blob/master/man/man1/rdmd.1

Transferred from: https://issues.dlang.org/show_bug.cgi?id=19382

User Basile-z reported (2018-11-09 16:00:38 CET):

for the D tools

make -f posix.mak fails when the dmd available on the machine is not the one built from sources.

we are aware that make -f posix.mak DMD=dmd works but people should not have to do that.

reported because of

https://www.reddit.com/r/d_language/comments/9vgf6j/building_dlangtools_error_127/

User Basile-z responded (2018-11-09 16:02:27 CET):

it's rather that by default it doesn't work when there's no generated dmd from the tools repository.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=7677

User @WalterBright reported (2012-Mar-09 11:41:05):

$ obj2asm /bin/ls
Fatal error: unrecognized flag '/bin/ls'

Reported:

http://www.reddit.com/r/programming/comments/qlssq/one_small_step_toward_reducing_programming/c3z7h7v

Transferred from: https://issues.dlang.org/show_bug.cgi?id=8173

User @thelastmammoth reported (2012-May-31 02:23:50):

Currently, running:

$ rdmd -g --build-only main &&  gdb main

yields:

warning: Could not find object file "/tmp/.rdmd-503/rdmd-main.d-.../main.o" - no debug information available for "main.d".

(but using the manual way with dmd works... with the names mangled but that's a different issue )

perhaps it would be possible to preserve debug info when -g (or -debug, etc) is present as opposed to having -o- (which appears when --chatty is on) ?

User @CyberShadow responded (2013-Mar-10 01:17:44):

(In reply to comment #0)

Currently, running
rdmd -g --build-only main && gdb main

yields:
warning: Could not find object file "/tmp/.rdmd-503/rdmd-main.d-.../main.o" -
no debug information available for "main.d".

What OS is this on? You specified "All" but it does not apply to Windows and does not happen on Linux.

perhaps it would be possible to preserve debug info when -g (or -debug, etc) is
present as opposed to having -o- (which appears when --chatty is on) ?

I can't make out any meaning from the above.

Judging by the error message, the problem is that rdmd deletes object (.o) files after a successful build.

Tried this today on OSX and could reproduce the issue. The dwarfdump output seems correct but the debugger (lldb in my case) seems not to be able to link it to the source code (e.g. b _Dmain gives ASM output.)

I want this or something like this to render a video (cause ffmeg commands are nightmares and I need to coordinate the size between the program and the ffmeg command)

#!rdmd gradienttest.d | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 255x255 -i pipe: -preset ultrafast out.mp4
import std;
void main(){
	foreach(i;0..255){
	foreach(y;0..255){
	foreach(x;0..255){
		stdout.rawWrite([cast(ubyte)x,cast(ubyte)y,cast(ubyte)i]);
	}}}
}

Given that rn you handle a magic "--shebang"

Id think a magic "--dumbpipe" argument seems fine

included code for what Id expect

import std;
void main(string[] args){
	assert(args[1][0..10]=="--dumbpipe");
	writeln("got args: ",args);
	string getarg_(int i){
		if(i==0){
			return args[2][2..$-2];//this seems like a hack even to me, but im unsure what edge cases to try to fix
		}
		if(i+2>=args.length){
			return "";
		}
		return args[i+2];
	}
	string getarg(int i){//debug info, remove if hack above is fixed?
		auto s=getarg_(i);
		writeln("parsing $",i," as ",s);
		return s;
	}
	string command;
	command~="dmd -i -run ";
	string s=args[1][11..$];
	while(s.length!=0){
		if(s[0]=='$' && s.length>1 && s[1].isDigit){
			command~=getarg([s[1]].to!int);
			s=s[2..$];
		} else {
			command~=s[0];
			s=s[1..$];
		}
	}
	writeln("running: ",command);
}

#!myrdmd --dumbpipe $0 | ffmpeg -f rawvideo -pixel_format rgb24 -video_size $1x$2 -i pipe: -preset ultrafast $0.mp4

./testmyrdmd.d 255 255
got args: ["myrdmd", "--dumbpipe $0 | ffmpeg -f rawvideo -pixel_format rgb24 -video_size $1x$2 -i pipe: -preset ultrafast $0.mp4", "./testmyrdmd.d", "255", "255"]
parsing $0 as testmyrdmd
parsing $1 as 255
parsing $2 as 255
parsing $0 as testmyrdmd
running: dmd -i -run testmyrdmd | ffmpeg -f rawvideo -pixel_format rgb24 -video_size 255x255 -i pipe: -preset ultrafast testmyrdmd.mp4

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18490

User @timotheecour reported (2018-02-22 11:42:12 CET):

see dlang/dlang.org#2169 (review) for context

segfault on:

+        $(SPEC_RUNNABLE_EXAMPLE_RUN
+        ---
+        size_t fun(T...)(T t, string file = __FILE__)
+        {
+            import std.stdio;
+            writeln(file, " ", t);
+            return T.length;
+        }
+        assert(fun(1, "foo") == 2);  // not 1 (IFTI)
+        assert(fun!int(1, "foo") == 1);  // IFTI
+        ---

Tentatively putting it as POSIX according to the link

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18423

User @WebDrake reported (2018-02-11 16:22:46 CET):

When rdmd invokes the D compiler to generate dependencies information, it makes a hardcoded assumption that this information gets written by the compiler to stdout. However, this assumption is not valid for all D compilers. In particular, compiler frontend messages are written out to a configurable stdmsg: dlang/dmd@dc8421f

... and this configurability is used by GDC to output such messages to stderr (reflecting GCC policy that stdout is reserved for -pipe output; see: #297 (comment)).

This means not only that rdmd will not work with more recent GDC releases, but that rdmd's assumptions about stdout are not valid in general (given the configurable stdmsg in the DMD frontend).

The gory details

The getDependencies function inside rdmd.d invokes the D compiler via the run command, passing in options (the depsGetter variable) and the name of a file to which to write the compiler output (depsFilename):

immutable depsExitCode = run(depsGetter, depsFilename);

Internally, run takes this and uses spawnProcess to invoke the D compiler:

private int run(string[] args, string output = null, bool replace = false)
{
    ...

    File outputFile;
    if (output.ptr)
        outputFile = File(output, "wb");
    else
        outputFile = stdout;
    auto process = spawnProcess(args, stdin, outputFile);
    return process.wait();
}

In other words, run opens a file whose name is given by depsFilename, and uses it to replace stdout for the spawned process. The contents of this file are then read using the nested readDepsFile() function defined internally inside getDependencies.

In consequence, if the information parsed by readDepsFile is not written to stdout by the D compiler, it will not get written into outputFile, and therefore not be available to parse.

User @WebDrake responded (2018-02-11 16:30:16 CET):

For context, this issue was observed while trying to update rdmd's integration tests to use gdmd: #307 (comment)

Marking as enhancement since things work at the moment and this is to support a new feature.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18402

Reported by @timotheecour (2018-02-08 21:21:34 CET), confirmed by me.
Timothee's report was:

rdmd: make -f posix.mak -j8 test => core.exception.AssertError@rdmd_test.d(373): Assertion failure

Tested today with -j8.

% make -f posix.mak test
for file in ascii iteration ; do \
		generated/osx/64/tests_extractor -i "./test/tests_extractor/${file}.d" | diff -p - "./test/tests_extractor/${file}.d.ext"; \
	done
parsing test/tests_extractor/ascii.d
parsing test/tests_extractor/iteration.d
generated/osx/64/tests_extractor -a betterc -i "./test/tests_extractor/attributes.d" | diff -p - "./test/tests_extractor/attributes.d.ext";
parsing test/tests_extractor/attributes.d
generated/osx/64/tests_extractor --betterC -i "./test/tests_extractor/betterc.d" | diff -p - "./test/tests_extractor/betterc.d.ext";
parsing test/tests_extractor/betterc.d
generated/osx/64/rdmd_test generated/osx/64/rdmd -m64 \
	   --rdmd-default-compiler=../dmd/generated/osx/release/64/dmd \
	   --test-compilers=../dmd/generated/osx/release/64/dmd \

object.Exception@rdmd_test.d(696): /var/folders/1_/wzk0c3d17t37j8r1q8_3zl840000gn/T/dmd would not have been executed
----------------
??:? pure @safe void std.exception.bailOut!(Exception).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x105fa83be]
??:? pure @safe bool std.exception.enforce!().enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x105faa7d4]
??:? void rdmd_test.runFallbackTest(immutable(char)[], immutable(char)[], immutable(char)[]) [0x105fa80fe]
??:? void rdmd_test.runCompilerAgnosticTests(immutable(char)[], immutable(char)[], immutable(char)[]) [0x105fa1146]
??:? _Dmain [0x105fa06fa]
make: *** [test_rdmd] Error 1

% dmd | head -n 1
DMD64 D Compiler v2.091.0
% git show HEAD
[...]
a1da2705dcffb4955f8065aee01a08602ad8216a

Tested on Mac OSX Catalina.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=17569

User @quickfur reported (2017-Jun-28 17:49:19):

This was reported on the forum:

Installing dmd if debian-goodies is installed fails. Both try to write a file
named '/usr/bin/dman'

[...snip...]

$ sudo dpkg -i dmd_2.074.1-0_amd64.1.deb
(Reading database ... 224610 files and directories currently installed.)
Preparing to unpack dmd_2.074.1-0_amd64.1.deb ...
Unpacking dmd (2.074.1-0) ...
dpkg: error processing archive dmd_2.074.1-0_amd64.1.deb (--install):
 trying to overwrite '/usr/bin/dman', which is also in package debian-goodies 0.74
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

$ sudo dpkg -i dmd_2.075.0~b1-0_amd64.deb
(Reading database ... 224610 files and directories currently installed.)
Preparing to unpack dmd_2.075.0~b1-0_amd64.deb ...
Unpacking dmd (2.075.0~b1-0) ...
dpkg: error processing archive dmd_2.075.0~b1-0_amd64.deb (--install):
 trying to overwrite '/usr/bin/dman', which is also in package debian-goodies 0.74
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

The package debian-goodies also ships a binary named dman, with this description:

  dman               - Fetch manpages from online manpages.debian.org service

Since this is something shipped with Debian itself, I recommend either (1) renaming our dman utility to something less ambiguous, or (2) excluding it from the .deb package for dmd altogether.

And on that note, just in case, I highly recommend against putting a Conflicts: line in the debian/control file, because there is no good reason why the debian-goodies package should be excluded by the dmd package. If anything, we should be the ones to give up the dman utility (I have never found a need for it, and I question whether anyone actually uses it besides the D developers).

I'm marking this as a major bug, because the conflict means that any Debian user who wants to try out D for the first time may get immediately turned away by the installation failure due to some silly marginal utility being unfortunately named, just because he happens to have the debian-goodies package installed.

User @MartinNowak responded (2017-Aug-08 20:37:47):

Another reported conflict is deepin-manual http://forum.dlang.org/post/[email protected].

User @WalterBright responded (2017-Aug-10 20:19:48):

Just rename it to dmanual

Transferred from: https://issues.dlang.org/show_bug.cgi?id=17216

User @timotheecour reported (2017-Feb-21 05:06:39):

v2.072.2
rdmd build 20170108

fun.d:whatever

rdmd -v --force --build-only -shared -oflibfun.dylib fun.d

creates libfun.dylib

otool -L libfun.dylib
libfun.dylib:
        libfun.dylib.tmp (compatibility version 0.0.0, current version 0.0.0)

libfun.dylib.tmp is wrong, and will cause errors when linking via -lfun, eg:
dyld: Library not loaded: libfun.dylib.tmp

This used to work on OSX (and works on linux)

User @timotheecour responded (2017-Feb-21 05:17:26):

same with latest:
DMD64 D Compiler v2.073.1
rdmd build 20170220

User @timotheecour responded (2017-Feb-21 05:33:14):

workaround:
-L-install_name -Llibfun.dylib

User @cqexbesd responded (2017-May-07 13:44:23):

This is still occurring in HEAD.

When rdmd calls dmd it gives an output name that ends with ".tmp" (https://github.com/dlang/tools/blob/master/rdmd.d#L483-L488) and that is what dmd passes to clang (used for linking under OSX). As the install_name arg isn't given thats what ends up being the install_name of the dylib and hence this problem.

There are two obvious ways of fixing it - get dmd to call clang with install_name or run install_name_tool after building.

Neither seem obviously great - IIUC rdmd doesn't know its building a shared library so doesn't know install_name is important. dmd does know but isn't aware that rdmd has passed it a dummy output name that will later get mv'ed to the final name.

I don't yet know if rdmd needs to use the .tmp then mv trick but I'm guessing its there to cope with some sort of error on some platform - if that could be avoided the problem would disappear. Anything else seems to require rdmd learning about shared libraries under OSX or dmd having more awareness of rdmd.

User @cqexbesd responded (2017-May-07 13:49:35):

OK .tmp is added at 085dc5d to work around a Windows problem so removing it without an alternative wouldn't be good.

User @cqexbesd responded (2017-May-07 14:29:52):

I now have a fix at https://github.com/cqexbesd/tools/tree/dlang_17216.

It's not great - it just makes the windows work around only happen under Windows.

Given https://github.com/cqexbesd/tools/blob/d27bcc1228863ba0a3b0ae5cf47a8d3a409bda45/rdmd.d#L461-L481 however I'm not sure that the windows workaround should still be needed. The rm code didn't exist when the workaround was added AFAIK. I might try removing the workaround and finding a windows user who can test it.

User @cqexbesd responded (2017-May-07 14:37:38):

Second proposed fix (removing the use of .tmp altogether) is at https://github.com/cqexbesd/tools/tree/dlang_17216_v2. I don't have a copy of Windows so it needs testing - but it works under OSX.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=19664

User @jacob-carlborg reported (2019-Feb-10 10:13:10):

All tools shipped with a DMD release should be built for the same architecture. For a 64 bit release, the following tools are built for 32 bit: "dumpobj", "obj2asm" and "shell". This is extra important for macOS since Mojave is the last release where 32 bit applications will work and with DMD 2.085.0 we have deprecated the support for 32 bit on macOS.

User @jacob-carlborg responded (2019-Jul-14 15:58:38):

Raising this to blocker since the public beta of macOS Catalina is out where 32 bit programs don't run.

CC @WalterBright : Those tools are distributed as by DigitalMars and not actually part of this repository, so you're the only one able to handle this AFAIK.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=18401

User @wilzbach reported (2018-02-08 21:05:07 CET):

These pages have a tendency to get outdated:

• https://github.com/dlang/tools/blob/master/man/man1/rdmd.1
• https://dlang.org/rdmd.html

We should generate them like we do for dmd. They are even outdated atm - search for --shebang on the man page.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=11827

User @andralex reported (2013-Dec-26 12:08:04):

#64 introduces this. Basically calls like exists("filename") and isDir("filename") make one stat call each, whereas calling stat once offers all information. The extra calls are not recorded with yap() either.

Excessive stat() calls are a major scalability issue.

Arguably the best fix would be to offer a nice API as noted in the pull request.

module source;

import core.sys.windows.windef;

extern (Windows)
BOOL DllMain(HINSTANCE hInstance, ULONG ulReason, LPVOID pvReserved) nothrow {
    return 0;
}

> rdmd -shared --build-only source.d

$ dir
> 2022-11-22  02:45 PM            80,896 source.exe

--chatty shows that RDMD used -of to specify a file name with an .exe extension:

spawn ["C:\Apps\DMD\dmd2\windows\bin\dmd.exe", "-shared", "-of.\source.exe.tmp", "-odC:\Users\\AppData\Local\Temp\.rdmd\rdmd-source.d-2F65A2D92BA75097EBFECC7FE8CC5156\objs", "-I.", "source.d"]
stat C:\Users<user>\AppData\Local\Temp.rdmd\rdmd-source.d-2F65A2D92BA75097EBFECC7FE8CC5156\objs
rmdirRecurse C:\Users<user>\AppData\Local\Temp.rdmd\rdmd-source.d-2F65A2D92BA75097EBFECC7FE8CC5156\objs
mv .\source.exe.tmp .\source.exe

Using -of manually is a workaround. But I think RDMD should try to be more intelligent about the default output file name based on whether -shared is passed in or not.

Describe the bug
windows has a helpful compatibility feature where having the words "setup" or "install" in an executable's name automatically causes it to require admin privileges to run

this affects rdmd because it names the compiled executable after the source file

D source files with those words in the name fail to run through rdmd with The requested operation requires elevation.

the trigger words are case-insensitive and there might be more of them

Platform
OS: Windows 7 64-bit
Compiler: DMD 2.098.0

To Reproduce
in command prompt:

1. echo void main(){} > setup.d
2. rdmd setup.d

C:\Users\qemu\Desktop>rdmd setup.d

std.process.ProcessException@std\process.d(1360): Failed to spawn process "C:\Users\qemu\AppData\Local\Temp\.rdmd\rdmd-setup.d-8A16DDD3DC3B0E380C43708A28BDC4D0\setup.exe" (The requested operation requires elevation.)
----------------
0x004661EC
0x004531B0
0x004044AA
0x0040452E
0x00467927
0x004678A1
0x0046773A
0x00454352
0x00405B28
0x764E344D in BaseThreadInitThunk
0x76FD9802 in RtlInitializeExceptionChain
0x76FD97D5 in RtlInitializeExceptionChain

one workaround to avoid the behavior is to set the __COMPAT_LAYER=RunAsInvoker environment variable, but that might be unacceptable for rdmd since it applies to subprocesses of the D program too. i don't know if there's a way to start just one child process with the setting

Transferred from: https://issues.dlang.org/show_bug.cgi?id=17071

User @timotheecour reported (2017-Jan-08 02:57:48):

$ rdmd -deps=foo.deps -c foo.d
#creates foo.deps foo.o
$ rm foo.deps

$ rdmd -deps=foo.deps -c foo.d
# BUG: foo.deps not recreated

$ rdmd --force -deps=foo.deps -c foo.d
#ok

Transferred from: https://issues.dlang.org/show_bug.cgi?id=15245

User @WebDrake reported (2015-10-25 13:19:03 CET):

None of the D tools adequately supports the version flag:

(1) rdmd supports the --version flag but gives only the day of build (e.g. 20151025) rather than the actual version tag;

(2) catdoc, changed, dget, dustmite and tolf all exit with an uncaught exception;

(3) ddmangle reports an unrecognized option and prints help.

Since the tools repository is subject to similar version tagging to dmd, druntime and phobos, it would seem appropriate that the tools support a --version flag with similar output to dmd --version.

User @CyberShadow responded (2015-10-25 23:49:22 CET):

At least DustMite is a separate project that is included with DMD for convenience, thus is not tied to the DMD release schedule. It does not have a version because it has no releases - a release is effectively a push to GitHub.

User @WebDrake responded (2015-10-25 23:59:39 CET):

It does not have a version because it has no releases - a release is effectively a push to GitHub.

... and yet the repo is tagged with dmd release numbers.

I'm not suggesting the tools have to be tied to dmd release schedule, but some kind of support of --version would make sense, even if it's just to report the hash and date of the most recent commit.

User @CyberShadow responded (2015-10-26 00:01:32 CET):

... and yet the repo is tagged with dmd release numbers.

The DustMite repo isn't. The code in the D-P-L tools repo is a copy from https://github.com/CyberShadow/DustMite , updated periodically.

Transferred from: https://issues.dlang.org/show_bug.cgi?id=14654

User @andralex reported (2015-Jun-05 16:10:40):

rdmd builds the given file and all of its dependents in one shot. For large projects, this becomes problematic because of many files are built unnecessarily if one file is touched.

One emerging pattern is to build one package (= directory) at a time, Haskell/Rust style. That allows users to organize dependencies in a scalable manner.

rdmd would collect the dependency graph, "color" the files have been affected by the changed files since last build, and then carve the entire file set by directory. If at least one file has been colored in a directory, that directory needs to be rebuilt.

This is a high impact project because it would instantly improve the lot of all rdmd users, and scales rdmd up for larger builds.

User @CyberShadow responded (2015-Jun-06 16:16:41):

If we want to improve build times with rdmd, the best thing we can do is get rid of rdmd (and only leave it as a thin wrapper around dmd). This is because rdmd needs to run dmd -o- to get the dependency graph, and then run dmd again to actually build the program. This means that lexing, parsing, semantics - everything except code gen and linking - must happen twice. For template or mixin-heavy code, this nearly doubles compilation time.

User @andralex responded (2015-Jun-06 16:46:38):

(In reply to Vladimir Panteleev from comment 1)

If we want to improve build times with rdmd, the best thing we can do is get
rid of rdmd (and only leave it as a thin wrapper around dmd). This is
because rdmd needs to run dmd -o- to get the dependency graph, and then run
dmd again to actually build the program. This means that lexing, parsing,
semantics - everything except code gen and linking - must happen twice. For
template or mixin-heavy code, this nearly doubles compilation time.

This improvement stands.

User @CyberShadow responded (2015-Jun-06 16:48:24):

Yes, they are orthogonal. But:

1. It is potentially higher-impact. Even with per-package compilation, DMD will still need to lex/parse/etc. ALL modules.

2. Any improvements we make now to rdmd may be harder to migrate into dmd

User @andralex responded (2015-Jun-06 16:58:03):

(In reply to Vladimir Panteleev from comment 3)

Yes, they are orthogonal. But:

1. It is potentially higher-impact. Even with per-package compilation, DMD
   will still need to lex/parse/etc. ALL modules.

It's almost surprising rdmd is so immensely useful, eh :o).

BTW there was work on collecting dependencies while actually building. One nice solution would be for dmd -v to generate a line at the end of imports to mean "done". Then rdmd can run dmd concurrently and terminate if no need to generate code.

2. Any improvements we make now to rdmd may be harder to migrate into dmd

I see it as the polar opposite. Improvements in rdmd clarify what sort of primitives and interfaces we need for highly modular compilation.

User @aG0aep6G responded (2015-Jun-14 22:52:52):

I implemented an alternative mentioned in the forum discussion: Compile all outdated files, and only those, with one compiler invocation to multiple object files.

#170

Transferred from: https://issues.dlang.org/show_bug.cgi?id=15533

User golovanov_alexey reported (2016-Jan-09 09:34:01):

Created attachment 1572

reduced test case (see moduleB import)

From forum: http://forum.dlang.org/thread/[email protected]

OS: Windows 7 (32 bit)
DMD: 2.069.2 and 2.070.0-b1
RDMD: build 20160103

Problem: import inside function produced linker error then -g switch used.
Test files in attach:

moduleA.d
test\moduleB.d
test\moduleC.d

module moduleA;

public void funcA () {
  import test.moduleB: funcB;
  return;
}

void main(string[] args) {
  return;
}

module test.moduleB;

//import test.moduleC: funcC;  //this works

public void funcB () {
  import test.moduleC: funcC;  //linker error
  funcC();

  return;
}

module test.moduleC;

public void funcC () {
  return;
}

C:\dmd2\windows\bin\rdmd.exe moduleA.d
//no errors, moduleA.exe created


C:\dmd2\windows\bin\rdmd.exe -g moduleA.d
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
C:\Users\user\AppData\Local\Temp\.rdmd\rdmd-moduleA.d-D3D7676836384146D1D9D907CF20EC26\objs\moduleA.exe.obj(moduleA.exe)
 Error 42: Symbol Undefined _D4test7moduleC5funcCFZv
--- errorlevel 1

Problem disappear, then i move import in test.moduleB to global level:

module test.moduleB;

import test.moduleC: funcC;  //this works

public void funcB () {
  //import test.moduleC: funcC;  //linker error
  funcC();

  return;
}

User golovanov_alexey responded (2016-Jan-16 09:34:43):

RDMD in both cases produces same rdmd.deps file in corresponding work dir.
And those rdmd.deps not contains any mention of moduleC.

Command dmd -deps=moduleA.deps moduleA.d
produces moduleA.deps file, which not contains any mention of moduleC too.

Is this DMD bug instead of RDMD ?

User golovanov_alexey responded (2016-Jan-16 16:57:47):

It seems this is old DMD bug
"https://issues.dlang.org/show_bug.cgi?id=7016 - local import does not create -deps dependency"

User golovanov_alexey responded (2017-May-20 10:36:09):

No. Issue 7016 fixed, but this 15533 sill exists.

DMD32 D Compiler v2.075.0-master-f546675.
Reduced test case.

C:\dmd2\windows\bin\rdmd.exe moduleA.d
//no errors, moduleA.exe created


C:\dmd2\windows\bin\rdmd.exe -g moduleA.d
OPTLINK (R) for Win32  Release 8.00.17
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
C:\Users\gav\AppData\Local\Temp\.rdmd\rdmd-moduleA.d-8C1729D9769E4E70D93C65D738058ED2\objs\moduleA.exe.obj(moduleA.exe)

 Error 42: Symbol Undefined _D4test7moduleC5funcCFZv
Error: linker exited with status 1

Reproduced locally (OSX), the code works with dmd -i but not with rdmd.