toivoh / debug.jl Goto Github PK

Currently the scoping analysis tests in test_decorate.jl are only comparing scoping analysis results to hand written predictions. There should be some way to compare the analysis with actual julia scoping behavior as well. This would help to alert for cases when scoping behavior has changed.

Julia seg faults when executing the statement using Debug.

Maybe it has something to do with the recent Expr change.

I have no idea how to figure out what's going on. Once this is fixed I'll issue a pull request to incorporate my context printing changes and then I'll work on the core guys to update the parser to incorporate file info.

(args)->body should behave pretty much like f(args...)=body when it comes to scoping.

Does anyone know why I'd be getting this error?

julia> Pkg.add("Debug")
INFO: Cloning cache of Debug from git://github.com/toivoh/Debug.jl.git
usage: git index-pack [-v] [-o <index-file>] [--keep | --keep=<msg>] [--verify] [--strict] (<pack-file> | --stdin [--fix-thin] [<pack-file>])
fatal: index-pack failed
ERROR: failed process: Process(`git clone -q --mirror git://github.com/toivoh/Debug.jl.git /Users/zaccranko/.julia/.cache/Debug`, ProcessExited(128)) [128]
 in wait at task.jl:51
 in add at pkg/entry.jl:314
 in add at pkg/entry.jl:71
 in anonymous at pkg/dir.jl:28
 in cd at file.jl:20
 in cd at pkg/dir.jl:28
 in add at pkg.jl:20

The debug hook should probably support interactive commands like

x     # to inspect value
x = 1 # to set value

and something to single step, possibly return?

The kinds of right hand sides in x=rhs will be pretty limited without any fancy parsing,
ints and variables should be feasible at least.

I am running Julia 2013-02-11. Ran Pkg.add("Debug") and then:

julia> using Debug
ERROR: find_in_path not defined
in include_from_node1 at loading.jl:76
in reload_path at loading.jl:96
in require at loading.jl:48
at /Users/c0g/.julia/Debug/src/Debug.jl:6

If I call @debug in the repl, I get told:

ERROR: @debug not defined

I've got a little test routine:

@debug function doitfor(x,y)
    println(x, " ", y)
    x += 1
    y *= 2
    z = 5
    for i in 1:10
        z = z+5
        z = z-3
        @bp
        z = z+6
    end
    println(x, " ", y, " ", z)
end

which in Julia in terminal results in:

julia> require("testdebug.jl")
55 1

at /home/john/Julia/data/testdebug.jl:11

      10           z = z-3
 -->  11           @bp
      12           z = z+6

debug:11> q
ERROR: interrupted
 in trap at /home/john/.julia/Debug/src/UI.jl:68
 in doitfor at no file:11
 in include at boot.jl:238
at /home/john/Julia/data/testdebug.jl:20

but in IJulia an "infinite" loop locks it up:

55 1

at In[12]:9

No context available!
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")
debug:9> ParseError("end of input")

Apologies for throwing issues at you so quickly. I've been adding the basic debug commands to my reference card so I was testing a few things. It is an excellent piece of software!

John

Currently,

@bp x > 0

will expand into

x > 0 ? @bp : nothing

and the debugger will trap at both the if node and the @bp node. It would probably be better if the user would experience @bp x > 0 as an atomic breakpoint.

Related: @bp should probably change the debugger state directly, so that it can be used outside of instrumented code. Need to change test_macro_trap and test_modscope to use something else, and reserve @bp for the interactive debugger.

Sometimes I am not able to step into a function I've defined. I'm not sure if this is a bug or a limitation of the current package.

Here's a small example.

@debug begin

function logistic(x)
  return 1./(1+exp(-x))
end

function f()
  s = 0.
  for i in 1:10
    @bp
    tmp = logistic(rand())
    s += tmp
  end
end

end # end debug

f()

Assume the code is in testdbg.jl and run it in the repl with include("testdbg.jl"). The following sequence of commands will not step intologistic, even though it intuitively should:n,s. I would think that this would step over the breakpoint and then into the function. However, the sequences,sdoes step into logistic. It seems that as long as one usesson **both** the@bpand the line to step into then we will enter the function. Maybe the@bpsyntax should be changed to allow code on the same line, e.g.@bp logistic(rand())and theBreakPointnode in the augmented AST can be skipped over so that only ones` is required to step into the code.

If we change the example a bit to

using Debug

@debug begin

function logistic(x)
  return 1./(1+exp(-x))
end

function f()
  @bp
  s = 0.
  for i in 1:10
    tmp = logistic(rand())
    s += tmp
  end
end

end # end debug

f()

and store it in a file testdbg2.jl and run it in the repl with include("testdbg2.jl") then the only way that I can step into the logistic function (or the for-loop) is by always using s. If I ever use n then I cannot step into logistic anymore.

Need to consider the scoping rules in global scope.
Should be as simple as suppressing AST transformation in debug_eval for global scopes?
Will have to assume that @debug is invoked at global scope. (Possibly check it?)

Attaching code to traps could be nice, too.

E.g. the syntax

$x = f(5)

would evaluate f(5) in the current scope, and assign the value to the variable Debug.Session.x.

Need to keep a pure interpolated symbol such as $x in the AST, make it look like a symbol, but not a normal one, and to provide the proper setter to graft.
Consider: Should all interpolations be evaluated at the point of occurrence instead of beforehand? Probably, would be inconsistent otherwise.

E g, with

debug_eval(scope, quote
    let
        x = 1
    end
end)

debug_eval should not substitute x, since it belongs to a scope that is nested inside the current one.
This probably means that getdefs should be adapted for scoping code for both debug_eval and @debug.

Consider the following code that is assumed to be correctly marked for debugging

function f()
  @bp
   for i in 1:10
      t = i.^2
   end
end

If the debugger is trapped at the for i in 1:10 line I issue the command n then the whole for-loop is executed. This is contrary to every debugger I've used and is also inconsistent with the behavior of other types of blocks in julia. For example, using n on an if statement causes the debugger to enter the block.

It would be very useful to be able to have the debugger to trap ERROR so you could inspect the stack on ERRORs.

It might be useful to insert some begin ... end blocks in instrumented code to allow adding debug_hook calls.
On the other hand, sticking to the blocks that exist gives the user some degree of control over debug trap placement. It's always possible to add a begin ... end block manually.

From the other side, maybe not every expression in a :blockshould generate a debug trap?
Should there be debug traps for expressions that don't have their own line number info?

Currently, lines that result from a macro expansion will be assumed to originate from the same file as the surrounding code, if they do not indicate otherwise.

Ideas:

finish: step out of function
where:  print current location
list:   list surrounding code

It seems hard to print excerpts from actual source files until the quality of source file info has improved. But it should be possible to print parts of the AST for the next expression or the surrounding code. Will need to access the AST for the surrounding code, and be able to print a window of lines from it.

I can't seem to see variables which have default values in the debugger as demonstrated by the following code:

using Debug

@debug function sumpow( n::Int, p::Int = 1 )
@bp
n^p + sumpow( n-1, p )
end

sumpow( 10 )

I'm running OS X with julia 0.2.0-2695.rc5cd44281 and testing the debug package.

Creating a source file with:

using Debug

@debug function f(a,b)
    @bp
    a + b
end

f(1,2)

and running inside a julia prompt with include("test.jl"), if I type a to see a's value and then an up arrow to traverse the command history, I get:

debug:4> ^[[A

down arrow produces:

debug:4> ^[[B

Instead of a Typ state for type declarations, there should probably be a TypeEnv <: Env that throws away all symbols, but is escaped by method declarations.

Since eval is now per module, debug_eval will have to find the right one to use.
Turn NoScope into ModuleScope and make it hold a reference to the right eval?

This is a great package. With the new Julia versions, the use of contains and add in Debug generated large numbers of warning messages. I've forked the repository and fixed the errors.

Substitute push! for add
Substitute in for contains and reverse the parameter order.

My copy is here

I use IPython and thus IJulia a lot so I'm going to try to figure out why it goes into an infinite loop on hitting the breakpoint currently.

Adding the ability to jump into functions in other files (or included modules) would be incredibly useful. I'm thinking a way to do it would be when we hit a :call node in the AST to look for the function definition in all source files that have been included or required (including the current file) and insert the AST for the function definition into the decorated AST and replace the original :call to jump to the injected function definition rather than the compile code.

Once could even probably step into functions in modules by searching through the .julia directory.

Does this sound "easily" doable?

Just wanted to throw this out there as another idea that would make the debugger really useful for julia users.

Thanks

Should there be commands to step between the current trap point and different containing scopes for interactive evaluation? Or should the intended scope be specified at each evaluation?

Should the entire call stack be kept, or should it just be possible to evaluate in the lexical parents of the current scope? If the call stack is kept, it should probably be possible to step both lexically and along the call stack. But keeping track of the call stack might be hard if coroutines are involved?

E g for (x, y) = (1, 2):

• Scoping: treat x and y as lhs

• debug_eval: translate it into

  temp = (1, 2)
  x = tupleref(temp, 1)
  y = tupleref(temp, 2)
  

Perhaps it would be possible to interface Debug.jl with DDD (or some other general purpose debugger GUI?) This might improve the debugging experience considerably.

Hi,
I tried to use the module in Windows with a julia 32bit build. I was not able to enter any of the commands.It always responded with ErrorException.
debug:3> parts
{}
debug:3> h
ErrorException("h not defined")
debug:3> s
ErrorException("s not defined")
debug:3> c
ErrorException("c not defined")

In my investigation it appeared that in the function trap (UI.j, line 63), you might want to use the function chomp, instead of [1:end-1] (equivalant to chop) . I suppose they are similar to similarly named perl functions. In my machine, the current expression always returned with a trailing "\r". This conflicts with the expectation of the program further down.

Regards,
Dinu

UI.jl (trap)
61: print("debug:", ndline, "> "); #flush(STDOUT)
62: while true
63: cmd = chomp(readline(STDIN))

julia> using Debug
Warning: using Base.Meta in module Meta conflicts with an existing identifier.
WARNING: has(d,k) is deprecated, use haskey(d,k) instead.
 in argstates at /home/sabae/.julia/Debug/src/Analysis.jl:120
WARNING: has(s::Set,x) is deprecated, use contains(s,x) instead.
 in postprocess_env! at /home/sabae/.julia/Debug/src/Analysis.jl:184
WARNING: -(a::Set,b::Set) is deprecated, use setdiff(a,b) instead.
 in postprocess_env! at /home/sabae/.julia/Debug/src/Analysis.jl:188
WARNING: |(s::Set...) is deprecated, use union(s...) instead.
 in postprocess_env! at /home/sabae/.julia/Debug/src/Analysis.jl:188
WARNING: expr(hd,a::Array{Any,1}) is deprecated, use Expr(hd,a...) instead.
 in instrument_args at /home/sabae/.julia/Debug/src/Graft.jl:98
WARNING: add_each! is deprecated, use union! instead.
 in instrument_args at /home/sabae/.julia/Debug/src/Graft.jl:88
WARNING: expr(hd,a...) is deprecated, use Expr(hd,a...) instead.
 in code_scope at /home/sabae/.julia/Debug/src/Graft.jl:39

These don't seem to affect anything, but thought I might record them here in any case.

E.g. save scope or call stack from the last trap point, allow to step around in the scopes and evaluate expressions. Hopefully, after an unhandled error in instrumented code, it would be enough to invoke Debug.pm() to activate post-mortem debugging.

Saving away scopes like this might keep some objects alive longer than otherwise intended, but most julia code should hopefully not depend on gc cleanup? Eventually, there could be an option whether to allow keeping scopes that have been exited.

The following example generates an error.
@debug function main_function(x)
function nested_function(y)
return y+2;
end
@bp
x+=2
println(x)
return x
end
main_function(2)

It works fine without the nested-function, but produces an error with it included (even though it is not used):
ERROR: ERROR: syntax error: break or continue outside loop
in include_from_node1 at loading.jl:76
in reload_path at loading.jl:96
in reload at loading.jl:60

The interactive debugger should be structured so that debugger commands can be tested by a script

using Debug results in:
Warning: using Base.Meta in module Meta conflicts with an existing identifier.

I set @debug begin...end around base/pkg.jl's "_resolve" function, as well as some other functions. With these additions, line 194 corresponds to if has(have,pkg).

After stepping through the early parts of the function, I got to a place where I wanted to set a breakpoint on each iteration of that loop. I got this:

debug:194> add($bp, $n)
no method add(Set{Node{T}},Node{ExValue})

debug:194> $bp
Set{Node{T}}()

debug:194> $n
^CNode(ExValue(:if,

debug:194> s

at /home/tim/src/julia/base/pkg.jl:195

Executing $n caused the prompt to hang, so I hit Ctrl-C.

Macros should probably be expanded before scoping analysis and code instrumentation/rewrite.
Can I just add a preprocessing step to expand the AST that is passed to @debug/debug_eval?

Should it be possible to step through code that is produced by macro expansion, and not present in the source?
At least code produced from quote blocks inside a macro will have line numbers that refer to the macro's source code.

Env:s could refer to the AST node that introduced the scope.
Loc:s could refer to the AST node of the immediately following expression.

Presumably the entire contents of a comprehension is one scope?

Something goes wrong running under tmux. Return just makes it say ^M instead of doing the thing.

debug:11> n^M

There should be a way to mark e.g. a method as debugable, without instrumenting it per default.
Such a method would become instrumented once the user steps into an invocation of it. Debug info would have to be kept from the start, but the method could hopefully run at normal speeds for normal invocations. This might only work for top-level methods. It should hopefully allow to mark much greater parts of code for debugging.

Suppose you have a loop,

for (k,v) in mydict
    # do some serious processing
end

Suppose an error happens for key "troublesome", and it so happens that this is the 247th key to be processed. It would be lovely to be able to set a conditional breakpoint, e.g.

for (k,v) in mydict
    @cbp k=="troublesome"
    # do some serious processing
end

I tried using the debugger on the example in the README and get the following error:
ERROR: error compiling anonymous: unsupported or misplaced expression typed-dict in function anonymous
in anonymous at no file:64

I am using version 1509530abd and am not really sure what is going on.

Thanks.

http://iainnz.github.io/packages.julialang.org/?pkg=Debug&ver=0.3

Should be useful for step over/step out etc.

I'm wondering if it would be possible to print program context while debugging like in pdb and adding an "l" (lowercase L) command to print some program lines around the current line.

I played around a bit and see that the AST is available so some program context could potentially be displayed. However, it would be really useful if the raw code could be attached before the julia parser converted it to the AST.

This would make debugging much easier when one doesn't have a text editor open.

If this is something you'd like to add I'd be happy to help.

Instead of entering debug mode upon entry to a @debug block, let it be triggered by calling a @bp macro or similar. Should it be a real macro, or just a name that @debug recognizes?

It might be useful to be able to leave @bp:s in non-instrumented code, and make the macro itself expand to nothing.