swensensoftware / unquote Goto Github PK

For example,

source <@ let [a;b] = [1;2] in a,b @>

produces

let patternInput = [1; 2] in if UnionCaseTest (patternInput, FSharpList`1.Cons) 
then (if UnionCaseTest (PropertyGet (Some (patternInput),
                            Microsoft.FSharp.Collections.FSharpList`1[System.Int32] Tail,
                            []), FSharpList`1.Cons) then (if UnionCaseTest (PropertyGet (Some (PropertyGet (Some (patternInput),
                                               Microsoft.FSharp.Collections.FSharpList`1[System.Int32] Tail,
                                               [])),
                            Microsoft.FSharp.Collections.FSharpList`1[System.Int32] Tail,
                            []), FSharpList`1.Empty) then (let a = patternInput.Head in let b = patternInput.Tail.Head in (a, b)) else raise (MatchFailureException("stdin", 18, 14))) else raise (MatchFailureException("stdin", 18, 14))) else raise (MatchFailureException("stdin", 18, 14))

i.e. :?> binary operator

Instead of,

> <@ fun () -> 3 @> |> source;;
val it : string = "fun unitVar0 -> 3"

should be

> <@ fun () -> 3 @> |> source;;
val it : string = "fun () -> 3"

For example:

> module Module = type Hello(x:int) = let x = x;;

module Module = begin
  type Hello =
    class
      new : x:int -> Hello
    end
end

> Sprint.sprintSig typeof<Module.Hello>;;
val it : string = "FSI_0264+Module+Hello"

Need to direct the compiler to generate XML doc comments and distribute the 
resulting .XML file along side the assembly.

For example,

<@ List.map id <| [1;2;3;] @> |> source

is

let mapping = fun x -> id x in fun list -> List.map mapping list <| [1; 2; 3]

but should be

(let mapping = fun x -> id x in fun list -> List.map mapping list) <| [1; 2; 3]

Example 1:

> source <@ String([|'h';'i'|]).Length @>;;
val it : string = "(String([|'h'; 'i'|])).Length"

Example 2:

> source <@ "asdf".Substring(1,2).Length @>;;
val it : string = "("asdf".Substring(1, 2)).Length"

Possibly a Quotation bug, resolution uncertain:

> let xl = [];;

val xl : 'a list

> <@ xl @>;;

  <@ xl @>;;
  ^^^^^^^^

C:\Users\Stephen\AppData\Local\Temp\stdin(9,1): error FS0030: Value 
restriction. The value 'it' has been inferred to have generic type
    val it : Expr<'_a list>    
Either define 'it' as a simple data term, make it a function with explicit 
arguments or, if you do not intend for it to be generic, add a type annotation.
> <@@ xl @@>;;
val it : Expr =
  Call (None,
      Microsoft.FSharp.Collections.FSharpList`1[System.Object] xl[Object](), [])
    {CustomAttributes = [NewTuple (Value ("DebugRange"),
          NewTuple (Value ("stdin"), Value (10), Value (4), Value (10),
                    Value (6)))];
     Type = Microsoft.FSharp.Collections.FSharpList`1[System.Object];}
> source <@ xl @>
;;
val it : string = "xl()"

UnionCaseTest expressions should not sprint "()" for zero arg unions:

> source <@ match None with | None -> true | _ -> false @>;;
val it : string =
  "let matchValue = None in (match matchValue with | None() -> true | _ -> false) && true"

should instead produce the following code:

let matchValue = None in (match matchValue with | None -> true | _ -> false) && 
true

Now that we have our own quotation evaluator, we are able to support expression 
types we were not able to before due to PowerPack's lack of support. One such 
expression is Quote. However, it appears impossible to differentiate between 
typed and untyped nested quotations (Quote expressions), all of them are as 
untyped quoations. So we still have to limit ourselves to only untyped 
quotation expressions.

This has the interesting feature that we can now use unquote to test quotation 
processors, including itself, as fully captured quotation expressions.

For example,

> unquote <@ sprintf "%A" 23 @>;;

(let clo1 = sprintf (PrintfFormat`5("%A")) in fun arg10 -> clo1 arg10) 23
System.Exception: Expected Application((Lambda _ | Value _), rhs) or 
Application(lhs,rhs), got 

Let (clo1,
     Call (None,
           Microsoft.FSharp.Core.FSharpFunc`2[System.Int32,System.String] PrintFormatToString[FSharpFunc`2](Microsoft.FSharp.Core.PrintfFormat`4[Microsoft.FSharp.Core.FSharpFunc`2[System.Int32,System.String],Microsoft.FSharp.Core.Unit,System.String,System.String]),
           [Coerce (NewObject (Void .ctor(System.String), Value ("%A")),
                    Microsoft.FSharp.Core.PrintfFormat`4[[Microsoft.FSharp.Core.FSharpFunc`2[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], FSharp.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[Microsoft.FSharp.Core.Unit, FSharp.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]])]),
     Lambda (arg10, Application (clo1, arg10)))

within

Application (Let (clo1,
                  Call (None,
                        Microsoft.FSharp.Core.FSharpFunc`2[System.Int32,System.String] PrintFormatToString[FSharpFunc`2](Microsoft.FSharp.Core.PrintfFormat`4[Microsoft.FSharp.Core.FSharpFunc`2[System.Int32,System.String],Microsoft.FSharp.Core.Unit,System.String,System.String]),
                        [Coerce (NewObject (Void .ctor(System.String),
                                            Value ("%A")),
                                 Microsoft.FSharp.Core.PrintfFormat`4[[Microsoft.FSharp.Core.FSharpFunc`2[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], FSharp.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[Microsoft.FSharp.Core.Unit, FSharp.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]])]),
                  Lambda (arg10, Application (clo1, arg10))), Value (23))

   at FSI_0008.Swensen.Unquote.Reduce.allArgsReduced@58(FSharpExpr expr, FSharpExpr subexpr) in ...\Reduce.fs:line 64
   at FSI_0008.Swensen.Unquote.Reduce.reduce(FSharpExpr expr) in ...\Reduce.fs:line 73
   at FSI_0008.Swensen.Unquote.Reduce.loop@87-3(FSharpExpr expr, FSharpList`1 acc) in ...\Reduce.fs:line 89

val it : unit = ()

This may be considered a sub-issue of issue 23.

e.g. instead of

> <@ not >> (=) @> |> source;;
val it : string = "(fun value -> not value) >> (fun x y -> x = y)"

should be

> <@ not >> (=) @> |> source;;
val it : string = "not >> (=)"

For example,

val it : unit = ()
> source 
        <@  
            match "hello world" with
            | InterpretedMatch @"llo" _ -> true
            | _ -> false
        @>;;
val it : string =
  "let matchValue = "hello world" in let activePatternResult = RegexUtils.|InterpretedMatch|_| "llo" matchValue in (match activePatternResult with | Some(_) -> true | _ -> false) && true"

However, as we learned with UnionCaseTest, the best output we can produced for 
even "simple" discriminant union or here active pattern pattern matching 
expressions is too complex to be helpful, therefore we urge Unquote users to 
avoid pattern matching in assertions and give this issue Low priority.

This will increase the size of the resulting Unquote assembly by about 2mb, but 
will remove any dependency on the PowerPack (having also eliminated our 
dependency on FSharp.PowerPack.Linq by implementing our own evaluation engine).

Tuples of length > 7 are represented with nested tuples in the 8th position 
("Rest") which is not being handled correctly.

e.g., instead of 

    source <@ let a,b = t in a,b @> =? 
        "let b = (let _,index1 = t in index1) in let a = (let index0,_ = t in index0) in (a, b)"

sprint as follows to better align with position based paradigm of tuples:

    source <@ let a,b = t in a,b @> =? 
        "let b = (let _,item2 = t in item2) in let a = (let item1,_ = t in item0) in (a, b)"

note: using lower-case "item" instead of upper-case (compared to underlying 
tuple property) since pattern match bindings should use lower-case.

e.g <@ for i in 1..2 do () @>

see: 
http://stackoverflow.com/questions/5284329/distinguishing-quoted-generic-value-c
all-and-generic-function-call

currently, unit functions are interpreted as generic values.

see screen shot

This applies to IfThenElse expressions which encompass AndAlso and OrElse 
derived patterns.

For example, instead of

> <@ if 3 + 2 = 5 then "hello" + "world" else "goodbye" + "world" @> |> 
unquote;;

if 3 + 2 = 5 then "hello" + "world" else "goodbye" + "world"
if 5 = 5 then "helloworld" else "goodbyeworld"
if true then "helloworld" else "goodbyeworld"
"helloworld"

We should have

> <@ if 3 + 2 = 5 then "hello" + "world" else "goodbye" + "world" @> |> 
unquote;;

if 3 + 2 = 5 then "hello" + "world" else "goodbye" + "world"
if 5 = 5 then "helloworld" else "goodbye" + "world"
if true then "helloworld" else "goodbye" + "world"
"helloworld"

This appears to be due to the "local optimization" of list union cases which 
aren't literal constructions:

> let x = [1;1;1];;

val x : int list = [1; 1; 1]

> <@ 1::x @> |> source;;
System.Exception: local list cons case not expected: Value (1)
   at [email protected](String message) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 400
   at Microsoft.FSharp.Core.PrintfImpl.go@512-3[b,c,d](String fmt, Int32 len, FSharpFunc`2 outputChar, FSharpFunc`2 outa, b os, FSharpFunc`2 finalize, FSharpList`1 args, Int32 i)
   at Microsoft.FSharp.Core.PrintfImpl.run@510[b,c,d](FSharpFunc`2 initialize, String fmt, Int32 len, FSharpList`1 args)
   at Microsoft.FSharp.Core.PrintfImpl.capture@529[b,c,d](FSharpFunc`2 initialize, String fmt, Int32 len, FSharpList`1 args, Type ty, Int32 i)
   at <StartupCode$FSharp-Core>[email protected](T1 inp)
   at FSI_0157.Swensen.Unquote.Sprint.sprint@392-394(Int32 context, FSharpExpr expr) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 398
   at <StartupCode$FSI_0171>.$FSI_0171.main@()
Stopped due to error

There are a handful of API naming breaking changes we should like to undertake 
for the upcoming Unquote version 2.0.0 release where we feel the benefit 
outweighs the cost, the first is the most significant
  1. Rename `Swensen.Unquote.Operators.source` and the extension method `Expr.ToSource()` to `Swensen.Unquote.Operators.decompile` and `Expr.Decompile()` respectively. We feel this will be a more consistent renaming among the `eval` and `reduce` operators and will not impact folks who are only using Unquote for its main intended purpose as a unit testing assertion library.
  2. Consolidate `Swensen.Unquote.ExprExt` and `Swensen.Unquote.TypeExt` into `Swensen.Unquote.Extensions`. Since these are all AutoOpen modules, if they the two being consolidated were only used in that fashion, then this will have no effect on users. And also this will have no effect on users only using Unquote for its unit testing features.
  3. Rename `Swensen.Unquote.Operators.Private` to `Swensen.Unquote.Operators.Internal` and mark it with the `System.Obsolete` so that it is hidden from Visual Studio IntelliSense (the EditorBrowsable attribute not being honored by F#, a recently filed bug: http://stackoverflow.com/questions/6527141/is-it-possible-to-mark-a-module-function-as-hidden-from-intellisense-discovery/6527933#6527933). This shouldn't effect any users as the module was never intended to be used publicly and is only public because due to references by other legitimately public inlined operators.  

> <@ (~~~) 1 @> |> eval;;
System.NotSupportedException: Specified method is not supported.

Server stack trace: 
   at Microsoft.FSharp.Core.Operators.op_LogicalNot[T](T value)

Exception rethrown at [0]: 
   at Swensen.Unquote.Evaluation.eval(FSharpList`1 env, FSharpExpr expr) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\code\Unquote\Evaluation.fs:line 260
   at Swensen.Unquote.Extensions.Expr`1.Eval[T](FSharpExpr`1 ) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\code\Unquote\Extensions.fs:line 25
   at <StartupCode$FSI_0034>.$FSI_0034.main@()
Stopped due to error

For example,

unquote <@ [1 + 2; 2; 3; 4] = [3; 2; 3; 4] @>

produces

[3; 2; 3; 4] = [3; 2; 3; 4]
true

but should produce

[1 + 2; 2; 3; 4] = [3; 2; 3; 4]
[3; 2; 3; 4] = [3; 2; 3; 4]
true

e.g. <@ while true do () @>

e.g.

> <@ 23L <<< 1 @> |> eval;;
System.NotSupportedException: Specified method is not supported.
   at Swensen.Unquote.DynamicOperators.invokeBinOp(String name, Type aty, Type bty, Object x, Object y)
   at Swensen.Unquote.Evaluation.eval(FSharpList`1 env, FSharpExpr expr)
   at Swensen.Unquote.Extensions.Expr`1.Eval[T](FSharpExpr`1 )
   at <StartupCode$FSI_0005>.$FSI_0005.main@()
Stopped due to error

e.g., all of the following should be re-sugared:

> <@ [1..5] @> |> unquote;;

Seq.toList (seq {1..5})
Seq.toList (seq seq [1; 2; 3; 4; ...])
Seq.toList seq [1; 2; 3; 4; ...]
[1; 2; 3; 4; 5]

val it : unit = ()
> <@ [1..5..10] @> |> unquote;;

Seq.toList (seq {1..5..10})
Seq.toList (seq seq [1; 6])
Seq.toList seq [1; 6]
[1; 6]

val it : unit = ()
> <@ [|1..10|] @> |> unquote;;

Seq.toArray (seq {1..10})
Seq.toArray (seq seq [1; 2; 3; 4; ...])
Seq.toArray seq [1; 2; 3; 4; ...]
[|1; 2; 3; 4; 5; 6; 7; 8; 9; 10|]

val it : unit = ()
> <@ [|1..5..10|] @> |> unquote;;

Seq.toArray (seq {1..5..10})
Seq.toArray (seq seq [1; 6])
Seq.toArray seq [1; 6]
[|1; 6|]

val it : unit = ()

This needs to implemented for the decompiler and reducer

e.g., currently falls back on ToString:

> source <@ (System.Func<int,int>(fun i -> i + 1)) @>;;
val it : string =
  "NewDelegate (System.Func`2[[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.Int32, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]],
             i,
             Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
                   [i, Value (1)]))"

Instead of 

> <@ fun x (g, f) -> 3 @> |> source;;
val it : string =
  "fun x tupledArg -> let g = (let item1,_ = tupledArg in item1) in let f = (let _,item2 = tupledArg in item2) in 3"

should be

> <@ fun x (g,f) -> 3 @> |> source;;
val it : string = "fun x (g, f) -> 3"

While these have been implemented in the decompiler for some time, they were 
not supported by PowerPack's Eval so crippled further support.  However, having 
implemented our own eval with support for all these kinds of set expressions, 
we are able to advance our overall support in the reduction and testing realm.

Related to issue 23, Let(_,x,Lambda(_)) when x |> isReduced should be 
considered reduced, since we don't want to see generated lambda names sprinted

So instead of

> unquote <@ 23 + 3 + 4 + 1, let x = 2 + 3 in (fun j -> j + x) @>;;
> 
(23 + 3 + 4 + 1, (let x = 2 + 3 in fun j -> j + x))
(26 + 4 + 1, (let x = 5 in fun j -> j + x))
(30 + 1, <fun:ToFSharpFunc@3051-1>)
(31, <fun:ToFSharpFunc@3051-1>)

should be

> unquote <@ 23 + 3 + 4 + 1, let x = 2 + 3 in (fun j -> j + x) @>;;
> 
(23 + 3 + 4 + 1, (let x = 2 + 3 in fun j -> j + x))
(26 + 4 + 1, (let x = 5 in fun j -> j + x))
(30 + 1, (let x = 5 in fun j -> j + x))
(31, (let x = 5 in fun j -> j + x))

Low priority since can't be reduced by current power pack Eval implementation. 
But we already support VarSet and FieldSet expressions, so would be good to be 
complete, and also future ready (I'm betting next major power pack release will 
have a better Eval implementation).

The current re-sugaring is ad-hoc and we can improve it for example, given

    let f (x:obj) (y:obj) = x |> string

Before this fix the following expression decompilation

    <@ 2 |> f "2" @> |> decompile

would be

    "2 |> let x = "2" in fun y -> f x y"

due to coercion throwing off our pattern matching. But after this fix we'll have

    "2 |> f "2""

Need to create and maintain an AssemblyInfo file with version info, etc.

e.g. instead of

> typedefof<int * int>.FSharpName;;
val it : string = "T1 * T2"

should be

val it : string = "'T1 * 'T2"

For example, given 

let (=~) input pattern = System.Text.RegularExpressions.Regex.IsMatch(input, 
pattern)

the quotation

<@ "hello world" =~ "worlds" @>

decompiles to

RegexUtils.op_EqualsTwiddle "hello world" "worlds"

Instead of,

<@ [1;2;3;] |> List.map (fun i -> i+1) @> |> source

producing

"[1; 2; 3] |> let mapping = fun i -> i + 1 in fun list -> List.map mapping list"

should produce

[1;2;3;] |> List.map (fun i -> i+1)

N.B. I consider this a high value but non-trivial feature.

e.g.

> source <@ System.Collections.Generic.Dictionary<string,int>() @>;;
val it : string = "Dictionary`2()"

Introduced while fixing issue #4:

> source <@ [] @>;;
System.Exception: unexpected list union case
   at FSI_0004.Swensen.Unquote.Sprint.isLiteralConstruction@312(FSharpExpr _arg1) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 318
   at [email protected](Int32 context, FSharpExpr expr) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 320
   at <StartupCode$FSI_0034>.$FSI_0034.main@()
Stopped due to error

For example,

source <@ try (null:string).Length with | e -> -1 @>

produces

TryWith (PropertyGet (Some (Value (<null>)), Int32 Length, []), e, Value (1), e,
         Value (-1))

Quotations involving && or || operators are captured as IfThenElse expressions. 
DerivedPatterns includes two patterns, AndAlso and OrElse which attempt to 
reverse engineer this form. But there are occasions when it doesn't get it 
right and specifically when a given expression matches both patterns.  For 
example

  1. false || false and false && true
  2. true || false and true && true

But we can make smarter decompilation choices based on short-circuiting 
knowledge. 

In the first case, we know "false || false" would be a better choice since 
"<expr> && true" is equivalent to just "<expr>" so "<expr> && true" with the 
literal "true" on the right hand side is an unlikely actual expression.

Similarly, in the second case, we know "true && true" is a better choice than 
"true || false".

some support missing, like instance properties with args

Example 1:

source <@ let a,b = (1,2) in a,b @>

produces

let patternInput = (1, 2) in let b = TupleGet (patternInput, 1) in let a = 
TupleGet (patternInput, 0) in (a, b)

(notice that patternInput expressions are a special case to be addressed at 
another time).

Example 2:

let t = (1,2)
source <@ let a,b = t in a,b @>

produces

let b = TupleGet (PropertyGet (None, System.Tuple`2[System.Int32,System.Int32] 
t, []), 1) in let a = TupleGet (PropertyGet (None, 
System.Tuple`2[System.Int32,System.Int32] t, []), 0) in (a, b)

For example,

> [3].GetType().GetGenericTypeDefinition().FSharpName;;
System.ArgumentNullException: Value cannot be null.
Parameter name: input
   at System.Text.RegularExpressions.Regex.Match(String input)
   at FSI_0038.Swensen.RegexUtils.|CompiledMatch|_|(String pattern, String str) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\RegexUtils.fs:line 28
   at FSI_0040.Swensen.Unquote.Sprint.sprintSig@169-20(Int32 context, Type t) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 172
   at FSI_0040.Swensen.Unquote.Sprint.sprintSig@169-20(Int32 context, Type t) in C:\Users\Stephen\Documents\Visual Studio 2010\Projects\Unquote\Unquote\Sprint.fs:line 186
   at <StartupCode$FSI_0058>.$FSI_0058.main@()
Stopped due to error

Found bug:

<@ (~+) 3s @> |> eval;;

fails. I am currently refactoring native numeric and conversion op 
implementations so that there is more code reuse and less change for these 
kinds of typo / oversight bugs.

e.g. the following is currently incredibly verbose:

    source <@ let a,b = (1,2) in a,b @> =? 
        "let patternInput = (1, 2) in let b = (let _,item2 = patternInput in item2) in let a = (let item1,_ = patternInput in item1) in (a, b)"

For example, source <@ "\n\r\t" @> should equal @"\n\r\t"

Now that we have our own Eval implementation, we can easily add Expr extension 
members to Reduce, ReduceFully, and the new Eval so that we can pass in an 
variable environment to handle synthetic quotations, e.g.

    Expr.Var(new Var("x", typeof<int>)).Eval([("x", 1 |> box |> ref)])

This will give us many benefits over dependence on PowerPack's quotation 
evaluation
  1. Faster
  2. Simpler and therefore less bug prone
  3. Allow us to support expressions which PowerPack doesn't / can't support like VarSet, PropertySet, FieldSet, and ForIntegerLoop
  4. Remove dependency on FSharp.PowerPack.Linq