lens / lens-aeson Goto Github PK

Not sure if this is correct behaviour that I just don't understand or if this is a bug.

("{\"a\": [ { \"b\": [ 1, 2, 3]}] }" :: Text) ^? key "a" . nth 0 . key "b" . values

returns

Just (Number 1.0)

What looks weird to me because if I remove the . values I get:

("{\"a\": [ { \"b\": [ 1, 2, 3]}] }" :: Text) ^? key "a" . nth 0 . key "b" 
Just (Array [Number 1.0,Number 2.0,Number 3.0])

Thanks in advance for any assistance.

I've got these two helper functions for making JSON instances. These have ended up in a Misc module in several of my projects:

--------------
-- Helpers for stealing JSON instances from an isomorphic type
-------------
isoParseJSON :: FromJSON a => Iso' b a -> Value -> Parser b
isoParseJSON i v = view (mapping (from i)) (parseJSON v)

isoToJSON :: ToJSON a => Iso' b a -> b -> Value
isoToJSON i b = toJSON (view i b)

The idea is that they are used for getting free instances from an isomorphic type:

-- Assume that Cat has instances for FromJSON and ToJSON
myIso :: Iso' Dog Cat
myIso = ...
instance FromJSON Dog where
  parseJSON = isoParseJSON myIso
instance ToJSON Dog where
  toJSON = isoToJSON myIso

For simple newtype wrapping, you can just use GND instead, but in more complicated situations, that may not be an option. Anyway, since this package is sort of where lens and aeson meet, I thought that it may be an appropriate home for these, even though the scope is a little different than what these functions do. Let me know what you think. Thanks.

Transferring over this issue as posted by bos ekmett/lens#423

At the moment, combinators like key are polymorphic over some types, but not all. For instance, I can use key on either a ByteString or a Value.

The http-client library defines a Response type, such that I can have a Response ByteString or a Response Value. I cannot currently figure out how to swizzle the various moving parts such that I can use key and other combinators on one of these.

(Of course I have a responseBody lens, so I can write responseBody . key "foo", but my wish is to be able to more simply write key "foo" instead.)

Citing from http://hydra.cryp.to/build/1064212/nixlog/1/raw:

Running 1 test suites...
Test suite doctests: RUNNING...
### Failure in src/Data/Aeson/Lens.hs:265: expression `"[1,2,3]" ^? _Value'
expected: Just (Array (fromList [Number 1.0,Number 2.0,Number 3.0]))
 but got: Just (Array [Number 1.0,Number 2.0,Number 3.0])
### Failure in src/Data/Aeson/Lens.hs:283: expression `"[1,2,3]" ^? _Array'
expected: Just (fromList [Number 1.0,Number 2.0,Number 3.0])
 but got: Just [Number 1.0,Number 2.0,Number 3.0]
Examples: 51  Tried: 51  Errors: 0  Failures: 2
Test suite doctests: FAIL

Yes

λ> (fromList [("test", Number 42)] :: Object) ^? key "test"

<interactive>:20:28:
    No instance for (AsValue Object) arising from a use of `key'
    Possible fix: add an instance declaration for (AsValue Object)
    In the second argument of `(^?)', namely `key "test"'
    In the expression: (fromList [] :: Object) ^? key "test"
    In an equation for `it': it = (fromList [] :: Object) ^? key "test"

It looks very strange.

Possible solutions:

1. Add instance AsValue Object.
2. Make key reuire AsObject instead of AsValue.

1bb9a09 makes this library work with GHC 8.2 and Cabal 2.0, but it's not part of any release yet.

Can you make a release on hackage?

FWIW the testsuite passes with GHC 8.2.

Instead of writing

livingOnTheEdge = foo ^?! key "this" . key "is" . key "annoying"

I can write

keys = foldl' (\x f -> f . key x) id

livingOnTheEdge = foo ^?! keys ["this", "is","less", "annoying"]

Seems common enough to have in the library

This may be a misunderstanding on my part, and if so, I apologize.

Because some of the identifiers in this library begin with underscores, GHC 7.8.3 is picking them up as "Typed Holes" whenever I use them.

01:05:02 /d/proj/Haskell/RunscopeExport$ cabal build
Building RunscopeExport-0.1.0.0...
Preprocessing executable 'RunscopeExport' for RunscopeExport-0.1.0.0...
[1 of 1] Compiling Main             ( Main.hs, dist\build\RunscopeExport\RunscopeExport-tmp\Main.o )

Main.hs:52:64:
    Found hole `_Value'
      with type: (Maybe Value -> Const (Maybe Value) (Maybe Value))
                 -> BL.ByteString -> Const (Maybe Value) BL.ByteString
    Relevant bindings include
      response :: Response BL.ByteString (bound at Main.hs:51:5)
      url :: ByteString (bound at Main.hs:50:9)
      bKey :: ByteString (bound at Main.hs:49:5)
      uuid :: ByteString (bound at Main.hs:48:12)
      getDetails :: ByteString -> IO Value (bound at Main.hs:48:1)
    In the second argument of `(.)', namely `_Value'
    In the second argument of `(^.)', namely `responseBody . _Value'
    In the second argument of `($)', namely
      `response ^. responseBody . _Value'

Do I need to (somehow?) disable typed holes?

Although it'll update existing values, key won't insert new values into an Object:

λ> set (key "b") (Number 2) "{\"a\": 1}"
"{\"a\":1}"

I don't know if this is the intended behavior but it seems a bit odd to me since at does it.

Pun intended. 🙂

I appreciate this warning about filtered. (I guess the reason the type is written as Optic' instead of Prism' is to discourage people from reviewing it?)

I'm sure there's a reason nonNull is an improper Prism' instead of an improper Traversal', I'm just not sure what the reason is. Maybe the fix is to document wherever optics laws are violated in the library, as lens tries to, not only here but

>>> let s = "-0" in over _Number id s == s
False

etc.

In aeson-2.0.0.0 and later, Object uses a KeyMap rather than directly exposing a HashMap. See haskell/aeson#866. In light of this, the current type of _Object (Prism' t (HashMap Text Value)) is not ideal—ideally, it would focus on a KeyMap instead. Some possibilities are:

1. Keep _Object's type as is, but offer another method of type Prism' t (KeyMap Value) alongside it. (_ObjectKeyMap, perhaps?)
2. Just change the type of _Object to Prism' t (KeyMap Value) . This would require a breaking API change.

Either option will likely require waiting a bit until we can require aeson >= 2 unconditionally.

Doctests fail on 32-bit archs (e.g., i386, armel, etc) with the following error:

Running 1 test suites...
Test suite doctests: RUNNING...
/<<PKGBUILDDIR>>/src/Data/Aeson/Lens.hs:331: failure in expression `"{\"a\": 4, \"b\": 7}" ^@.. members'
expected: [("a",Number 4.0),("b",Number 7.0)]
 but got: [("b",Number 7.0),("a",Number 4.0)]

Everywhere else in lens we adopted the practice of adding pattern synonyms for the prisms we supply. Here this would look something like:

pattern JSON :: (FromJSON a, ToJSON a, AsJSON t) => () => a -> t
pattern JSON a <- (preview _JSON -> Just a) where
  JSON a = _JSON # a

pattern Value_ :: (FromJSON a, ToJSON a) => () => a -> Value
pattern Value_ a <- (fromJSON -> Success a) where
  Value_ a = toJSON a

pattern Number_ :: AsNumber t => Scientific -> t
pattern Number_ n <- (preview _Number -> Just n) where
  Number_ n = _Number # n

pattern Double :: AsNumber t => Double -> t
pattern Double d <- (preview _Double -> Just d) where
  Double d = _Double # d

pattern Integer :: AsNumber t => Integer -> t
pattern Integer i <- (preview _Integer -> Just i) where
  Integer i = _Integer # i

pattern Integral :: (AsNumber t, Integral a) => a -> t
pattern Integral d <- (preview _Integral -> Just d) where
  Integral d = _Integral # d

pattern Primitive :: AsPrimitive t => Primitive -> t
pattern Primitive p <- (preview _Primitive -> Just p) where
  Primitive p = _Primitive # p

pattern Bool_ :: AsPrimitive t => Bool -> t
pattern Bool_ b <- (preview _Bool -> Just b) where
  Bool_ b = _Bool # b

pattern String_ :: AsPrimitive t => Text -> t
pattern String_ p <- (preview _String -> Just p) where
  String_ p = _String # p

pattern Null_ :: AsPrimitive t => t
pattern Null_ <- (preview _Null -> Just ()) where
  Null_ = _Null # ()

It seems that https://tools.ietf.org/html/rfc7159 removed the need to really distinguish between object and array and 'the rest of the stuff'.

What ix does on a Value is, it filters it as an Object, and then looks up a key. But filtering for objects is an arbitrary decision: there are two possible notions of an index for a JSON value:

• numerical (for arrays)
• textual (for objects, called "names" in the spec)

Granted, people are going to choose the textual notion of indexing a value 99% of the time. But they can just use key, of which ix @Value is a type-restricted version. ix @Value doesn't add anything practical, even if it were conceptually sound (which it isn't IMO).

I'm using a lot this wrapper to parse maybes

nullIsNothing :: Prism' Value q -> Prism' Value (Maybe q)
nullIsNothing p = prism'
  do
    \case
      Nothing -> Null
      Just q -> review p q
  do
    \case
      Null -> Just Nothing
      x -> Just <$> preview p x

Would you insert it ?

Can we bump the bounds of lens-aeson to support aeson 0.11? The current bounds are stopping the major package wreq from compiling, which is stopping our libraries from compiling with aeson.

Cheers!

• Andy

phadej/aeson-optics@ff31ca0 is what I did for aeson-optics

lens-aeson does not build with the latest version of the vector package.

Currently _JSON has the type (FromJSON a, ToJSON a) => Prism' t a, but I do not think that there is any reason for it not to be (FromJSON a, ToJSON b) => Prim t t a b. t does not care what the JSON it holds represents. I made pull request #28 some time ago, but it was ignored. Is this not worth the minor breaking change, or do you plan to merge it at a later time? Or did you just not have time to review it?

One reason I could think of against this change would be, that if we instantiate _JSON :: Prism t t a b, where we do not have ToJSON a and FromJSON b, the prism laws are not well defined. For example the law preview l (review l b) ≡ Just b requires, that we can instantiate the prism with Prism' t a, which isn't possible in the above setting. But is that really such a big problem? And if it is, we can still give _JSON the type (FromJSON a, ToJSON a, FromJSON b, ToJSON b) => Prism t t a b.

edit: I think the last point is moot anyway, as a and b are not part of the parameters to AsJSON and therefore _JSON :: Prism' t a just has to be lawful if it is defined, that is if a is both FromJSON and ToJSON. At least that's how I would interpret the prism laws.

... which you can correct with a revision

FYI: there's aeson-2.1 release too, which shouldn't need any code changes to lens-aeson, so can be done in the same revision.

I'd like to be able to write the following:

over (at "foo") act value

but there is no instance for At Value. Using key "foo" works except that there's no way to create if the key 'foo' is not already in the object, and there's no way to remove, unless I'm missing something major?

I have now written the following a couple of times:

fromMaybe [] . Data.Vector.toList $ x ^? _Array

Would it make sense to have a _List prism that so I could just write:

x ^? _List

?

Would it make sense to include:

traverseStrings :: Applicative f => (Text -> f Text) -> Value -> f Value
traverseStrings f = go
  where go (String t)    = String <$> f t
        go (Array arr)   = Array <$> traverse go arr
        go (Object obj)  = Object <$> traverse go obj
        go rest          = pure rest

and similar traverseNumbers and traverseBools, and corresponding Traversal's?

I'm not sure about naming though.

In

strictTextUtf8 :: Iso' Text.Text Strict.ByteString
strictTextUtf8 = iso StrictText.encodeUtf8 StrictText.decodeUtf8

decodeUtf8 is partial, it errors if the encoding is not correct.

*Data.Aeson.Lens> :set -XOverloadedStrings 
*Data.Aeson.Lens> :t view _Key ("\255" :: ByteString)
view _Key ("\255" :: ByteString) :: Key
*Data.Aeson.Lens> view _Key ("\255" :: ByteString)
"*** Exception: Cannot decode byte '\xff': Data.Text.Internal.Encoding.streamDecodeUtf8With: Invalid UTF-8 stream

I'm not sure what would be the best way to proceed:

1. Removing instances is a breaking change
2. making them prisms is difficult
3. Using decodeUtf8Lenient would at least remove error (though not make it a true Iso)

I'm leaning to go 1. for optics-aeson (2. is also possible), but lens-aeson could do 3.?

EDIT: strictTextUtf8 is used in _Value too, but there it's used on aeson produced bytestrings, which should be valid UTF8, so it's "safe".

When I type

"{}" & _Object . at "a" . _JSON .~ Just (3 :: Int)

into ghci, it outputs one " and then hangs. I would expect it to output "{\"a\":3}". Why does this happen? Is there a different way to add a new key to the Object that I should use?