haskellari / postgresql-libpq Goto Github PK

While building package postgresql-libpq-0.9.5.0 (scroll up to its section to see the error) Process exited with code: ExitFailure 1 during stack build.

stack v2.9.1
cabal v3.6.2.0
ghc v9.2.5

We seem to have different treatment for Bytestrings as handled by LibPQ:

-- Create a table some_table with a nullable binary_column bytea type
--     CREATE TABLE some_table (id SERIAL,binary_column BYTEA);
ghci> import Database.PostgreSQL.LibPQ
ghci> c <- connectdb "postgresql://use-your-connection-string"
ghci> Just res <- execParams c "INSERT INTO \"some_table\" (binary_column) values ($1), ($2)" [Just (Oid 17,"",Binary), Just (Oid 17,mempty,Binary)] Binary

Now if you look in the table, you’ll have one NULL column and one with an empty bytea content

This seem to come from the fact that mempty and "" have different internal representation in Haskell, in spite of being equal:
(requires OverloadedStrings)

ghci> import Data.ByteString
ghci> a = mempty :: ByteString
ghci> b = "" :: ByteString
ghci> a == b
True
ghci> :force a
a = bytestring-0.11.5.3:Data.ByteString.Internal.Type.BS
      0x0000000000000000 GHC.ForeignPtr.FinalPtr 0
ghci> :force b
b = bytestring-0.11.5.3:Data.ByteString.Internal.Type.BS
      0x0000004200647b40
      (GHC.ForeignPtr.PlainPtr (ByteArray# <mutableByteArray>)) 0

This difference of treatment seem to be introduced by commit
89c7cb8

Reverting that commit indeed stores the two different values as an empty bytea.

This change seems to fix the symptom I get a couple levels up the abstraction chain in Persistent:

finish :: Connection
       -> IO ()
finish (Conn fp _) = do
    uninterruptibleMask_ $ finalizeForeignPtr fp

yesodweb/persistent#1199

I discovered this fixed that symptom here: yesodweb/persistent#1199 (comment)

The persistent-postgresql CI is currently failing with an error message emitted by libpq, probably the underlying C library.

Cf yesodweb/persistent#1197

The error message is here:

Failures:

  src/MigrationTest.hs:41:5: 
  1) Migration is idempotent
       uncaught exception: IOException of type OtherError
       libpq: failed (server closed the connection unexpectedly
       	This probably means the server terminated abnormally
       	before or while processing the request.
       could not send SSL negotiation packet: Success
       )

  To rerun use: --match "/Migration/is idempotent/"

  test/PgIntervalTest.hs:37:5: 
  2) Postgres Interval Property tests Round trips
       uncaught exception: IOException of type OtherError
       libpq: failed (server closed the connection unexpectedly
       	This probably means the server terminated abnormally
       	before or while processing the request.
       could not send SSL negotiation packet: Success
       )
       (after 25 tests)
         15.333255892191s

I have no idea if this is happening for anyone else, but the only results for Googling it are my own tweets complaining about it. Posting here as a pointer to an issue in case anyone else runs across it.

Tested using

 custom-setup
   setup-depends:
       base   >=4.3  && <5
-    , Cabal  >=1.10 && <3.7
+    , Cabal  >=1.10 && <3.9

Just got bitten by a property test that tried to save a string with a zero byte in it to the database + got it silently truncated. I know that's the documented behavior in libpq, but it seems less than ideal in Haskell, a language that can contain zero bytes in strings unlike C.

Or perhaps it should be documented in escapeStringConn to look at the libpq docs for edge cases? In this case, should the postgresql-simple library throw an error? I'd understand postgresql-libpq not wanting to deviate from libpq behavior, but it seems like a good thing to catch unexpected behavior in postgresql-simple (I only bring it up because it seems like the authors of the two libraries are the same)

This function seems pure to me, aside from the allocation that it does. But Haskell allocates in many places, and we still call those functions pure. I don't understand why unescapeBytea must be in IO? What kind of side effects can it perform?

I've checked that this package compiles with the newer version of base.

Notices from the server are usually printed to standard error by libpq in defaultNoticeProcessor. Some applications may want to log these notices in a different way, but postgresql-libpq seems to only offer a way to disable them with disableNoticeReporting. Could a way to override notice processing with PQsetNoticeProcessor be exposed in postgresql-libpq?

Hackage has Win32-2.13.2.1 with what appear to be changes unrelated to how it's used in postgresql-libpq.

(I ran into this in a CI build with current stackage nightly, failing with

Error: While constructing the build plan, the following exceptions were encountered:
In the dependencies for postgresql-libpq-0.9.4.3:
Win32-2.12.0.1 from stack configuration does not match >=2.2.0.2 && <2.11  (latest matching
                   version is 2.10.1.1)
needed due to postgrest-9.0.1.20220630 -> postgresql-libpq-0.9.4.3

A build with allow-newer is in process here, which compiled postgresql-libpq successfully with Win32-2.12.0.1.)

Surfaced in the esqueleto CI:

What is odd is that the configured version of cabal-install in the CI is 3.8.1.0, but Cabal-the-library is 3.10.2.1.

This may require a Hackage revision or a new release.

According to CONTRIBUTING.md, I am just submitting an issue instead of a pull request.

Hi again @phadej 👋

Asking for opinion before a PR.

In followup to haskellari/postgresql-simple#114 — I've found a heavy heap fragmentation issue in my LO-intense workload — and determined that it's not the Haskell heap which got fragmented, but rather the C malloc one.

☝️ Here, the reallocBytes call, while releasing the extra unused space under maxLen — will create differing-length allocations, which depend on LO sizes. So if I do tens of thousands of varying-size LO reads — this puts stress onto malloc to handle fragmentation well.

The fixed graph on the left — is exact same test running against exact same Haskell executable, but with LD_PRELOAD=jemalloc.so. The baseline on the right — with stock allocator in libc6 2.35-0ubuntu3.1 on Ubuntu 22.04.2 LTS.

Would you approve adding a flag jemalloc to the cabal-file here?

https://jemalloc.net/ to perhaps save you a search query 😅