problems compiling on mac os x 10.7.1 about osrm-backend HOT 19 CLOSED

wha'ts the status of compiling osrm with gcc 4.2?

from osrm-backend.

Perhaps it's time to build a working gcc formula first. I'll see if I
can dig up an Intel Mac for building. My 2008/9 MacBook is a bit too low
powered for that purpose.

Am 03.10.2011 22:45, schrieb Emil Tin:

i'm trying to get osrm working on mac os x 10.7.1, by using a homebrew formula i created a homebrew formulat:
https://github.com/emiltin/homebrew-alt/blob/master/duplicates/gcc.rb

osrm must be compiled with gcc newer than the 4.2 that's cames with os x 10.7.1, so the brew fomula first builds gcc 4.6, when uses that for building osrm. but the resulting osrm extractor crashes when parsing a protobuf file:

~/osm$ osrm-extract cph.osm.pbf
[extractor] extracting data from input file /Users/emil/osm/cph.osm.pbf
[error] Failed to open file extractor.ini for reading.
[STXXL-MSG] STXXL v1.3.1 (release)
[STXXL-ERRMSG] Warning: no config file found.
[STXXL-ERRMSG] Using default disk configuration.
[STXXL-MSG] 1 disks are allocated, total space: 1000 MiB
osrm-extract(59807) malloc: *** error for object 0x101a4f8a0: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

this type of error seems to be a known compilation issue on newer mac systems:

http://newartisans.com/2009/10/a-c-gotcha-on-snow-leopard/
https://trac.macports.org/ticket/22234

so it might be related to the way gcc 4.6 is being build.

tonight i had a chat on IRC with the creator of the gcc 4.6 brew fomula, csharpsteen:

[18:55] MistyM: csharpsteen: emiltin was wondering about the GCC formula you wrote at homebrew-alt
[18:55] emiltin: hi
[18:56] csharpsteen: What was the question?
[18:56] emiltin: i'm trying to create a formula for open source routing machine, osrm: https://gist.github.com/1257319
[18:56] emiltin: it requires a newer version of gcc, so i'm using your formula to build gcc 4.6
[18:57] emiltin: but the resulting osrm binaries suffers from bugs related to malloc
[18:57] csharpsteen: Hmm. Never tried building it with clang.
[18:57] emiltin: http://newartisans.com/2009/10/a-c-gotcha-on-snow-leopard/
[18:58] emiltin: https://trac.macports.org/ticket/22234
[18:58] csharpsteen: When I developed the formula, I built it using gcc 4.2.1 from XCode 3.2.6. The GCC test suite seemed to pass alright---as I recall the only unexpected failures were in the link-time optimization tests.
[18:58] emiltin: it seems --enable-fully-dynamic-string is required, perhaps also when building the standard libs?
[18:59] emiltin: i could get the fomula to build without using clang
[18:59] emiltin: could not
[18:59] wolfman2000 left the chat room. (Remote host closed the connection)
[19:00] csharpsteen: Which version of XCode do you have?
[19:01] abyxcos: Which formula? I built gc-4.6.0 with clang (failed with llvm-gcc)
[19:01] emiltin: i'm on 10.7.1 with xcode 4.1, gcc version 4.2.1
[19:01] abyxcos: Don't know if I pulled in -cxx though.
[19:01] emiltin: abyxcos: yes the gcc 4.6 formula, i could also only get it to work using clang
[19:02] emiltin: i included the cxx option
[19:03] emiltin: here's the build g++ version: https://gist.github.com/1259929
[19:03] csharpsteen: Mmmm... was not aware of the "fully-dynamic-string" issue---looks like it should be used if MACOS_VERSION >= 10.6
[19:03] abyxcos: Mhh, I have no g++-4.6. Want me to rebuild it?
[19:03] emiltin: and this is the bug i'm encountering in the binaries that gcc 4.6 builds: osrm-extract(76625) malloc: ** error for object 0x1084d78a0: pointer being freed was not allocated
[19:04] abyxcos: On the off chance, have you tried building OSRM with llvm-gcc or clang?
[19:05] abyxcos: I'm actually fighting with qemu-gcc right now
[19:05] emiltin: abyxcos: well i'm trying to build a formula for building osrm
[19:05] abyxcos: It won't build with the 10.7 clang snapshot (something about pointers to registers), and llvm-gcc causes it to segfault.
[19:05] csharpsteen: Also, on XCode 4.1, you should be able to force a vanilla GCC using brew install --use-gcc. Thant might work if there are problems with using Clang to bootstrap GCC.
[19:05] emiltin: but it uses openmp, which clang doesn't support
[19:06] abyxcos: csharpsteen: Well, unless you don't have a gcc to begin with
[19:06] emiltin: csharpsteen: problem is it requires a gcc never than the system one
[19:06] AdamBucketz joined the chat room.
[19:06] emiltin: that's why i'm using your 4.6 formula
[19:06] abyxcos: emiltin: What about recompiling gcc with a clang-built gcc?
[19:07] csharpsteen: I'm still talking about building GCC 4.6.x, not OSRM.
[19:07] emiltin: oh ok
[19:07] emiltin: how is --use-gcc different than no option when building gcc 4.6
[19:07] emiltin: ?
[19:08] csharpsteen: It sets CC, etc to gcc-4.2 instead of gcc which points to LLVM-GCC on XCode 4.x
[19:08] csharpsteen: So, --use-gcc forces a build to use vanilla GCC instead of LLVM-GCC
[19:09] csharpsteen: Unless you're running the XCode 4.2 beta, which doesn't have a vanilla GCC
[19:09] onewheelskyward: I found that out the hard way.
[19:09] emiltin: ok. it's worth a try i guess. but my suspicion is that your formula somehow requires --enable-fully-dynamic-string
[19:10] emiltin: i did a quick attempt at https://github.com/emiltin/homebrew-alt/blob/master/duplicates/gcc.rb
[19:10] abyxcos: Isn't gcc-4.2 also llvm-gcc?
[19:10] emiltin: but the build failed
[19:10] csharpsteen: abyxcos: No.
[19:11] csharpsteen: emiltin: I agree---I will add the --enable-fully-dynamic-string flag to the GCC duplicates on Homebrew-alt
[19:11] emiltin: but test it, because when i tried it the 'ld' tool bugged out
[19:12] emiltin: your suggestion was to try to build gcc 4.6 with something like: brew install --use-gcc --use-clang --enable-cxx https://github.com/adamv/homebrew-alt/raw/master/duplicates/gcc.rb
[19:12] emiltin: ?
[19:12] emiltin: which would use a vanilla gcc, not llvm-gcc?
[19:13] emiltin: ups. i mean: brew install --use-gcc --enable-cxx https://github.com/adamv/homebrew-alt/raw/master/duplicates/gcc.rb
[19:14] csharpsteen: emiltin: I'm suggesting that if you still get weird behavior from GCC 4.6, an option to explore would be re-building it with vanilla GCC instead of Clang
[19:14] emiltin: right. i'm trying that right now
[19:14] csharpsteen: But the dynamic string business seems to match the malloc errors you are getting, so I think that is the most likely culprit at the moment
[19:15] emiltin: i might try with my modified version of your formula at https://github.com/emiltin/homebrew-alt/blob/master/duplicates/gcc.rb which adds the dynamic option
[19:15] emiltin: unfortunately it didn't solve it for me to just to add it to your fomula
[19:15] abyxcos: So, it helps to pick the right host architecture when using qemu...
[19:16] emiltin: will everything not be build for the native architecture if no options are passed?
[19:18] emiltin: csharpsteen: any plans to update the formula to gcc 4.6.1? not that i need it...
[19:19] csharpsteen: emiltin: Yeah, I'll do that as well since it sounds like the testsuite needs to be run
[19:19] emiltin: cool
[19:20] onewheelskyward: So, I've created a brew for http://csl.sublevel3.org/jp2a/ , but I'm wondering if it falls into the category of less maintained than the organizers prefer. It hasn't been updated since 2009, but it's a jpeg-to-ascii generator which isn't going to shift a whole lot. Should I audit the brew and submit it, or just keep it in my fork?
[19:23] emiltin: csharpsteen: here's my fomula for OSRM, that's using your fomula. https://github.com/emiltin/homebrew/blob/master/Library/Formula/osrm.rb
[19:23] emiltin: (work in progress i might add...)
[19:23] csharpsteen: onewheelskyward: Is that the one submitted here: Homebrew/legacy-homebrew#7956 ?
[19:23] emiltin: everything builds, but running the osrm-extract binary with a protobuf file as input causes there malloc bugs
[19:24] onewheelskyward: csharpsteen: Indeed. Nevermind, then!
[19:25] csharpsteen: onewheelskyward: I don't have a problem with formulae from inactive projects---unless they start growing a list of patches. Then I'm tempted to axe them.
[19:25] onewheelskyward: Roger that.
[19:25] onewheelskyward: I thought this one was a good candidate because of its low dependency count.
[19:26] csharpsteen: Yeah, looks simple to me.
[19:26] onewheelskyward: And oh-so-handy for putting pictures of coworkers in the MOTD on the dev server.
[19:26] MistyM: lol
[19:27] locojay joined the chat room.
[19:28] mavin is now known as mavin|gone.
[19:29] csharpsteen: emiltin: Note that a formula that drags something in from Homebrew-alt is unlikely to be accepted into mxcl/master
[19:30] emiltin: csharpsteen: hmm yes. it unfortunate that osrm doesn't compile with gcc 4.2
[19:30] csharpsteen: It would probably be accepted in Homebrew-alt
[19:31] emiltin: so you need to use the url when installing, right?
[19:31] csharpsteen: Yeah. For now.
[19:31] emiltin: i just succeeded in building gcc 4.6 with plain vanilla gcc 4.2, using brew install --use-gcc --enable-cxx https://github.com/adamv/homebrew-alt/raw/master/duplicates/gcc.rb
[19:32] csharpsteen: I'm working on a couple of things that should make it easier to use formulae from repositories other than mxcl master
[19:32] mdwright left the chat room.
[19:32] csharpsteen: See: Homebrew/legacy-homebrew#6086 and keep an eye on Homebrew/legacy-homebrew#7643
[19:33] wilmoore_ joined the chat room.
[19:33] wilmoore left the chat room. (Read error: Connection reset by peer)
[19:35] emiltin: looks cool!
[19:36] emiltin: i rebuild osrm with the gcc4.6 build with vanilla gcc4.2 but sadly, the errors persist: osrm-extract(42260) malloc: *** error for object 0x10097c8a0: pointer being freed was not allocated
[19:40] csharpsteen: Did that rebuild include the dynamic-string fixes?
[19:41] emiltin: no
[19:41] emiltin: i will try if i can build gcc4.6 with that option when i'm using plain gcc4.2 to build
[19:42] emiltin: does this way of adding the option look reasonable to you? https://github.com/emiltin/homebrew-alt/blob/master/duplicates/gcc.rb
[19:44] csharpsteen: I added it on line 115 as args << '--enable-fully-dynamic-string' if MACOS_VERSION >= 10.6
[19:45] emiltin: alright, basically the same thing. let me try to run it
[19:46] killtheliterate left the chat room.
[19:48] Swimming_Bird left the chat room. (Quit: Computer has gone to sleep.)
[19:50] darkwhite left the chat room. (Read error: Connection reset by peer)
[19:50] darkwhite joined the chat room.
[19:50] darkwhite left the chat room. (Changing host)
[19:50] darkwhite joined the chat room.
[19:51] wilmoore_ left the chat room. (Ping timeout: 252 seconds)
[20:03] mavin|gone is now known as mavin.
[20:03] emiltin: csharpsteen: your formula bombs out when using the dynamic option: https://gist.github.com/1260079
[20:04] AdamBucketz left the chat room.
[20:05] csharpsteen: emiltin: Hmm, it just made it to make check on my machine---most of the way through the test suite with no unexpected failures so far.
[20:06] emiltin: interesting. you're on xcode 4.1?
[20:07] csharpsteen: I'm still on Snow Leopard using XCode 3.2.6
[20:08] emiltin: ok. maybe there's some change that affect this issue
[20:09] emiltin: interestingly, the build process fails the same same errors:
[20:09] emiltin: collect2: ld terminated with signal 6 [Abort trap: 6]
[20:09] emiltin: ld(98192) malloc: *** error for object 0x10a108be0: pointer being freed was not allocated
[20:13] Swimming_Bird joined the chat room.
[20:13] jimubao joined the chat room.
[20:14] emiltin: csharpsteen: i'm guess i'm out of ideas right now. maybe if you update gcc to 4.6.1 things will be different

Karlsruher Institut für Technologie (KIT)
Institut für Theoretische Informatik

Dennis Luxen
Wissenschaftler

Am Fasanengarten 5
Gebäude 50.34, Zimmer 220
76131 Karlsruhe

Telefon: +49 721 608-46781
Fax: +49 721 608-43088
E-Mail: [email protected]

KIT - Universität des Landes Baden-Württemberg und nationales
Großforschungszentrum in der Helmholtz-Gemeinschaft

from osrm-backend.

i agree. i informed the maintainer of the gcc 4.6 formule of the problems, and he said he would be updating the formula, also to use gcc 4.6.1 which is out now.

from osrm-backend.

Great, looking forward to a GCC formula.

Am 04.10.2011 17:38, schrieb Emil Tin:

i agree. i informed the maintainer of the gcc 4.6 formule of the problems, and he said he would be updating the formula, also to use gcc 4.6.1 which is out now.

from osrm-backend.

Small update: OSRM builds and runs fine with gcc 4.6.1 on Linux. It should be manageable to get it to compile on os x.

from osrm-backend.

So, I was able to build on MacOS X without OpenMP support using the Apple supplied GCC 4.2 manually. As soon as the Homebrew Project accepts pending formula additions of the necessary libraries, I will submit a formula to build OSRM either using the Apple supplied compiler or by perhaps using a manually installed one.

from osrm-backend.

great news! certainly easier to build with the default gcc 4.2 - of course it will not be as optimized, but that matters less when you just want to experiment. what homebrew formula did you update? do you have a link to the pull request? thanks!

from osrm-backend.

The library that's still missing is STXXL. The pull request can be found here:

Homebrew/legacy-homebrew#8356

The other necessary tools and libraries are already in homebrew.

Am 03.11.2011 um 11:28 schrieb Emil Tin:

great news! certainly easier to build with the default gcc 4.2 - of course it will not be as optimized, but that matters less when you just want to experiment. what homebrew formula did you update? do you have a link to the pull request? thanks!

---

Reply to this email directly or view it on GitHub:
#13 (comment)

from osrm-backend.

cool. yes the osrm formula used the existing formula for stxxl, but i guess you needed a never version looking forward to trying to compile osrm on my machine.
https://github.com/emiltin/homebrew/blob/a752b03d5912cb8af9d68d85e5fe52fcef7a471d/Library/Formula/osrm.rb

from osrm-backend.

i also did a brew for stxxl back when i looked at compiling:

https://github.com/emiltin/homebrew/blob/a752b03d5912cb8af9d68d85e5fe52fcef7a471d/Library/Formula/stxxl.rb

you might want to add the keg_only option since it's a library without any binaries. see https://github.com/mxcl/homebrew/wiki/Formula-Cookbook

from osrm-backend.

Thanks for the keg_only hint. Will keep that in mind in the future.

i also did a brew for stxxl back when i looked at compiling:

https://github.com/emiltin/homebrew/blob/a752b03d5912cb8af9d68d85e5fe52fcef7a471d/Library/Formula/stxxl.rb

you might want to add the keg_only option since it's a library without any binaries. see https://github.com/mxcl/homebrew/wiki/Formula-Cookbook

---

Reply to this email directly or view it on GitHub:
#13 (comment)

from osrm-backend.

what's the status on compiling osrm on os x?

from osrm-backend.

Still waiting for homebrew to accept the libstxxl formula.

from osrm-backend.

saw the gcc 4.5 formulas was updated https://github.com/adamv/homebrew-alt/commit/62e6cac26b8a797b262b933fd4c693d04456639b

haven't tried compiling osrm with it.

from osrm-backend.

On OS X, with mostly Homebrew-installed dependencies ... would one simply need to edit the SConstruct file to point at libxml2 library/header locations, as installed via Homebrew? Also, glad to see libstxxl is headed to Homebrew!

from osrm-backend.

@jlar you can install osrm on mac using my formula from github:

brew install https://raw.github.com/emiltin/homebrew/master/Library/Formula/osrm.rb --emil

it's work in progress.. but it works for me. it pulls osrm from my fork, which was updated yesterday. i'm going to look into creating a makefile that can be part of the osrm repo, instead of compiling things manually in the formula.

note that my formula compiles osrm with the default gcc 4.2, but without openmp and optimization.

from osrm-backend.

maybe the best way forward would be to get building via scons working on mac, to be consistent across platforms. otherwise rake would be an altenative to makefiles.

from osrm-backend.

thanks a lot @emiltin - I went ahead and was attempting your formula yesterday, before your reply. I thought I was unable to install because of a bad setup, on my part, of libxml2 but it turns out that all I was missing was the --emil flag ... it has now installed :)

from osrm-backend.

Implemented with 25d15fe

from osrm-backend.