How to run rubicon-objc on linux about rubicon-objc HOT 14 CLOSED

I'm going to close this as a duplicate of #176, which is a more accurate representation of the core problem (supporting GNUStep), plus it frames the problem as a potential enhancement, rather than a bug.

from rubicon-objc.

Rubicon is supposed to work on Linux, but to be honest that isn't tested very much, since the main use case for Rubicon is accessing the APIs on Apple systems (Mac, iOS, etc.). Any bug reports or feedback for Linux are welcome though, we would like to support Linux too if possible.

The error you're getting means that Rubicon can't find a certain C function in the Python binary. There's multiple reasons why this could be happening. Maybe your Python binary is "stripped" and doesn't include a complete symbol table. It might also be because the function in question is not part of Python's public C API - on Mac that doesn't make a difference and the function can be used anyway, but perhaps Linux builds enforce the API more strictly and don't let external users access functions that aren't in the public API.

It would be useful to know a little more about your setup. What Ubuntu version are you using? What Python version are you using, and how did you install it? If you built Python yourself from source, did you use any custom build flags, or did you make any modifications to the source code?

Also, could you run the following code in your Python prompt, and post the output here? (This code tries to look up two C functions in the Python binary, one that's in the public API and one that isn't. I'm trying to find out if your Python binary has only public symbols, or no symbols at all.)

>>> import ctypes
>>> ctypes.pythonapi
>>> ctypes.pythonapi.Py_IsInitialized
>>> ctypes.pythonapi.PyType_stgdict

from rubicon-objc.

Thanks for responding!

I am using Ubuntu 18.04 desktop. I use Python 3.6.4 via anaconda 4.4.10 without any customization. Here are the outputs:

Python 3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ctypes
>>> ctypes.pythonapi
<PyDLL 'None', handle 7fe393ba1170 at 0x7fe39243e7b8>
>>> ctypes.pythonapi.Py_IsInitialized
<_FuncPtr object at 0x7fe3926b3818>
>>> ctypes.pythonapi.PyType_stgdict
Traceback (most recent call last):
File "", line 1, in
File "/home/jie/anaconda3/lib/python3.6/ctypes/init.py", line 361, in getattr
func = self.getitem(name)
File "/home/jie/anaconda3/lib/python3.6/ctypes/init.py", line 366, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: PyType_stgdict

from rubicon-objc.

Interestingly, ctypes.pythonapi.PyType_stgdict is also the symbol that causes #104 when iOS apps are stripped for distribution in the app store...

from rubicon-objc.

OK, that confirms my theory that only the public API symbols are accessible. I don't know if that's the case on iOS too, I would expect that the iOS build strips all symbols on the main binary except the entry point (because you don't usually dlopen your main binary). PyType_stgdict is the only symbol that we look up on the pythonapi library, so it's a bit hard to tell what exactly is happening.

In any case, it should be relatively easy to emulate the PyType_stgdict function in Python. It's not very complex, it basically just checks some types and accesses a field. I'll see if I can get that working, so we don't have to access private symbols anymore.

from rubicon-objc.

@dgelessus Thank you for the fix! I've tested it on my Ubuntu system, it did fix the stgdict problem. However, it seems that rubicon-objc depends on library Foundation from pyobjc, which is again written for MacOS (there exists hard-coded mac-specific paths).

from rubicon-objc.

@Jie-Cheng Thanks for testing, good to hear that the first part is working 😃

Yes, rubicon-objc requires the Core Foundation and Foundation frameworks in addition to the Objective-C runtime (libobjc). There shouldn't be any Mac-specific paths hardcoded anywhere, we use the ctypes.util.find_library function to locate all libraries and frameworks. If you're talking about _lib_path and _framework_path here, these are fallbacks used only when find_library didn't find the library in question (this is necessary on iOS, where find_library never works due to sandboxing).

I'm also not very familiar with how Objective-C is used on Linux. How did you install the Objective-C runtime, Core Foundation, and Foundation? Did you use something like GNUstep, or did you use Apple's open-sourced libobjc and Core Foundation? (Apple's Foundation is closed-source, so there's no way to use that on Linux of course.) And where are these libraries located on your system, are they in the regular system library locations (/usr/lib, /usr/local/lib, etc.) or in a different place? Are the Linux versions of Core Foundation and Foundation regular libraries, or are they frameworks like on Mac?

Also, could you test if ctypes.util.find_library can find all the necessary libraries?

>>> from ctypes.util import find_library
>>> find_library("objc")
>>> find_library("CoreFoundation")
>>> find_library("Foundation")

All three find_library calls should return a full path to the library in question.

from rubicon-objc.

@dgelessus Thanks for responding! The previous problem was because I didn't install objective-c compiler at all (forgive me for being stupid, this is my first time to use objc). Now I've installed gobjc', 'gnustep' and 'gnustep-devel and there is no path issue. However, find_library can only find objc, not CoreFoundation or Foundation.

Interestingly, I was able to compile and run a simple HelloWorld program with objc which imports Foundation.h. Not sure if it implies that libfoundation is installed.

from rubicon-objc.

OK, so it seems that GNUstep's Foundation library is not called libFoundation.so. Since you're able to compile and run ObjC code that uses Foundation classes, can you run the following code and post the output?

#include <objc/runtime.h>

#import <Foundation/Foundation.h>

int main(int argc, char **argv) {
	@autoreleasepool {
		NSLog(@"%s", class_getImageName([NSObject class]));
		NSLog(@"%s", class_getImageName([NSString class]));
	}
	return 0;
}

This code looks up which libraries the NSObject and NSString classes are defined in and prints out their paths. In my case (OS X 10.11) the paths are /usr/lib/libobjc.A.dylib and /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation. The Foundation path is probably quite different on Linux.

Also it seems that unlike Apple's Foundation, GNUstep's Foundation doesn't depend on Core Foundation. This shouldn't be a problem though - Core Foundation is almost never used in rubicon-objc, the only place where it's required is the asyncio eventloop integration, so unless you're using that there's no need to install Core Foundation.

from rubicon-objc.

Hi @dgelessus , sorry for the late reply, I have been traveling. I have tested your code. Unfortunately, it does not compile because gcc does not recognize the class_getImageName method.

from rubicon-objc.

sorry for the late reply, I have been traveling.

No problem 😃

I have tested your code. Unfortunately, it does not compile because gcc does not recognize the class_getImageName method.

Ah, it seems that GNUstep's libobjc doesn't have that function. Can you try this code instead? (It uses NSBundle, which exists in GNUstep's Foundation.)

#import <Foundation/Foundation.h>

int main(int argc, char **argv) {
	@autoreleasepool {
		NSBundle *nsObjectBundle = [NSBundle bundleForClass:[NSObject class]];
		NSLog(@"%@", [nsObjectBundle bundleIdentifier]);
		NSLog(@"%@", [nsObjectBundle bundlePath]);
		NSLog(@"%@", [nsObjectBundle executablePath]);
		NSBundle *nsStringBundle = [NSBundle bundleForClass:[NSString class]];
		NSLog(@"%@", [nsStringBundle bundleIdentifier]);
		NSLog(@"%@", [nsStringBundle bundlePath]);
		NSLog(@"%@", [nsStringBundle executablePath]);
	}
	return 0;
}

from rubicon-objc.

OKAY, this time I got something:

2018-06-27 09:43:36.058 test[28991:28991] (null)
2018-06-27 09:43:36.058 test[28991:28991] /usr/share/GNUstep/Libraries/gnustep-base/Versions/1.25/Resources
2018-06-27 09:43:36.058 test[28991:28991] /usr/lib/libgnustep-base.so.1.25
2018-06-27 09:43:36.058 test[28991:28991] (null)
2018-06-27 09:43:36.058 test[28991:28991] /usr/share/GNUstep/Libraries/gnustep-base/Versions/1.25/Resources
2018-06-27 09:43:36.058 test[28991:28991] /usr/lib/libgnustep-base.so.1.25

from rubicon-objc.

OK, that explains it - GNUstep has the Foundation classes, but in the gnustep-base library and not as a framework. This shouldn't be a big problem, I think the only change we need is to use gnustep-base instead of Foundation when on Linux.

Thank you for the help with debugging this! I'll try to fix it soon.

from rubicon-objc.

Reopened as that was not the only fix required (#122 is still needed).

from rubicon-objc.