compilation about ccj HOT 20 CLOSED

My experience with autotools would be entirely from your Sparse_MFE using them. So I know the purpose and configure&&make&&make install but that's about it.
I'll start by reading through this?
https://www.gnu.org/software/automake/manual/html_node/Autotools-Introduction.html

Ian

from ccj.

from ccj.

As far as I understand it, in the old makefile, simfold is made into a library .a file with "make depend" every time the makefile is run and then it uses that library. As far as I can see the Martin Mann page has something to that effect but I have not read it yet.

Seems I've gotten the accounts all mixed up, this is supposed to be Ian not Hosna. Also sorry for possibly a lot of emails/notifications, I am new to this part of github.
Ian

from ccj.

Hi @IanWark,
we should refactor this, such that simfold is independently compiled as library. Then, it can be used by Origignal_CCJ and Sparse_CCJ. I noted that simfold in both CCJ variants is almost identical with the exception of your new change for shape constraints in Sparse_CCJ.

from ccj.

I set up autotools for compiling simfold separately. This is in a new branch "refactor"; before the refactoring is completed, this should not be merged!
Ian, this showed one problem with your changes that include SHAPE signal. Since simfold should be used as a library, the simfold lib code must not access source files of Sparse_CCJ. Currently, it includes cmd_line_options.h and uses a shape data structure that does not exist inside the library. Do you see the problem? Could you fix this? It would probably work to define the shape data structure inside the lib and introduce gloabl variables for the use_shape flag and a pointer to the shape data structure (see 'extern' variables in simfold/include/externs.h for other such global variables e.g. 'debug' or 'no_dangling_ends' that control the behavior of the lib)

from ccj.

one additional remark: if you checkout the refactor branch, you can compile the simfold lib and the drivers in subdir simfold with 'configure && make && make install' after running autoreconf -i

from ccj.

For this release we're getting rid of the shape stuff for now anyways, so all we need to do for now is delete that. In the future though that'll need to be sorted.

Ian

from ccj.

I see you've already done that, good. So it is probably fine right now. Just need to set up auto tools for Original_CCJ and Sparse_CCJ and update the readmes, and then work on refactoring the shape data next week or something.

from ccj.

Hi Ian, this sounds like a good plan :). The CCJ implementations can now use the 'autotoolized' simfold lib. You should find an example how to use external libs via autotools e.g. in Martin's tutorial. (The goal would be to compile&install simfold once and then use it from the CCJ variants. Of course, for shape also the simfold lib has to be extended once again.)

from ccj.

Okay so essentially I have no idea what I am doing.
I've uploaded the files I am trying to use in the Sparse_CCJ folder (with .txt at the end as github doesn't recognize the file type otherwise)

I check for the library in configure.ac. When I run ./configure, it says
"checking for main in -lsimfold... yes"
Meaning, as far as I understand, it has found the library.

However, when I run make, I get
"h_struct.h:4:21: fatal error: structs.h: No such file or directory"
structs.h is in simfold/include

I have tried a whole bunch of different things, but I'm not very certain where the source of my problem even is, as I am very unfamiliar with autotools. I assume in the makefile.am?

configure.ac.txt
Makefile.am.txt

from ccj.

Hi @IanWark,
some files had been missing in the simfold installation. I fixed the simfold autotools setup and set up autotools for sparse_ccj. I am now using pkgconfig, which means that simfold's (new) pkgconfig file simfold.pc has to be in the pkgconfig path. For this purpose, I configure sparse_ccj with something like
./configure PKG_CONFIG_PATH=$path_to_simfold_inst/lib/pkgconfig
(Default: $path_to_simfold_inst=/usr/local)

My changes are in branch refactor

from ccj.

Thanks,
I thought something like that was happening but I thought I fixed it and the problem was something else. Guess not. I pushed some changes today to the Sparse_CCJ makefile.am and configure.ac to fix a problem with it not finding the libraries, which seems to work fine now.

However, the new problem is it "cannot open file ./params/turner-loop-rna.dat", or other various simfold files in the simfold/params folder. These aren't C files or anything so I am not sure how to include them into the library.
The files are used in multiple places in simfold and only once in CCJ, in CCJ.cpp. If I copy the params folder into Sparse_CCJ and change the file paths in CCJ.cpp (without touching the simfold files that use them), it works totally fine. Is that enough of a solution or should it be done better?

I'll probably continue to look at this tomorrow but I figured I would let you know where I'm at.

Ian

from ccj.

Hi Ian, the best way is to install the parameter files together with the simfold lib. This is done by including the files as data; I am doing this now in the simfold Makefile.am to install them to a nice place:
$(prefix)/share/simfold/params

Unfortunately, this leaves us with another dependency, since CCJ assumes that the simfold parameters are found in a subdir of the current directory. One could introduce a command line argument to CCJ for setting the simfold installation dir (alternatively, tell the configure script, where the lib is found (introducing a configure argument --with-simfold=<path_to_sdimfold>; this saves the additional simfold argument).

In principle, I like your idea to set the path to the simfold lib via rpath. (Just note that this is not strictly necessary, you could as well set LD_LIBRARY_PATH.) However, the way you do this is not exactly clean; one should not rely on specific positions in SIMFOLD_LIBS (this could break to easily). A cleaner solution would be to pass the simfold path to the CCJ configure call (--with-simfold, see above)

from ccj.

one other issue, you should keep the line
AM_CXXFLAGS="$SIMFOLD_CFLAGS $AM_CXXFLAGS"

Otherwise, the simfold includes will not be found if simfold is installed in a non-standard location.

from ccj.

Hi, @IanWark
I pushed a solution using a new environment variable SIMFOLD_HOME.
Now, Sparse_CCJ should be configured like
./configure --prefix=/some/inst/path SIMFOLD_HOME=/simfold/inst/path

from ccj.

Hi,
I see what you mean with setting the rpath and I agree.

The problem is the dependency. How do we get the SIMFOLD_HOME path and use it in CCJ.cpp? Right now CCJ.cpp looks for files in specific places relative to the executable. I'm not sure how to get that path from the configure script to the actual compiled program. It could of course be a command line argument like you say but I'd prefer not, as the command line usage of CCJ is kind of complicated already.

Thanks,
Ian

from ccj.

Or am I misunderstanding what you were saying.
An example of when it calls those share/simfold/params files is:
strcpy (config_file, "./simfold/params/multirnafold.conf");
or
fill_data_structures_with_new_parameters ("./simfold/params/turner_parameters_fm363_constrdangles.txt");

from ccj.

I figured it out.

from ccj.

from ccj.

I pushed some changes and Sparse_CCJ finds the data files and works fine now as far as I can tell.

I also changed the error messages in configure.ac as they would not trigger before the more generic "PKG_CHECK_MODULES" error message, so it should hopefully be more clear to users if there is an error.

One other thing I noticed is when I run autoreconf -i on simfold I get messages about version mismatch errors for libtools.
"libtoolize: m4/libtool.m4' is newer: use --force' to overwrite
libtoolize: m4/ltoptions.m4' is newer: use --force' to overwrite
libtoolize: m4/ltversion.m4' is newer: use --force' to overwrite"

and if I try to make with that I get
"libtool: Version mismatch error. This is libtool 2.4.2, but the
libtool: definition of this LT_INIT comes from libtool 2.4.6.
libtool: You should recreate aclocal.m4 with macros from libtool 2.4.2
libtool: and run autoconf again."

I ran autoreconf -i --force and got the newer versions of those three files, put them in a just-cloned version's simfold/m4 folder and it worked on my computer. I'm probably going to push that but I wanted to double check that that makes sense.

Thanks
Ian

from ccj.