A quick look at the INSTALL-Octave file included in the ltfat package gives some instructions for compiling and installing the package on Octave. I have added the steps described to the of-ltfat project build script. However, it seems these steps are only for adding the current checkout to the current octave instance, not for creating and installing a package. Looking in the mat2doc directory there is a make_all_packages.sh script.
Running this script gives this error:
mkdir -p /home/susnak/Dropbox/ltfat_win64
mkdir: cannot create directory ‘/home/susnak’: Permission denied
So it looks like it is set up only for a particular developers machine. Looking inside the script, we probably only need this line:
mat2doc.py `pwd`/.. mat –script=release_keep_tests.py –octpkg –unix
I’ve added a new project for mat2doc, which also requires bibtex2html, also added as a new project. This, in turn, requires ocaml – I’ll install that from source rather than adding as a project. It also requires hevea.
mat2doc puts the output package file in $HOME/publish with the name of the project directory of-ltfat, to install it we need to rename without of- on the beginning.
Fixed in build 42.
Neil Hopcroft
A digital misfit
Fixing of-optim
The of-optim package fails with:
invalid doc file format
make: *** [../doc/optim.texi] Error 25
This looks like an error extracting documentation from the code files. There is a bootstrap script in the src/ directory. But this bootstrap just does the same ‘make doc’, leading to the same error.
Looking further inside, it seems the problem is munge-texi.pl is identifying all the files as invalid, perhaps its validity test is incorrect. I have added a patch to remove the validity test and the documentation extraction appears to work from the command line.
Fixed in build 46.
Fixing of-odepkg
This error
/bin/sh: -O2: command not found
make: *** [cash/mebdfdae.o] Error 127
looks like a configuration error, checking the package src directory there is an autogen.sh script. Running ./autogen.sh, ./configure and make resolves this problem.
Fixed in build 20
Fixing of-galois
The installation of the of-galois package appears to work without error, but the “pkg(‘describe’,’galois’)” fails, a brief check shows that the package installs with the name of ‘galois field’.
Fixed in build 37.
Fixing of-multi-precision
The build of this project fails to find mkoctfile, in the src/Makefile there is a comment “# assumptions to make if not using ./configure script” after which it defines MKOCTFILE to $(HOME)/usr/bin/mkoctfile
Setting MKOCTFILE on the make command line can fix this, but only with make invoked directly – the octave pkg function calls make as part of its installation process.
An alternative to this is to make a link to mkoctfile in $HOME of the tomcat user.
Fixed in build 24.
Fixing of-fl-core build
The fl-core package is giving this error:
fl_compose.cc: In function ‘int get_available_cpus()’:
fl_compose.cc:757:19: error: ‘_SC_NPROCESSORS_ONLN’ was not declared in this scope
return sysconf( _SC_NPROCESSORS_ONLN);
^
fl_compose.cc:757:39: error: ‘sysconf’ was not declared in this scope
return sysconf( _SC_NPROCESSORS_ONLN);
^
make: *** [fl_compose.oct] Error 1
A quick search of the octave source code shows a potential alternative, _SC_NPROCESSORS_CONF, if _SC_NPROCESSORS_ONLN is not defined. But it also says “This works on glibc, Mac OS X 10.5, FreeBSD, AIX, OSF/1, Solaris, Cygwin, Haiku” for both CONF and ONLN.
For the moment I have replaced the call to sysconf with a hard-coded ‘1’, representing a single core. Not an ideal solution but it should work for my current purposes.
Fixed in build 23.
Fixing of-ngspice
The of-ngspice project fails with
/usr/bin/ld: cannot find -lngspice
collect2: error: ld returned 1 exit status
A search for libngspice fails to turn anything up – so perhaps we’re not building the library? Checking the ngspice project output there is no mention of a library there. However, there is a ‘–with-ngshared’ option to configure, which seems like what we want.
Fixed in build 17.
Fixing of-fem-fenics build
The first failure in the restructured octave forge packages is fem-fenics, with:
uBLAS_factory.cc: In member function ‘virtual octave_value uBLAS_factory::matrix(const dolfin::Matrix&) const’:
uBLAS_factory.cc:36:12: error: ‘const class dolfin::Matrix’ has no member named ‘data’
aa = A.data ();
^
make: *** [uBLAS_factory.o] Error 1
There is some discussion about this error, and some of the issues around it, which although it makes some sense seems like it hasn’t actually been fixed in the code. The fix appears to be removing the uBLAS backend, but that seems like a chunk of work that will come along sometime. In the meantime I just need the build to complete, so I have added a patch:
sed 's/A.data ..;/NULL; return retval;/' uBLAS_factory.cc.orig > uBLAS_factory.cc
which replaces the unfound value with NULL and returns without attempting to use it. This is not ideal but should be enough to get the build to move on.
Next the build was missing the dolfin package during the install stage, adding the correct PKG_CONFIG_PATH resolves this.
Then libQtCore was not found, adding /usr/local/Trolltech/Qt-4.8.6/lib/ to the LD_LIBRARY_PATH resolves this.
Now I am seeing
/fem-fenics-0.0.4/x86_64-unknown-linux-gnu-api-v50+/Function.oct: undefined symbol: _ZNK5boost3mpi12communicatorcvP19ompi_communicator_tEv
error: called from ‘doc_cache_create>create_cache’ in file /usr/share/tomcat7/.jenkins/workspace/octave/scripts/help/doc_cache_create.m near line 106, column 18
More work needed – this looks like it might be “boost::mpi::communicator::operator ompi_communicator_t*() const” missing from my boost_mpi library.
The clarity of a well ordered mind^wbuild – octave forge project restructuring progress
Following on from some of my previous planning, I have now created a bunch of new of-* packages. Where the name of package is taken from the git or mercurial repository name, using the of-* naming convention I have set up a page in Jenkins to show only the octave forge packages making it easier to see what is going on with them.
This leads me to successful builds with the majority of packages while a few errors remain:
- of-fem-fenics
- of-fixed
- of-fl-core
- of-galois
- of-ltfat
- of-multi-precision
- of-ngspice
- of-odepkg
- of-optim
- of-optiminterp
- of-stk
- of-video
I will take a closer look at these errors over the next few weeks.
During this move to the new structure I have lost the tests for the installed packages – I’m not quite sure what to do about this yet, I was thinking of setting up a new Jenkins project to run just the tests from the old octave-packages project, but again that seems like it doesn’t really reflect the reality of how changes happen in the code. More thinking needed.
Restructuring the CI build for Octave Forge packages
Following a discussion on the maintainers mailing list about some recent problems with sourceforge, the Octave Forge packages are moving away from the sourceforge version control system. I have, for a while, been thinking about restructuring my octave-packages project. The current structure has two layers, one for the svn code from sourceforge, then a second with a number of mercurial projects. From a CI build point of view, only the svn repository is checked for changes, although whenever a build is made the mercurial packages are updated.
According to the octave forge webpage, the svn repository still exists, although it is now mostly a place for unmaintained code. There is also now some new packages using git.
Now I’m thinking about switching over to a bunch of projects, one for each octave forge package, as well as keeping the first layer of the current set up for the svn packages. This would make the whole thing a bit more manageable as the current build takes hours, nothing is properly cleaned and everything gets rebuilt whether it is needed or not. One of the big problems is that the octave-packages project has a dependency on dolfin, which gets a lot of updates, meaning there are lots of rebuilds requested. Moving to individual packages would mean that could be cut down to a single package being rebuilt for every dolfin build.
The octave forge packages also have their own dependency map, which could then be reflected in the build structure.
It is going to take quite a lot of setting up to get the new projects arranged and building, but it should make for clearer builds. What is going to get more complicated, though, is collecting the test results, since that is currently done at the end of the octave-packages build. What I could do would be to keep the current test code and move that to a project that runs daily.