Neil Hopcroft

A digital misfit

More on the dolfin build breaks

Time to roll up my sleeves and get my hands dirty in the code following my previous chasing of various build errors.
The build error I am seeing now is

/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx: In function ‘PyObject* _wrap_MeshGeometry_set(PyObject*, PyObject*)’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:18767:60: error: no matching function for call to ‘dolfin::MeshGeometry::set(std::size_t&, const std::vector&)’
(arg1)->set(arg2,(std::vector< double > const &)*arg3);
^

This is in the swig autogenerated file modulePYTHON_wrap.cxx, but I can’t work out why it is trying to use them. Disabling the Python wrappers in the cmake options allows the build to complete.
While looking at this, I noticed that cmake is complaining that PetSc was built without cusp support, so PetSc support is disabled. Looking further into it, cusp requires a Cuda compatible GPU, which is not available on the AWS cloud (leastwise, not on the machine I am using there, it might be on the chunkier machines, but I doubt it). So I am going to have to live without Petsc support in dolfin.
This also means that I can disable the petsc, petsc4py, slepc and slepc4py projects, since they are only used by dolfin.
The build is finally fixed in build #506.


Fixing of-fixed

The fixed package is the most complex of the packages to get installed. There has been some fairly significant code rot since its last release and it doesn’t build any more. There are 55 errors in my first attempt to build it.

/usr/local/include/octave-4.1.0+/octave/../octave/oct-cmplx.h:49:10: error: no match for ‘operator==’ (operand types are ‘const volatile FixedPoint’ and ‘const volatile FixedPoint’)
if (ax == bx) \
^

There are some problems with complex comparisons and HDF5 configuration. Fixing these at least gets us a build that completes, however we are still stuck with a problem during installation:

“‘dispatch’ undefined near line 2 column 1
error: called from ‘/usr/share/tomcat7/octave/fixed-0.7.10/PKG_ADD’ in file /usr/share/tomcat7/octave/fixed-0.7.10/PKG_ADD near line 2, column 1”

Looking inside fixed.cc and fsort.m there are some requests to include calls to dispatch() in the PKG_ADD file – disabling these resolves the undefined dispatch error message.
Now I see

/usr/share/tomcat7/octave/fixed-0.7.10/x86_64-unknown-linux-gnu-api-v50+/fixed.oct: failed to load: /usr/share/tomcat7/octave/fixed-0.7.10/x86_64-unknown-linux-gnu-api-v50+/fixed.oct: undefined symbol: _ZNK5ArrayI17FixedPointComplexE17resize_fill_valueEv

Which is “Array::resize_fill_value() const” missing – this is probably something I commented out in Array-f.cc to get that to build. Lets try putting that back in now. This brings us back to the original problem.
By some magic, this was fixed in build #133, which I think reflects some changes to octave to undo an incompatible change – following this change it might be possible to undo some of the ‘get-it-to-build’ fixes I have made which may have removed or broken functionality.


Build break – dolfin build #338

Build #338 of dolfin failed with this error:

[ 98%] Building CXX object dolfin/swig/modules/function/CMakeFiles/_function.dir/modulePYTHON_wrap.cxx.o
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/function/modulePYTHON_wrap.cxx: In function ‘PyObject* _wrap_Function_non_matching_eval(PyObject*, PyObject*)’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/function/modulePYTHON_wrap.cxx:11623:41: error: ‘const class dolfin::Function’ has no member named ‘non_matching_eval’
((dolfin::Function const *)arg1)->non_matching_eval(*arg2,(dolfin::Array< double > const &)*arg3,(ufc::cell const &)*arg4);
^

Which seems like it is probably caused by this commit:

Commit 88a38a716b887b9d8d7ba0d5ce9e33a410907bbf by gnw20
Remove deprecated functions slated for removal with 1.7 release.

Then, in build #340 the error becomes:

CMake Error at CMakeLists.txt:889 (message):
Generation of form files failed:

Traceback (most recent call last):

File “/usr/local/bin/ffc”, line 213, in
sys.exit(main(sys.argv[1:]))
File “/usr/local/bin/ffc”, line 187, in main
compile_form(ufd.forms, ufd.object_names, prefix, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/compiler.py”, line 154, in compile_form
analysis = analyze_forms(forms, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/analysis.py”, line 67, in analyze_forms
unique_elements = sort_elements(unique_elements)
File “/usr/local/lib/python2.7/site-packages/ufl/algorithms/analysis.py”, line 205, in sort_elements
sorted_elements = topological_sorting(nodes, edges)
File “/usr/local/lib/python2.7/site-packages/ufl/utils/sorting.py”, line 36, in topological_sorting
S = nodes[:]

TypeError: ‘set’ object has no attribute ‘__getitem__’

Traceback (most recent call last):

File “/usr/share/tomcat7/.jenkins/workspace/dolfin/cmake/scripts/generate-form-files”, line 80, in
raise RuntimeError(“Unable to compile form: %s/%s” % (root, f))

RuntimeError: Unable to compile form: dolfin/ale/Poisson2D.ufl

This build was caused by completion of ffc build #45, looking at the changelog for ffc, this change is the most likely candidate:

Commit bd892c555ed4f751e7dc149c15b2b0799da75a68 by martinal
Fixes and simplifications of element analysis.

I notice the ffc build doesn’t include a clean step, so trying to add that before investigating further.
This error was fixed in build #352, but I can’t see a specific change that would have caused the fix – this always worries me, having something magically fix itself without really understanding why. But I’m not going to investigate too much further – there were a lot of upstream changes feeding into this build, so it could be any one of those.
However, we are still left with a further error:

/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/function/modulePYTHON_wrap.cxx: In function ‘PyObject* _wrap_Function_non_matching_eval(PyObject*, PyObject*)’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/function/modulePYTHON_wrap.cxx:11551:41: error: ‘const class dolfin::Function’ has no member named ‘non_matching_eval’
((dolfin::Function const *)arg1)->non_matching_eval(*arg2,(dolfin::Array< double > const &)*arg3,(ufc::cell const &)*arg4);
^
make[2]: *** [dolfin/swig/modules/function/CMakeFiles/_function.dir/modulePYTHON_wrap.cxx.o] Error 1

This changes in build #362, to become:

/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:27303:64: error: no matching function for call to ‘dolfin::Cell::get_vertex_coordinates(double*&) const’
((dolfin::Cell const *)arg1)->get_vertex_coordinates(arg2);
^
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:27303:64: note: candidate is:
In file included from /usr/share/tomcat7/.jenkins/workspace/dolfin/dolfin/function/FunctionSpace.h:38:0,
from /usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:3862:
/usr/share/tomcat7/.jenkins/workspace/dolfin/dolfin/mesh/Cell.h:338:10: note: void dolfin::Cell::get_vertex_coordinates(std::vector&) const
void get_vertex_coordinates(std::vector& coordinates) const
^
/usr/share/tomcat7/.jenkins/workspace/dolfin/dolfin/mesh/Cell.h:338:10: note: no known conversion for argument 1 from ‘double*’ to ‘std::vector&’
make[2]: *** [dolfin/swig/modules/mesh/CMakeFiles/_mesh.dir/modulePYTHON_wrap.cxx.o] Error 1

And, finally, in build #366 to become:

Linking CXX shared library libdolfin.so
/usr/bin/ld: cannot find -lvtkftgl
/usr/bin/ld: cannot find -lvtkftgl
collect2: error: ld returned 1 exit status

Which looks to be something missing from the vtk build. Again, vtk doesn’t include a clean step (why is this missing from the default Jenkins project setup?), so I’m going to try adding that and rebuilding before I do any further investigating.
Build #375 introduces:

[ 79%] Building CXX object dolfin/CMakeFiles/dolfin.dir/fem/FiniteElement.cpp.o
/usr/share/tomcat7/.jenkins/workspace/dolfin/dolfin/fem/FiniteElement.cpp: In member function ‘void dolfin::FiniteElement::tabulate_dof_coordinates(boost::multi_array&, const std::vector&, const dolfin::Cell&) const’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/dolfin/fem/FiniteElement.cpp:52:17: error: ‘const class ufc::finite_element’ has no member named ‘tabulate_dof_coordinates’
_ufc_element->tabulate_dof_coordinates(coordinates.data(),
^
make[2]: *** [dolfin/CMakeFiles/dolfin.dir/fem/FiniteElement.cpp.o] Error 1

Which is fixed again in #376.
Then #380 gives a lot more library problems:

Linking CXX shared library libdolfin.so
/usr/bin/ld: cannot find -lvtkGUISupportQtOpenGL
/usr/bin/ld: cannot find -lvtkRenderingOpenGL
/usr/bin/ld: cannot find -lvtkRenderingLIC
/usr/bin/ld: cannot find -lvtkgl2ps
/usr/bin/ld: cannot find -lvtkRenderingContextOpenGL
/usr/bin/ld: cannot find -lvtkRenderingVolumeOpenGL
/usr/bin/ld: cannot find -lvtkftgl
/usr/bin/ld: cannot find -lvtkRenderingGL2PS
/usr/bin/ld: cannot find -lvtkGUISupportQtOpenGL
/usr/bin/ld: cannot find -lvtkRenderingOpenGL
/usr/bin/ld: cannot find -lvtkRenderingLIC
/usr/bin/ld: cannot find -lvtkgl2ps
/usr/bin/ld: cannot find -lvtkRenderingContextOpenGL
/usr/bin/ld: cannot find -lvtkRenderingVolumeOpenGL
/usr/bin/ld: cannot find -lvtkftgl
/usr/bin/ld: cannot find -lvtkRenderingGL2PS
collect2: error: ld returned 1 exit status

These were introduced by changes in vtk build #92. The most likely candidate for this error is “Commit 7bb212d348f7e913edc6504b14fbec797ee44f78 FindPythonLibs Py3k fixes from cmake master.” which alters library and include directories for python 3, but we’re using 2.7.
Build #431 introduces another error:

CMake Error at CMakeLists.txt:883 (message):
Generation of form files failed:

Traceback (most recent call last):

File “/usr/local/bin/ffc”, line 213, in
sys.exit(main(sys.argv[1:]))
File “/usr/local/bin/ffc”, line 187, in main
compile_form(ufd.forms, ufd.object_names, prefix, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/compiler.py”, line 174, in compile_form
wrapper_code = generate_wrapper_code(analysis, prefix, object_names, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/wrappers.py”, line 40, in generate_wrapper_code
return _generate_dolfin_wrapper(analysis, prefix, object_names, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/wrappers.py”, line 47, in _generate_dolfin_wrapper
(capsules, common_space) = _encapsulate(prefix, object_names, analysis, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/wrappers.py”, line 80, in _encapsulate
(i, form_data) in enumerate(form_datas)]
File “/usr/local/lib64/python2.7/site-packages/ffc/wrappers.py”, line 100, in _encapsule_form
make_classname(prefix, “form”, i),

NameError: global name ‘make_classname’ is not defined

Which was probably introduced by changes in ffc build #65 and persists until build #441, when we return to the missing vtk libraries error.
Build #468 again introduces what appears to be an ffc error:

CMake Error at CMakeLists.txt:883 (message):
Generation of form files failed:

Traceback (most recent call last):

File “/usr/local/bin/ffc”, line 213, in
sys.exit(main(sys.argv[1:]))
File “/usr/local/bin/ffc”, line 187, in main
compile_form(ufd.forms, ufd.object_names, prefix, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/compiler.py”, line 159, in compile_form
ir = compute_ir(analysis, prefix, parameters)
File “/usr/local/lib64/python2.7/site-packages/ffc/representation.py”, line 108, in compute_ir
for (i, fd) in enumerate(form_datas)]
File “/usr/local/lib64/python2.7/site-packages/ffc/representation.py”, line 381, in _compute_form_ir
ir[“max_%s_subdomain_id” % integral_type] = form_data.max_subdomain_ids.get(integral_type, 0)

AttributeError: ‘FormData’ object has no attribute ‘max_subdomain_ids’

Which remains in build #472.
To attempt to resolve the link errors for vtk libraries, I have made a number of symlinks removing the version numbers from the library names, but I can’t test this until the most recent ffc error has been resolved.
The symlinks resolve the link errors in build #475, but I’m left with some further errors:

/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx: In function ‘PyObject* _wrap_CellType_refine_cell(PyObject*, PyObject*)’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:16065:41: error: ‘const class dolfin::CellType’ has no member named ‘refine_cell’
((dolfin::CellType const *)arg1)->refine_cell(*arg2,*arg3,*arg4);
^
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx: In function ‘PyObject* _wrap_MeshGeometry_x__SWIG_0(PyObject*, int, PyObject**)’:
/usr/share/tomcat7/.jenkins/workspace/dolfin/build/dolfin/swig/modules/mesh/modulePYTHON_wrap.cxx:18402:47: error: lvalue required as unary ‘&’ operand
result = (double *) &(arg1)->x(arg2,arg3);
^

I’ll tackle this in another post – this has gotten long enough already.


More on oftests hanging

So it seems that updating SymPy hasn’t resolved the hang problem. So I have now added a workaround to skip the hanging test. This should at least get the tests running to completion again and producing results – the skipped test will show up as an error-ing test in the results.
Except that this isn’t the only test that runs into this problem:

  • sympref
  • @sym/dsolve

Forcing an error during these tests allows the project build to run to completion.


Build break – mesa #181

The mesa project build is failing with “make[5]: *** No rule to make target `brw_cs.cpp’, needed by `brw_cs.lo’. Stop.”
Looking at the change logs, this seems like it was caused by Commit dc70c86b9b485cb5006a55cc2efd1f154dbfd469 “i965: Move compute shader code around” – where brw_cs.cpp turned into brw_cs.c
Build #189 introduces a new error “make[3]: *** No rule to make target `nir/nir_lower_samplers.cpp’, needed by `nir/nir_lower_samplers.lo’. Stop.”, caused by Commit de7ffdb38343b7f9292184ca827987de9852c80c “nir: rename nir_lower_samplers.c{pp,}”, which again turned a file from .cpp to .c
Checking the project configuration, it is also missing a clean step, which is exactly the kind of thing that fixes this sort of problem.
Fixed in build #205


New project: SymPy

Following some warnings in the oftests build, I have added SymPy to the CI build. This is a standard python setup.py build.
Running this build gives

Processing dependencies for sympy==0.7.7.dev0
Searching for mpmath>=0.19
Reading https://pypi.python.org/simple/mpmath/
/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py:2512: PEP440Warning: ‘mpmath (0.4.src)’ is being parsed as a legacy, non PEP 440, version. You may find odd behavior and sort order. In particular it will be sorted as less than 0.0. It is recommend to migrate to PEP 440 compatible versions.
PEP440Warning,

…but then goes on to install mpmath-0.19-py2.7.egg.
Rebuilding it doesn’t give the same warnings, so it has resolved this dependency problem.


oftests hanging

I have set the Octave Forge tests to run every week, since the complete run of all tests should take nearly a day. It has recently been hanging, though, with an error about closing an IPC pipe:

Running some tests that reset the IPC and produce output
Resetting the octsympy communication mechanism
Closing the Python pipe…
Build was aborted

Earlier in the log there is a message

OctSymPy: Communication established. SymPy v0.7.6.
skipping: char(x) of assumptions suboptimal in <= 0.7.6

So it is worth updating to the latest version of SymPy anyway, and, if we’re lucky, that will fix the IPC problem too.


Build break – pango #82

Pango build #82 fails with

./.libs/libpangoft2-1.0.so: undefined reference to `hb_ft_face_create’
./.libs/libpangoft2-1.0.so: undefined reference to `hb_ft_face_create_cached’
collect2: error: ld returned 1 exit status

There are no changes to pango itself in this build, so it is caused by changes to harfbuzz in build #28.
Looking at the changelog there, I see that Commit 2a9627c5641cd000e2adff0e42a0dc687b53ec70 includes a comment “Hope that doesn’t break too much client code.”, which is a somewhat suspicious. Although looking at the code itself, I don’t see any actual changes to the missing functions. Looking more closely at the configure logs, it seems that harfbuzz has lost its freetype dependency. The config.log in harfbuzz shows a number of missing pkg_config packages. Updating the PKG_CONFIG_PATH for the configure step fixes this, and, in turn, the pango build.
Fixed in build #85.


Build break – freetype #18

The freetype project build #18 is giving “libtool: link: require no space between `-L’ and `-lpng12′”, -L is intended to specify a path but no path is filled in there, thus it thinks -lpng12 is the path rather than a further option. Looking at the configure stage of the build shows:

Package libpng was not found in the pkg-config search path.
Perhaps you should add the directory containing `libpng.pc’
to the PKG_CONFIG_PATH environment variable
No package ‘libpng’ found

Which would give an explanation for the missing path. Specifying PKG_CONFIG_PATH for configure resolves this problem.
Fixed in build #19.