Neil Hopcroft

A digital misfit

Putting the Continuous Integration into the octave CI build

The whole point of a CI server is to do continuous integration, that is making sure everything still works whenever there are any changes upstream as well as any changes to a projects own code. Looking at the dependencies for the octave build, there are a number of things I had to build myself to get the build working on the AWS server I’m using. I have taken that as a list of packages to build in the CI server. The dependencies look like this:

  • octave-packages
    • octave
      • arpack
      • fltk
      • librsvg
        • pango
      • mesa
        • libxcb
          • xorg-macros
      • qhull
      • qrupdate

None of these projects appear to be under significant development at the moment so I have set them to poll for updates daily, and the dependencies are set up within Jenkins such that any rebuild will build all of the projects that depend upon it.
There is one significant omission from this list: QT. Although I found a number of places that give links to the QT 4.8 active development branch, none of them are working for me.


Octave build break in CI build

My Octave CI build has broken with this change:

“Changeset 19439:57b6e06ed811 by Mike Miller :
maint: update library version numbers for upcoming release

* libgui/Makefile.am (liboctgui_current): Bump to 1.
* libinterp/Makefile.am (liboctinterp_current): Bump to 3.
* liboctave/Makefile.am (liboctave_current): Bump to 3.
* configure.ac (OCTAVE_API_VERSION): Bump to api-v50+.”

This break lead to an error during tests:

libinterp/corefcn/tsearch.cc-tst ……………………….error: ‘nargin’ undefined near line 35 column 7
error: called from
__unimplemented__ at line 35 column 3
__run_test_suite__ at line 113 column 7
./fntests.m at line 57 column 1
error: scope not found!

Followed by a core dump:

*** Error in `/usr/share/tomcat7/.jenkins/workspace/octave/src/.libs/lt-octave-cli’: free(): invalid pointer: 0x00007f9c9c235400 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7d19d)[0x7f9c94cd219d]
/usr/share/tomcat7/.jenkins/workspace/octave/liboctave/.libs/liboctave.so.3(_ZN12octave_shlib9shlib_repD0Ev+0xc3)[0x7f9c9bdd8e03]
/lib64/libc.so.6(+0x38e59)[0x7f9c94c8de59]
/lib64/libc.so.6(+0x38ea5)[0x7f9c94c8dea5]
/usr/share/tomcat7/.jenkins/workspace/octave/libinterp/.libs/liboctinterp.so.3(octave_execute_interpreter+0x5f2)[0x7f9c9c76e7f2]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f9c94c76af5]
/usr/share/tomcat7/.jenkins/workspace/octave/src/.libs/lt-octave-cli[0x400ba1]

Given the nature of the change I would guess that some dependency information hasn’t been updated somewhere, either because there is another source change needed or because my build hasn’t been properly cleaned between builds – I noticed the Jenkins ‘clean build’ option was not set.


Octave CI build test results

Now that the CI build is set up, it is worth reviewing the test results. The ‘make check’ step of the build runs the test suite and displays the results (from build 127 on my Jenkins server):
Summary:

PASS 12717
FAIL 10
XFAIL 14
SKIPPED 270
...
208 (of 875) .m files have no tests

Where PASS means the test ran successfully, FAIL means it failed, XFAIL means an expected failure (that is probably a known bug or error in the test) and SKIPPED means a test that is not applicable to the current configuration.
Drilling a bit more into these results, FAIL and SKIPPED are the interesting results.

Test name Pass Fail Skipped
libinterp/corefcn/graphics.cc-tst 35 3 0
libinterp/dldfcn/audiodevinfo.cc-tst 0 0 4
libinterp/dldfcn/convhulln.cc-tst 0 0 4
libinterp/octave-value/ov-class.cc-tst 48 0 2
libinterp/octave-value/ov-java.cc-tst 0 0 2
libinterp/octave-value/ov-typeinfo.cc-tst 56 0 1
scripts/audio/@audioplayer/audioplayer.m 0 0 4
scripts/audio/@audiorecorder/audiorecorder.m 0 0 4
scripts/audio/wavwrite.m 4 0 7
scripts/deprecated/delaunay3.m 0 0 1
scripts/general/fieldnames.m 2 0 3
scripts/general/methods.m 1 0 1
scripts/geometry/convhull.m 0 0 1
scripts/geometry/delaunay.m 9 0 6
scripts/geometry/delaunayn.m 1 0 2
scripts/geometry/griddata.m 13 0 1
scripts/geometry/griddata3.m 0 0 2
scripts/geometry/griddatan.m 0 0 2
scripts/geometry/voronoi.m 6 0 2
scripts/gui/uimenu.m 0 3 0
scripts/image/imfinfo.m 0 0 1
scripts/image/imformats.m 0 0 5
scripts/image/imread.m 0 0 4
scripts/image/imwrite.m 9 0 10
scripts/java/javachk.m 5 0 1
scripts/java/usejava.m 5 0 1
scripts/plot/util/allchild.m 2 1 0
scripts/plot/util/copyobj.m 1 0 1
scripts/plot/util/findall.m 0 1 0
scripts/plot/util/graphics_toolkit.m 0 2 0
scripts/plot/util/hgsave.m 3 0 1
scripts/sparse/eigs.m 5 0 149
scripts/sparse/svds.m 1 0 5
scripts/testfun/test.m 57 0 1
jit.tst 0 0 42

From this there are two significant areas of functionality being skipped, eigs and jit. confgure gives these warnings: “WARNING: ARPACK library found, but does not seem to work properly — disabling eigs function” and “WARNING: Missing LLVM file TargetData.h. JIT compiler is disabled.”
Looking in the config.log file, I notice that configure is having trouble loading some relevant libraries during its testing, a number of these libraries are installed in /usr/local/lib, which isn’t on the library search path. Adding LD_LIBRARY_PATH=/usr/local/lib to the beginning of the configure line seems to improve things somewhat, but the build including that change is still in progress.
Further investigation of the LLVM failure shows that llvm/IR/Function.h requires support for ISO C++ 2011 standard in the compiler, which is not included in the gcc 4.8.2 I am using. This seems like a problem for another day.


Octave Packages CI build dependencies

Getting the CI build of Octave Packages working I have encountered a number of dependencies.

Package Maintained Version Successful Dependencies
actuarial No 1.1.0 Yes
ad No No Build error
ann No No Build error
audio No 1.1.4 Yes miscellaneous>=1.1.0
benchmark No 1.1.1 Yes
bim Yes 1.1.4 Yes fpl, msh
bioinfo No 0.1.2 Yes
cgi Yes 1.0.1 Yes
civil-engineering No 1.0.7 Yes
communications Yes 1.2.1 Yes signal >= 1.1.3
control Yes 2.8.0 Yes
data-smoothing Yes 1.3.0 Yes optim >= 1.0.3
database Yes 2.3.1 Yes
dataframe Yes 1.1.0 Yes
dicom Yes 0.1.1 Yes
divand Yes 1.1.2 Yes
econometrics Yes 1.1.1 Yes optim
engine No 1.0.9 Yes
fem-fenics Yes No Build error
fenv No 0.1.0 Yes
financial Yes 0.4.0 Yes io >= 1.0.18
fits Yes 1.0.6 Yes cfitsio pkg-config
fixed No Build error
fl-core Yes No Build error
fpl Yes 1.3.4 Yes
fuzzy-logic-toolkit Yes 0.4.5 Yes
ga Yes 0.10.0 Yes
galois No signal >= 1.0.0, build error
general Yes 1.3.4 Yes
generate_html Yes 0.1.7 Yes
generate_latex 0.0.1 Yes
geometry Yes 1.7.0 Yes general >= 1.3.0
gnuplot No 1.0.3 Yes
graceplot No io >= 1.0.0, general, Build error
graph 0.0.0 Yes
gsl No 1.1.0 Yes gsl pkg-config
image-acquisition Yes 0.2.1 Yes
image Yes 2.3.0 Yes
informationtheory No 0.1.8 Yes
instrument-control Yes 0.2.1 Yes
integration No 1.0.7 Yes
interval Yes 0.1.5 Yes mpfr pkg-config
io Yes 2.2.7 Yes
irsa No 1.0.7 Yes
java Yes
jhandles No java, Java support
level-set Yes
linear-algebra Yes 2.2.2 Yes general >= 1.3.0
lssa Yes 0.1.2 Yes
ltfat Yes
macosx No Invalid package name
mapping No 1.2.0 Yes
mechanics Yes 1.3.1 Yes linear-algebra >= 2.0.0, general >= 1.2.2, geometry >= 1.2.1
miscellaneous Yes 1.2.1 Yes general >= 1.3.1
missing-functions No 1.0.2 Yes
mpi Yes No mpic++ pkg-config
msh Yes 1.0.10 Yes splines
multicore No 0.2.15 Yes
multi-precision No Build error
mvn Yes 1.1.0 Yes
nan Yes No Invalid package name
ncarray Yes No Invalid package name
netcdf Yes 1.0.6 Yes netcdf pkg-config
ngspice No ngspice pkg-config
nlwing2 No 1.2.0 Yes
nnet No 0.1.13 Yes
nurbs Yes 1.3.10 Yes Build error
ocs Yes 0.1.3 Yes odepkg
oct2mat No No io >= 1.0.0, build error
octcdf Yes No netcdf pkg-config
octclip Yes 1.0.3 Yes
octgpr No 1.2.1 Yes
octproj Yes 1.1.2 Yes
ode 1.0.1 Yes
odebvp No 1.0.6 Yes
odepkg Yes 0.8.4 Yes patch pkg-config
optics Yes 0.1.1 Yes
optim Yes No struct >= 1.0.10, build error
optiminterp Yes No Build error
outliers No 0.13.9 Yes
parallel Yes 2.2.1 Yes
pdb No No Build error
perl No Build error
plot No 1.1.0 Yes
project-web No Build error
prony No Build error
quaternion Yes 2.4.0 Yes
queueing Yes 1.2.3 Yes
real2rgb No Build error
robotis 0.0.1 Yes quarternions_oo
secs1d Yes No bim, Build error
secs2d Yes No Build error
secs3d Yes 0.0.1 Yes bim, fpl
signal Yes 1.3.1 Yes control >= 2.4.5, general >= 1.3.2
simp No 1.1.0 Yes
sockets Yes 1.2.0 Yes
soctcl No Build error
sparsersb No rsb pkg-config
specfun Yes 1.1.0 Yes
special-matrix No 1.0.7 Yes
splines Yes 1.2.7 Yes
statistics Yes 1.2.4 Yes io >= 1.0.18
stk Yes No Build error
strings Yes 1.1.0 Yes
struct Yes 1.0.10 Yes
symband No 1.0.10 Yes
symbolic Yes
system-identification No Build error
tcl-octave No No Build error
triangular No Build error
tsa Yes No Build error
video No No Build error
vrml Yes 1.0.13 Yes linear-algebra, miscellaneous, struct, statistics
wavelet 0.1.0 Yes
windows Yes No Invalid package name
xraylib No No libxrl pkg-config
zenity No No general >= 1.3.0

Octave Packages Continuous Integration Build

Having set up a CI build for Octave and determined the available packages, the next step was to set up a CI build for the packages.
The packages themselves come from two sources, mercurial and subversion. I didn’t want a proliferation of Jenkins projects, so I set up a single project that polls the subversion code and triggers following the main Octave build. This project also updates the mercurial code when it launches. The frequency of builds this setup generates should be adequate at the moment, but can be reviewed later.
Once the code has been updated the build creates the octave packages, runs a package build, then attempts to install the package.
There are three kinds of issues I’ve encountered using this approach:

  • Octave package dependencies
  • Dependencies external to Octave
  • Incorrect package naming

Octave packages

There is a list of Octave packages, split as maintained and unmaintained packages. Most of the source code for these packages can be found in two places:

  • svn://svn.code.sf.net/p/octave/code/trunk/octave-forge
  • hg clone http://hg.code.sf.net/p/octave/*
Package Maintained Source
actuarial No http://hg.code.sf.net/p/octave/actuarial
ad No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/ad
ann No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/ann
audio No http://hg.code.sf.net/p/octave/audio
benchmark No http://hg.code.sf.net/p/octave/benchmark
bim Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/bim
bioinfo No http://hg.code.sf.net/p/octave/bioinfo
cgi Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/cgi
civil-engineering No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/civil
communications Yes http://hg.code.sf.net/p/octave/communications
control Yes http://hg.code.sf.net/p/octave/control
data-smoothing Yes http://hg.code.sf.net/p/octave/data-smoothing
database Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge
dataframe Yes http://hg.code.sf.net/p/octave/dataframe
dicom Yes http://hg.code.sf.net/p/octave/dicom
divand Yes http://hg.code.sf.net/p/octave/divand
econometrics Yes http://hg.code.sf.net/p/octave/econometrics
engine No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/engine
fem-fenics Yes http://hg.code.sf.net/p/octave/fem-fenics
fenv No http://hg.code.sf.net/p/octave/fenv
financial Yes http://hg.code.sf.net/p/octave/financial
fits Yes http://hg.code.sf.net/p/octave/fits
fixed http://hg.code.sf.net/p/octave/fixed
fl-core Yes http://hg.code.sf.net/p/octave/fl-core
fpl Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/fpl
fuzzy-logic-toolkit Yes http://hg.code.sf.net/p/octave/fuzzy-logic-toolkit
ga Yes http://hg.code.sf.net/p/octave/ga
galois http://hg.code.sf.net/p/octave/galois
general Yes http://hg.code.sf.net/p/octave/general
generate_html Yes http://hg.code.sf.net/p/octave/generate_html
generate_latex svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/generate_latex
geometry Yes http://hg.code.sf.net/p/octave/geometry
gnuplot No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/gnuplot
graceplot svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/graceplot
graph http://hg.code.sf.net/p/octave/graph
gsl No http://hg.code.sf.net/p/octave/gsl
image-acquisition Yes http://hg.code.sf.net/p/octave/image-acquisition
image Yes http://hg.code.sf.net/p/octave/image
informationtheory No http://hg.code.sf.net/p/octave/info-theory
instrument-control Yes http://hg.code.sf.net/p/octave/instrument-control
integration No http://hg.code.sf.net/p/octave/integration
interval Yes http://hg.code.sf.net/p/octave/interval
io Yes http://hg.code.sf.net/p/octave/io
irsa No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/irsa
java Yes
jhandles svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/jhandles
level-set Yes
linear-algebra Yes http://hg.code.sf.net/p/octave/linear-algebra
lssa Yes http://hg.code.sf.net/p/octave/lssa
ltfat Yes
macosx svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/MacOSX
mapping No http://hg.code.sf.net/p/octave/mapping
mechanics Yes http://hg.code.sf.net/p/octave/mechanics
miscellaneous Yes http://hg.code.sf.net/p/octave/miscellaneous
missing-functions No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/missing-functions
mpi Yes http://hg.code.sf.net/p/octave/mpi
msh Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/msh
multicore No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/multicore
multi-precision http://hg.code.sf.net/p/octave/multi-precision
mvn Yes http://hg.code.sf.net/p/octave/mvn
nan Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/NaN
ncarray Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/ncArray
netcdf Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/netcdf
ngspice http://hg.code.sf.net/p/octave/ngspice
nlwing2 No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/nlwing2
nnet No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/nnet
nurbs Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/nurbs
ocs Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/ocs
oct2mat No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/oct2mat
octcdf Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/octcdf
octclip Yes http://hg.code.sf.net/p/octave/octclip
octgpr No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/octgpr
octproj Yes http://hg.code.sf.net/p/octave/octproj
ode svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/ode
odebvp No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/odebvp
odepkg Yes http://hg.code.sf.net/p/octave/odepkg
optics Yes http://hg.code.sf.net/p/octave/optics
optim Yes http://hg.code.sf.net/p/octave/optim
optiminterp Yes http://hg.code.sf.net/p/octave/optiminterp
outliers No http://hg.code.sf.net/p/octave/outliers
parallel Yes http://hg.code.sf.net/p/octave/parallel
pdb No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/pdb
perl svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/perl
plot No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/plot
project-web http://hg.code.sf.net/p/octave/project-web
prony http://hg.code.sf.net/p/octave/prony
quaternion Yes http://hg.code.sf.net/p/octave/quaternion
queueing Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/queueing
real2rgb svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/real2rgb
robotis svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/robotics
secs1d Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/secs1d
secs2d Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/secs2d
secs3d Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/secs3d
signal Yes http://hg.code.sf.net/p/octave/signal
simp No http://hg.code.sf.net/p/octave/simp
sockets Yes http://hg.code.sf.net/p/octave/sockets
soctcl svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/soctcl
sparsersb svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/sparsersb
specfun Yes http://hg.code.sf.net/p/octave/specfun
special-matrix No http://hg.code.sf.net/p/octave/special-matrix
splines Yes http://hg.code.sf.net/p/octave/splines
statistics Yes http://hg.code.sf.net/p/octave/statistics
stk Yes http://hg.code.sf.net/p/octave/stk
strings Yes http://hg.code.sf.net/p/octave/strings
struct Yes http://hg.code.sf.net/p/octave/struct
symband No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/symband
symbolic Yes
system-identification svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/main/system-identification
tcl-octave No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/tk_octave
triangular svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/triangular
tsa Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/tsa
video No http://hg.code.sf.net/p/octave/video
vrml Yes http://hg.code.sf.net/p/octave/vrml
wavelet http://hg.code.sf.net/p/octave/wavelet
windows Yes svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/windows
xraylib No svn://svn.code.sf.net/p/octave/code/trunk/octave-forge/extra/xraylib
zenity No http://hg.code.sf.net/p/octave/zenity

Octave Continuous Integration build

The nature of the Octave project is that it is a large project with lots of dependencies, this is one of the aspects that attracts me to it as a project to consider contributing toward. This does, however, mean that building it is quite complicated.
My aim was to set up a CI build in my Jenkins server, running on the AWS cloud, to launch a build whenever there was a code change. There are a few challenges here, firstly not all of the dependencies are available within the AWS yum repository, secondly not all of the dependencies are correctly checked by the configure script, leaving unexpected build failures, finally the versions of those things available with the yum repository are not necessarily recent enough to satisfy the dependencies.
My approach to missing yum dependencies has been to obtain the source code for the missing items and build and install them as required, arguably it would have been better to find a more complete yum repository, and perhaps that is what I would do next time – I didn’t know the extent of the missing dependencies when I set out.
As for the unexpected build failures, there isn’t really much you can do except to fix them as you encounter them. There is probably some scope for updating the configure dependencies to better reflect the actual dependencies, but that is beyond what I am aiming to do here.
The build itself is quite simple and takes the form:

./bootstrap
./configure
make
make check
make install

The process I used was to get the build steps into Jenkins as early as possible, then run the builds there – that way there is a repeatable process and a good way of recording build outcomes. The following are some of the dependencies I encountered while getting the build set up.
Things available in the AWS yum repository:

  • gperf
  • flex
  • bison
  • gnuplot
  • gcc-gfortran
  • libpcre
  • fftw3
  • glpk
  • curl
  • blas
  • lapack
  • suitesparse
  • ncurses
  • readline
  • freetype

Things built from source:

  • icotool
  • makeinfo
  • texi2dvi
  • librsvg
  • qhull
  • hdf5
  • arpack
  • mesa
  • qt
  • qrupdate
  • fltk

For most of these packages the default installation path was /usr/local, which highlighted some configuration problems in the AWS system I was using, the newly built version weren’t being picked up by pkg-config and the libraries weren’t being found. I hacked around these problems by copying the .pc files to a pkgconfig directory on the path and specifying /usr/local/lib/ as the LD_LIBRARY_PATH when launching the builds.


Maven Continuous Integration builds

Jenkins CI includes Maven integration, to take advantage of it I have added a main Maven pom.xml build file. To build a set of projects add them as modules and select ‘pom’ packaging. The Maven build then resolves the dependencies among the included projects and builds them in the appropriate order, publishing the results to the repository as they are built.

When setting up a CI build with deployment goals I publish them to the SNAPSHOT repository, which automates handling of version identifiers for deployed items without overwriting existing versions. This will need a little more thinking to make real releases.

These builds didn’t pick up the JUnit tests I had liberally scattered through the source code, but a quick reorganisation of the test code to put it all under the test/ directory, and appropriate change to the pom.xml file, makes the tests build and run. Of course they are still failing, but at least they are running.

Adding some plugins to Jenkins helps to visualise how the project is going, I’m using JUnit Plugin and Test Results Analyzer Plugin, which give some nice analysis of how the tests are going and graphs to show progress over recent builds.


Mavenification

It turns out that it is not that difficult to convert a project to be built using Maven. Here are the steps I went through:

  1. Set up a repository – I’m using Artifactory
  2. Set permissions on the repository to allow a user to write to it
  3. Get the settings.xml file for Maven configuration
  4. Add the user authentication information to the downloaded settings file
  5. For each project in Eclipse
  6. Add a Maven nature
  7. Add the repository to the distributionManagement section of the generated pom.xml file
  8. Add a debug configuration for ‘Maven build…’
  9. Give a goal of ‘deploy’ and the path to the settings file from the repository
  10. Attempt a build and resolve any missing dependencies
  11. When the build is successful the output will be published to the repository

The two most complicated parts of this process were getting the user permissions right and resolving dependencies. The permissions is the usual dance of trying to use the right username and setting the correct permissions for all of the repositories, it is documented and fairly clear once you’ve got it all set up right. Atrifactory has an ‘access.log’ (which you can see in System Logs admin screen) which shows permission results, it is a good place to check if you are getting errors when deploying. During the setup phase I was building again with the same version number, which requires the ‘delete’ permission.

More interesting is resolving the dependencies, mostly searching Google for the package that reported an error gave a link on Maven Repository which contains an XML snippet to add the dependency information to the pom.xml file. I didn’t encounter any conflicts during this process as I was, for the most part, using versions already known to work but obtained from other sources. I’ll be writing a bit more about versions shortly, when I try to bring everything up to date.


Event finder – a revival

Some months ago I did quite a lot of work on a Java EE server to look for events at venues. This was going quite well but being significantly hampered by my poor internet connection – the next stage was to be a massive search and web crawl to collect a lot of data.

Now that I have an AWS heavy utilisation instance available it is time to revive this project. The plan for this looks roughly like this:

  • Get the code building in a new environment
  • Bring all the components up to date
  • Maven-ify the build and dependency management
  • Write some integration tests
  • Set up a continuous integration build
  • Get project deployable
  • Set up staging and live environments

The original work was done in a Linux (Ubuntu) environment, but my new laptop runs Windows 8.1 (yuck!), so I’m using that for getting the build working again from a clean machine. It is times like this that you are thankful you had the foresight to split everything platform dependent in the code into separate Settings and Implementation classes. Of course there are still a lot of failures when I try to run anything, but most of those are because I haven’t got PostgreSQL set up correctly yet.