Xapian Search Engine
The Xapian build system (based on autoconf) works out of the box with Solaris 11. We have built Xapian using both GCC 4.7.3 and Oracle Solaris Studio 12.3 compiler suites. SPT in general uses Solaris Studio 12.3 to compile applications, since in our experience they tend to be more efficient and stable on Solaris.
Under certain operating conditions (heavily threaded environments), the file related system calls return a
errno of 150 (symbolic name EINPROGRESS). Oracle documentation explains this status as follows: An operation that takes a long time to complete (such as a connect) was attempted on a non-blocking object.
Xapian uses the
stat system call to verify the status of the database directory and files before opening them. While testing our Xapian based applications we run into this issue quite frequently. We decided to modify the Xapian source code to try and ignore EINPROGRESS status, which in our testing has not lead to any problems (no database corruption or invalid search results).The following modifications were applied to the Xapian sources. Note that these modifications and line numbers relate to version 1.2.13 of the xapian-core source distribution.
dbfactory.cc
The
WritableDatabase constructor invokes stat on the database directory specified. The check for the errno associated with the stat invocation is performed in backends/dbfactory.cc on line 424 which was modified as follows:424 if (errno != ENOENT) 425 throw DatabaseOpeningError("Couldn't stat '" + path + "'", errno);
to
424 if (errno != ENOENT && errno != EINPROGRESS) 425 throw DatabaseOpeningError("Couldn't stat '" + path + "'", errno);
ioutils.cc
The
io_unlink utility function used to delete files, uses the unlink system call to delete the file. This also at times returns the EINPROGRESS status, which we continue to ignore. The check for the return from the unlink system call is performed on line 47 in common/io_utils.cc, which was modified as follows:47 if (errno != ENOENT) { 48 throw Xapian::DatabaseError(filename + ": delete failed", errno); 49 }
to
47 if (errno != ENOENT && errno != EINPROGRESS) { 48 throw Xapian::DatabaseError(filename + ": delete failed", errno); 49 }
chert_database.cc
The
ChertDatabase constructor uses stat to verify the database directory, which again may fail with the EINPROGRESS error code. The check is performed on line 135 of the backends/chert/chert_database.cc implementation file, which was modified as follows:135 } else if (errno != ENOENT || mkdir(db_dir, 0755) == -1) { 136 fail = true; 137 }
to
135 } else if (mkdir(db_dir, 0755) == -1) { 136 fail = true; 137 }
chert_table.cc
The
ChertTable::do_open_to_write method uses the open system call to open the database file for read/write. The open system call also returns the EINPROGRESS error status. We modified the backends/chert/chert_table.cc implementation file at line 1505 as follows to ignore this status:1505 if (lazy && !create_db && errno == ENOENT) { 1506 revision_number = revision_; 1507 RETURN(true); 1508 }
to
1505 if (lazy && !create_db && (errno == ENOENT || errno == EINPROGRESS)) { 1506 revision_number = revision_; 1507 RETURN(true); 1508 }
libtool
The
libtool script generated by the configure script also needs a small modification to build Xapian. The autoconf system properly determines that it needs the Apache stdcxx library to compile, while the libtool attempts to link against the default Cstd library. We removed the specification of the Cstd library from the libtool script for the build to succeed.10219 postdeps="-library=Cstd -library=Crun"
to
10219 postdeps="-library=Crun"
Notes
We have so far been using only the default chert backend with Xapian. If you use the other backends such as brass or flint, you will need to apply modifications similar to the modifications we applied to the chert backend.
The following shell script may be used to build and install Xapian
#!/bin/ksh ./configure --prefix=/opt/xapian \ CC=cc CXX=CC CFLAGS="-m64" CXXFLAGS="-O -m64" \ LIBS="-lsocket -lnsl -lpthread" LDFLAGS="-m64" sed -e 's;postdeps="-library Cstd -library=Crun";postdeps="-library=Crun";g' libtool > /tmp/libtool \mv /tmp/libtool . gmake -j8 gmake check pfexec gmake install