Sans Pareil Technologies, Inc.

Key To Your Business

Xapian Search Engine



The Xapian build system (based on autoconf) works out of the box with Solaris 11. We have built Xapian using both GCC 4.7.3 and Oracle Solaris Studio 12.3 compiler suites. SPT in general uses Solaris Studio 12.3 to compile applications, since in our experience they tend to be more efficient and stable on Solaris.

Under certain operating conditions (heavily threaded environments), the file related system calls return a errno of 150 (symbolic name EINPROGRESS). Oracle documentation explains this status as follows:

An operation that takes a long time to complete (such as a connect) was attempted on a non-blocking object.



Xapian uses the stat system call to verify the status of the database directory and files before opening them. While testing our Xapian based applications we run into this issue quite frequently. We decided to modify the Xapian source code to try and ignore EINPROGRESS status, which in our testing has not lead to any problems (no database corruption or invalid search results).

The following modifications were applied to the Xapian sources. Note that these modifications and line numbers relate to version 1.2.13 of the xapian-core source distribution.

dbfactory.cc

The WritableDatabase constructor invokes stat on the database directory specified. The check for the errno associated with the stat invocation is performed in backends/dbfactory.cc on line 424 which was modified as follows:

424 if (errno != ENOENT)
425        throw DatabaseOpeningError("Couldn't stat '" + path + "'", errno);



to

424 if (errno != ENOENT && errno != EINPROGRESS)
425        throw DatabaseOpeningError("Couldn't stat '" + path + "'", errno);



ioutils.cc

The io_unlink utility function used to delete files, uses the unlink system call to delete the file. This also at times returns the EINPROGRESS status, which we continue to ignore. The check for the return from the unlink system call is performed on line 47 in common/io_utils.cc, which was modified as follows:

47    if (errno != ENOENT) {
48         throw Xapian::DatabaseError(filename + ": delete failed", errno);
49    }



to

47    if (errno != ENOENT && errno != EINPROGRESS) {
48         throw Xapian::DatabaseError(filename + ": delete failed", errno);
49    }



chert_database.cc

The ChertDatabase constructor uses stat to verify the database directory, which again may fail with the EINPROGRESS error code. The check is performed on line 135 of the backends/chert/chert_database.cc implementation file, which was modified as follows:

135    } else if (errno != ENOENT || mkdir(db_dir, 0755) == -1) {
136        fail = true;
137    }



to

135    } else if (mkdir(db_dir, 0755) == -1) {
136        fail = true;
137    }



chert_table.cc

The ChertTable::do_open_to_write method uses the open system call to open the database file for read/write. The open system call also returns the EINPROGRESS error status. We modified the backends/chert/chert_table.cc implementation file at line 1505 as follows to ignore this status:

1505    if (lazy && !create_db && errno == ENOENT) {
1506        revision_number = revision_;
1507        RETURN(true);
1508    }



to

1505    if (lazy && !create_db && (errno == ENOENT || errno == EINPROGRESS)) {
1506        revision_number = revision_;
1507        RETURN(true);
1508    }



libtool

The libtool script generated by the configure script also needs a small modification to build Xapian. The autoconf system properly determines that it needs the Apache stdcxx library to compile, while the libtool attempts to link against the default Cstd library. We removed the specification of the Cstd library from the libtool script for the build to succeed.

10219 postdeps="-library=Cstd -library=Crun"



to

10219 postdeps="-library=Crun"