SAP Sybase IQ can use external pre-filter and term-breaker libraries written in C or C++ to prefilter and tokenize the documents during index creation or query processing. These libraries can be dynamically loaded into the process space of the database server.
The external dynamically loadable pre-filter and term-breaker libraries are specified in the text configuration, and need to be loaded by the database server. Each library contains an exported symbol that implements the external function specified in the text configuration object. This function returns a set of function descriptors that are used by the caller to perform the necessary tasks.
The external pre-filter and term-breaker libraries are loaded by the database server with the first CREATE TEXT INDEX request, when a query for a given column is received that requires the library to be loaded, or when the TEXT index needs to be updated.
The libraries are not loaded when an ALTER TEXT CONFIGURATION call is made, nor are they automatically unloaded when a DROP TEXT CONFIGURATION call is made. The external pre-filter and term-breaker libraries are not loaded if the server is started with the startup option to disallow the loading of external libraries.
Because these external C/C++ libraries involve the loading of non-server library code into the process space of the server, there are potential risks to data integrity, data security, and server robustness from poorly or maliciously written functions. To manage these risks, each server can explicitly enable or disable this functionality.
The ISYSTEXTCONFIG system view stores information about the external libraries associated with a text configuration object.