Text configuration objects control the terms that are placed in a TEXT index when it is built or refreshed, and how a full text query is interpreted.
When the database server creates or refreshes a TEXT index, it uses the settings for the text configuration object specified when the TEXT index was created. If a text configuration object is not specified, the database server chooses one of the default text configuration objects, based on the type of data in the columns being indexed. In an SAP Sybase IQ database, the default_char text configuration object is always used.
Text configuration objects specify which prefilter library and which term breaker are used to generate terms from the documents to be indexed. They specify the minimum and maximum length of terms to be stored within the TEXT index, along with the list of terms that should not be included. Text configuration objects consist of these parameters:
Document pre-filter – removes unnecessary information, such as formatting and images, from the document. The filtered document is then picked up by other modules for further processing. The document pre-filter is provided by a third-party vendor.
Document term-breaker – breaks the incoming byte stream into terms separated by term separators or according to specified rules. The document term-breaker is provided by the server or a third-party vendor.
Stoplist processor – specifies the list of terms to be ignored while building the TEXT index.