SSL overview

SSL is an industry standard for sending wire- or socket-level encrypted data over client-to-server and server-to-server connections. Before the SSL connection is established, the server and the client exchange a series of I/O round trips to negotiate and agree upon a secure encrypted session. This is called the SSL handshake.


SSL handshake

When a client application requests a connection, the SSL-enabled server presents its certificate to prove its identity before data is transmitted. Essentially, the SSL handshake consists of the following steps:

For more specific information about the SSL handshake and the SSL/TLS protocol, see the Internet Engineering Task Force Web site.


SSL in Open Client/Open Server

SSL provides several levels of security.


SSL filter

When establishing a connection to an SSL-enabled Adaptive Server, the SSL security mechanism is specified as a filter on the master and query lines in the interfaces file (sql.ini on Windows). SSL is used as an Open Client/Open Server protocol layer that sits on top of the TCP/IP connection.

The SSL filter is different from other security mechanisms, such as DCE and Kerberos, which are defined with SECHMECH (security mechanism) lines in the interfaces file (sql.ini on Windows). The master and query lines determine the security protocols that are enforced for the connection.

For example, a typical interfaces file on a UNIX machine using transport layer interface (tli) and SSL looks like this:

SERVER <retries><time-outs>

query tli tcp /dev/tcp tli_add1 ssl master tli tcp /dev/tcp tli_add1 ssl

A typical sql.ini file on Windows NT using SSL looks like this:

[SERVER]
query=TCP,hostname,address1, ssl
master=TCP,hostname,address1, ssl

where hostname is the name of the server to which the client is connecting and address1 is the port number of the host machine. All connection attempts to a master or query entry in the interfaces file with an SSL filter must support the SSL protocol. A server can be configured to accept SSL connections and have other connections that accept plain text (unencrypted data), or use other security mechanisms.

For example, an Adaptive Server interfaces file on UNIX that supports both SSL-based connections and plain-text connections looks like:

SYBSRV1
    master tli tcp /dev/tcp \x00020abc123456780000000000000000 ssl
    query tli tcp /dev/tcp \x00020abc123456780000000000000000 ssl
    master tli tcp /dev/tcp \x00020abd123456780000000000000000

Or, the same entry with the new style of Sybase interfaces file on UNIX looks like:

SYBSRV1
    master tli tcp hostname 2748 ssl
    query tli tcp hostname 2748 ssl
    master tli tcp hostname 2749

An example of a socket-style interfaces file looks like:

SYBSRV1
    master tcp ether hostname 2748 ssl
    query tcp ether hostname 2748 ssl
    master tcp ether hostname 2749

In these examples, the SSL security service is specified on port number 2748(0x0abc). On SYBSRV1, Adaptive Server listens for clear text on port number 2749(0x0abd), which is without any security mechanism or security filter.


Validating the server by its certificate

Any Open Client/ Open Server connection to an SSL-enabled server requires that the server have a certificate file, which consists of the server’s certificate and an encrypted private key. The certificate must also be digitally signed by a CA.

Open Client applications establish a socket connection to Adaptive Server similarly to the way that existing client connections are established. Before any user data is transmitted, an SSL handshake occurs on the socket when the network transport-level connect call completes on the client side and the accept call completes on the server side.

To make a successful connection to an SSL-enabled server:

When establishing a connection to an SSL-enabled Adaptive Server, Adaptive Server loads its own encoded certificates file at start-up from:

UNIX – $SYBASE/$SYBASE_ASE/certificates/servername.crt

NT – %SYBASE%\%SYBASE_ASE%\certificates\servername.crt

where servername is the name of the Adaptive Server as specified on the command line when starting the server with the -S flag or from the server’s environment variable $DSLISTEN.

Other types of servers may store their certificate in a different location. See the vendor-supplied documentation for the location of your server’s certificate.


The trusted roots file

The list of known and trusted CAs is maintained in the trusted roots file. The trusted roots file is similar in format to a certificate file, except that it contains certificates for CAs known to the entity (client applications, servers, network resources, and so on). The System Security Officer adds and deletes CAs using a standard ASCII-text editor.

The trusted roots file for Open Client/Open Server is located in:

UNIX – $SYBASE/config/trusted.txt

NT – %SYBASE%\ini\trusted.txt

Currently, the recognized CAs are Thawte, Entrust, Baltimore, VeriSign and RSA.

By default, Adaptive Server stores its own trusted roots file in:

UNIX – $SYBASE/$SYBASE_ASE/certificates/servername.txt

NT – %SYBASE%\%SYBASE_ASE%\certificates\servername.txt

Both Open Client and Open Server allow you to specify an alternate location for the trusted roots file:

For a description of SSL and public-key cryptography, see the Open Client Client-Library Reference Manual.