The outer table is usually the one that has:
The smallest number of qualifying rows, and/or
The largest numbers of I/Os required to locate rows.
The inner table usually has:
The largest number of qualifying rows, and/or
The smallest number of reads required to locate rows.
For example, when you join a large, unindexed table to a smaller table with indexes on the join key, the optimizer chooses:
The large table as the outer table, so that the large table is scanned only once.
The indexed table as the inner table, so that each time the inner table is accessed, it takes only a few reads to find rows.