First, make sure you are in the manual lobby, not the automatic lobby.
Then, during table configuration, in the area where it lists people who have joined the table, there should be a link "play against the computer" (or something similar) in each player's area.