From what I have gathered so far, you will be able to customize each character in at least two ways. One of those ways is through a talent tree type system.
http://darthhater.com/2009/12/03/smuggler-class-trees/
That link is the most solid info I've seen on that subject thus far. Looks like there will be at least two major talent trees for each class, and I've seen speculation that there could also be minor trees inside each of the major ones.
The other way for customization will be character alignment type choices (light side/dark side). From what I understand, some specific abilities will be available according to your character's alignment. I have found several references where people mention that, but I forget where it was mentioned by Bioware.
http://darthhater.com/2009/12/18/charting-a-path-to-class-customization/
That link makes some pretty sensible conclusions from the information that has been confirmed thus far and seems to suggest that there could actually be many different paths to follow for each individual class. It does have a lot of speculation, though.
In addition to the skill trees and alignment differences, we also know that you will have the option of using companion characters in TOR (much like you could in KOTOR, but only one at a time in TOR). These characters will also react differently to you according to your alignment, though we don't know how much impact that will have on the game yet.
Bottom line: Yes, there are only four classes per side, but it looks like there are going to be quite a few ways to customize each class. That should allow for quite a few different play options.