This is one of these hardware homebrew projects that look so professional and solid, that you can imagine to buy it as is. A guy named Joe Stranger developed a detachable analog USB Joystick for the Sony PSP.

Joe Stranger got his inspiration by video gaming hardware modding legend Benjamin Heckendorn, who hacked a PS2 analog stick to work with the PSP recently.

See the YouTube video below that shows the analog Joystick for the PSP in action.
The PSP mod looks contained in the Joystick, but actually there needs to be a wire added inside the Sony PSP to make the USB port work for the analog Joystick.
More details on MaxConsole and on PSPUpdates.