It's hard to pinpoint a single inventor, but Ralph Baer ("Father of Video Games") is the top contender.
Earlier, Spacewar! (1962) used a custom joystick, but not for a dedicated home console. Baer's work pioneered the modern controller concept—a dedicated input device for a consumer console.
Later innovations (D-pad, analog sticks, triggers, haptics) evolved through many companies (Nintendo, Sega, Sony, Microsoft). The controller's history is continuous improvement.