Never use glue on a Nintendo 64 controller joystick. It sounds like a quick fix, but it almost always destroys the module permanently. Here’s why adhesive fails and what to do instead.
Bottom line: Spend $12–15 on a steel-bowl replacement module. It outperforms the original design and solves the problem for good.