The first dedicated handheld electronic game system with interchangeable (cartridge‑like) games and an LCD screen is Nintendo's Game & Watch series, launched April 1980 with Ball.
Verdict: Game & Watch (1980) is widely recognized as the first successful handheld system (brand + multiple titles + form factor). Microvision (1979) was first cartridge‑based handheld but not a sustainable platform.