Pinpointing the very first handheld video game is surprisingly difficult because the answer changes based on how you define "video game," "handheld," and "commercial release." Here are the leading contenders, each significant in its own right.
Each of these devices contributed crucial innovations—programmability, dedicated hardware, LCD screens, interchangeable cartridges—that collectively evolved into the Game Boy, PSP, Switch, and modern mobile gaming. Rather than a single "first," it's more accurate to view portable gaming as an evolutionary chain.