Yes, it's completely normal. The Nintendo 64 has no BIOS menu, dashboard, or video signal without a cartridge inserted.
Once a game boots, you'll see the Nintendo 64 logo → game title screen. Welcome to 1996!