No—one console = one active voice headset for Xbox Live party/game chat.
Only Player 1 (signed-in profile) can speak in game chat. Player 2 (guest) has no voice representation.
For couch co-op + Live chat: Use Discord on phones + one headset on console for game audio.