Possible? Yes. Easy? No. A handful of designers earn full-time income; most have day jobs. Here's the realistic landscape:
Keep day job. Design nights/weekends. Publish 1 game/year. Reinvest profits. After 3-5 games + audience, evaluate full-time. Most "full-time" designers still diversify income.
Passion fuels persistence. Persistence beats talent. Start your first prototype this week!