Custom Songs in Guitar Hero Smash Hits (Xbox 360): Not Supported
You cannot add custom songs to Guitar Hero Smash Hits on Xbox 360.
Why
No DLC store / music store in Smash Hits (unlike World Tour, Warriors of Rock, GH5).
No SD card / USB custom song support on 360 version.
Song list is baked on-disc (48 tracks, all "greatest hits" from GH1/2/80s/Encore).
What You Can Do
Play the 48 on-disc tracks (includes "Raining Blood," "Number of the Beast," "Painkiller," etc.).
Unlock all songs via cheat code (if you just want access): At main menu, enter Blue, Blue, Red, Green, Green, Yellow, Yellow (or check GameFAQs for your region's exact combo).
Play other GH games with DLC: Guitar Hero 5, Band Hero, Warriors of Rock (360) support downloadable songs (delisted from marketplace, but previously purchased songs still work; some via disc export).
PC Alternative:Clone Hero (free, community-driven) supports custom charts, custom songs, all instruments, online multiplayer. Best modern GH experience.