Watch David Byrne Perform ‘When We Are Singing’ on ‘Colbert’

The musician also explained why he chose to incorporate footage of ICE into his live performances