I'd let him come out! You can tame zebra finches and other
such small birds and if he's happy being with you why deny
him that? I have a canary chick (well he's fully grown now)
called Roger, who sits on my watering can every day when I
change the water. All the other canaries fly off when I get
close but he and another Bourkes' happily sit on my shoulder!
Good luck to you with your little pet!