Three weeks might seem like a long time to you but not
necessarily to him. It sounds like he needs more time to not
take offense at you intruding on what he considers his
territory. That's not at all unusual so don't feel that you
have a big problem or a unique situation. For now, just open
the cage door and allow him to come out if and when he is
ready. Once he is out, then interact with him as usual.
Trusting you on neutral ground can require less of a leap of
faith from them than allowing you to put your hand in their
territory. Just be positive with him as soon as he is out
and give lots of attention. Patience will do the trick.