I had something that started up a few days ago where my attempts to connect to any server via SSH in Terminal simply stalled, after the key exchange had successfully taken place. Not really sure what was wedged, but the solution turned out to be to first remove openssh with brew, then reinstall it, with keychain support:

brew remove --force openssh
brew install openssh --with-keychain-support

Next, we want to modify the LaunchAgents plist for openssh:

sudo sed -i '' 's/\/usr\/bin\/ssh-agent/\/usr\/local\/bin\/ssh-agent/' /System/Library/LaunchAgents/org.openbsd.ssh-agent.plist  

We can add a little helper script to ~/.bash_profile:

eval $(ssh-agent)
    function cleanup {
        echo \"Killing SSH-Agent\"
        kill -9 $SSH_AGENT_PID
    }
    trap cleanup EXIT

And finally, set up the system to use the updated ssh, rather than the OS X stock one, universally (some apps will look for ssh in /usr/bin, rather than respecting $PATH settings.

sudo mv /usr/bin/ssh /usr/bin/ssh_old
sudo ln -s /usr/local/bin/ssh /usr/bin/ssh

Seems to be working fine again now.