VS Code Git integration with ssh stops working

VS Code git integration stops working when adding new ssh keys and doesn't ask for the passphrase of your private key, just saying 'Permission denied'. Here's a solution. I recently updated my ssh keys for GitHub account on my Mac and after adding the public key in GitHub everything worked as expected from the command line.