Can't access Azure Key Vault from Azure Data Lake Analytics
I have a U-SQL script with a custom extractor that accesses Azure Keystore to get some credentials.
I followed this tutorial . And I have the equivalent code to get the token from AD and then call the provided URI for the actual credentials:
public static async Task<string> GetToken(string authority, string resource, string scope)
{
var authContext = new AuthenticationContext(authority);
var clientCred = new ClientCredential(applicationId, authenticationKey);
AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);
if (result == null)
{
throw new InvalidOperationException("Failed to obtain the AD token");
}
return result.AccessToken;
}
public static async Task<string> GetSecret(string secretUri)
{
var keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(GetToken)
);
var sec = await keyVaultClient.GetSecretAsync(secretUri);
return sec.Value;
}
My credentials were successfully launched in the store and I have a URI to access them - something like:
https://my-key-vault-name.vault.azure.net:443/secrets/MyCredentialsName/123abc
I registered my app with Azure AD and got the App ID and Authentication Key for it, and I allowed my app to read the secret from Key Vault. In my U-SQL script, I referenced all the required assemblies.
When I run my script locally, everything works fine (meaning the connection from the local machine to AD and to Key Vault is fine), but when I submit it for execution to a remote Data Lake Analytics account, I got the following error:
The remote name could not be resolved: 'My-key-vault-name.vault.azure.net'
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) at System.Net.Http.HttpClientHandler.GetResponseCallback (IAsyncResult ar)
My admin rights in the Azure resource group are limited, but I can access the Firewall tab on the Data Lake Analytics server blade - I tried to turn the firewall on and off by turning it on / off Allow access to Azure services
, but the error persists.
As I refer to as dependencies Microsoft.Azure.KeyVault 2.0.6
, Microsoft.Azure.KeyVaultWebKey 2.0.4
, Microsoft.IdentityModel.Clients.ActiveDirectory 3.13.9
.
Any ideas on how I can try and resolve this?
source to share
The U-SQL code running in ADLA prevents you from connecting to resources outside the container / VM. The reason is that:
Custom U-SQL code calls can scale from over 100s to 1000s of containers, calling millions of millions. This can easily lead to a (hopefully inadvertently) distributed denial of service connection to the service you are trying to achieve, resulting in a possible DDOS service and blocking of Azure IP ranges.
Local launch is not currently running on a container, so it does not have this limitation.
What are you trying to achieve with this call? Note that the data in the store may already be transparently encoded using Azure Key Vault.
source to share