If we try to open an HTTPS connection with ureq, and the host part of the URL is an IP address, then we get an InvalidDNSNameError. Let’s figure out what’s going on and how to work around it.

The full code for this post is here.

To start off, let’s see the actual error. We use ureq::get to construct the GET request to https://1.1.1.1, we execute it with call, and we convert the response to a string to make sure there’s no laziness happening.

fn experiment1() -> OrError<()> {
    let resp: String = ureq::get("https://1.1.1.1").call()?.into_string()?;
    println!("Response:\n{}", resp);
    Ok(())
}

Running it, experiment1 fails with the following error:

$ ./target/debug/post-48-demo experiment1
Error: https://1.1.1.1/: Dns Failed: InvalidDNSNameError

Caused by:
    InvalidDNSNameError

Where’s that error coming from? Following the code and skipping the uninteresting bits:

That is the call that fails with InvalidDNSNameError. The issue for this is webpki#54, and that roughly says that IP addresses aren’t valid CNs according to the spec.

So, what do we do? A comment in issue ureq#393 suggests a workaround, so let’s try that. The gist of it is that we’re going to replace the IP address in the URL with a dummy hostname, configure a DNS resolver to respond with the IP address for the dummy host, and configure a certificate verifier which ignores the hostname.

fn experiment3() -> OrError<()> {
    use certificate_verifier_no_hostname::CertificateVerifierNoHostname;
    use fixed_resolver::FixedResolver;
    use rustls::ClientConfig;
    use std::sync::Arc;
    use url::Url;

    // `original_url` is our original URL, `ip_addr` is the extracted
    // IP address, and `url` is the updated URL with the dummy
    // hostname.
    let original_url: Url = "https://1.1.1.1".parse()?;
    let ip_addr = original_url.host().unwrap().to_string();
    let mut url = original_url.clone();
    url.set_host(Some("dummy-hostname-for-workaround"))?;

    // Create a TLS config with the default certificate roots and a
    // certificate verifier that ignores hostnames.
    //
    // TODO Configure just your own CA here to get any security from
    // the certificate verification.  Otherwise, any valid certificate
    // will be accepted.
    let mut tls_config = ClientConfig::new();
    tls_config
        .root_store
        .add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);
    tls_config
        .dangerous()
        .set_certificate_verifier(Arc::new(CertificateVerifierNoHostname));

    // Execute the `ureq` call with the special TLS config and a
    // resolver that always returns `ip_addr`.
    //
    // The extra tricks here are that we have to specify a port so
    // that we can parse a `SocketAddr` (although it's not used for
    // anything), and we have to set the `Host` header in case the
    // remote host is dispatching on that.
    let resp: String = ureq::builder()
        .tls_config(Arc::new(tls_config))
        .resolver(FixedResolver {
            ip_addr: format!("{}:443", ip_addr).parse()?,
        })
        .build()
        .request_url("GET", &url)
        .set("Host", &ip_addr)
        .call()?
        .into_string()?;
    println!("Response:\n{}", resp);
    Ok(())
}

mod fixed_resolver {
    //! A DNS resolver that always returns the same value regardless of
    //! the host it was queried for.

    use std::net::SocketAddr;

    pub struct FixedResolver {
        pub ip_addr: SocketAddr,
    }

    impl ureq::Resolver for FixedResolver {
        fn resolve(&self, _netloc: &str) -> std::io::Result<Vec<SocketAddr>> {
            Ok(vec![self.ip_addr])
        }
    }
}

mod certificate_verifier_no_hostname {
    use rustls::{
        Certificate, OwnedTrustAnchor, RootCertStore, ServerCertVerified, ServerCertVerifier,
        TLSError,
    };
    use std::time::SystemTime;
    use webpki::{DNSNameRef, EndEntityCert, SignatureAlgorithm, TrustAnchor};

    pub struct CertificateVerifierNoHostname;

    static SUPPORTED_SIG_ALGS: &[&SignatureAlgorithm] = &[
        &webpki::ECDSA_P256_SHA256,
        &webpki::ECDSA_P256_SHA384,
        &webpki::ECDSA_P384_SHA256,
        &webpki::ECDSA_P384_SHA384,
        &webpki::ED25519,
        &webpki::RSA_PSS_2048_8192_SHA256_LEGACY_KEY,
        &webpki::RSA_PSS_2048_8192_SHA384_LEGACY_KEY,
        &webpki::RSA_PSS_2048_8192_SHA512_LEGACY_KEY,
        &webpki::RSA_PKCS1_2048_8192_SHA256,
        &webpki::RSA_PKCS1_2048_8192_SHA384,
        &webpki::RSA_PKCS1_2048_8192_SHA512,
        &webpki::RSA_PKCS1_3072_8192_SHA384,
    ];

    impl ServerCertVerifier for CertificateVerifierNoHostname {
        /// Will verify the certificate is valid in the following ways:
        /// - Signed by a valid root
        /// - Not Expired
        ///
        /// Based on a https://github.com/ctz/rustls/issues/578#issuecomment-816712636
        fn verify_server_cert(
            &self,
            roots: &RootCertStore,
            intermediates: &[Certificate],
            _dns_name: DNSNameRef<'_>,
            ocsp_response: &[u8],
        ) -> Result<ServerCertVerified, TLSError> {
            // Get the end-entity cert, tthe chain, and the trust root. Error out if
            // chain is empty.
            let (cert, chain, trustroots) = prepare(roots, intermediates)?;

            // Validate the certificate is valid, signed by a trusted root, and not
            // expired.
            let now = SystemTime::now();
            let webpki_now =
                webpki::Time::try_from(now).map_err(|_| TLSError::FailedToGetCurrentTime)?;

            let _cert: EndEntityCert = cert
                .verify_is_valid_tls_server_cert(
                    SUPPORTED_SIG_ALGS,
                    &webpki::TLSServerTrustAnchors(&trustroots),
                    &chain,
                    webpki_now,
                )
                .map_err(TLSError::WebPKIError)
                .map(|_| cert)?;

            if !ocsp_response.is_empty() {
                //trace!("Unvalidated OCSP response: {:?}", ocsp_response.to_vec());
            }
            Ok(ServerCertVerified::assertion())
        }
    }

    #[allow(clippy::type_complexity)]
    fn prepare<'a, 'b>(
        roots: &'b RootCertStore,
        presented_certs: &'a [Certificate],
    ) -> Result<(EndEntityCert<'a>, Vec<&'a [u8]>, Vec<TrustAnchor<'b>>), TLSError> {
        if presented_certs.is_empty() {
            return Err(TLSError::NoCertificatesPresented);
        }

        // EE cert must appear first.
        let cert =
            webpki::EndEntityCert::from(&presented_certs[0].0).map_err(TLSError::WebPKIError)?;

        let chain: Vec<&'a [u8]> = presented_certs
            .iter()
            .skip(1)
            .map(|cert| cert.0.as_ref())
            .collect();

        let trustroots: Vec<webpki::TrustAnchor> = roots
            .roots
            .iter()
            .map(OwnedTrustAnchor::to_trust_anchor)
            .collect();

        Ok((cert, chain, trustroots))
    }
}

The comments in the source explain what does what. More importantly, the code has several limitations:

  • The custom certificate verifier is scary because it will accept any valid certificate, regardless of who it was issued for. In practice, this might be fine if we’re using our own certificates. In that case, we can configure our CA as the only root CA (add_server_trust_anchors), and then an adversary would have to present one of our own certificates in order to spoof our server. However, that shouldn’t be possible unless we’ve already been compromised.
  • The certificate verifier has a hardcoded list of ciphers, which might get outdated.
  • The code doesn’t handle redirects at all.
  • The ureq::Agent struct can’t be reused on different hosts because of the dummy resolver.

To conclude, connecting to a host specified by its IP address over HTTPS doesn’t work in ureq at the moment, but with a bit of misdirection and a hundred lines of code, we can work around it.