Rstudio Server Redirect to /auth-sign-in

I am using ShinyProxy with the Rocker RStudio server and disabling authentication by setting the DISABLE_AUTH=true environment variable. This is working for several versions, but with version rocker/rstudio:4.0.2 which is RStudio Server version 1.3.1093 I am getting the following"

Mixed Content: The page at ‘https://my-host.com/app/my_app’ was loaded over HTTPS, but requested an insecure frame ‘http://my-host.com/auth-sign-in’. This request has been blocked; the content must be served over HTTPS.

It appears that despite authentication being disabled, it is still redirecting to /auth-sign-in. When I run the image locally I do not see that behavior. Authentication is disabled and it loads as expected. Any ideas?