ShinyProxy with nginx reverse-proxy SSL and client behind full-tunnel VPN

The double-proxy nature of the titular setup seems to break the server component of Shiny apps. If I’m not behind my full-tunnel VPN, then everything works fine - SSL and all - or, if I skip the reverse proxy and go directly to http://host.com:8080, then everything works fine, though of course now I’m unencrypted. But, once I’m on the VPN, everything works fine up until I spin up any given app, at which point just the UI loads, and any server-generated objects - plots, tables, etc. - fail to load. No error message or anything is generated - just white space where server output should be. The container also dies (the screen goes gray and unresponsive) after only 20 seconds or so. The UI objects appear responsive, i.e. I can slide a slider and it stays where I put it, even though there’s no server-level response.

I know that this is almost certainly due to a problem with the reverse-proxy headers, but I’ve tried everything I could find on that front. Here’s my current nginx config:

location /
  proxy_pass          http://localhost:8080;
  proxy_redirect      off;

  proxy_http_version  1.1;
  proxy_set_header    Upgrade $http_upgrade;
  proxy_set_header    Connection $connection_upgrade;
  proxy_read_timeout  600s;
  proxy_buffering     off;

  proxy_set_header    Host               $host;
  proxy_set_header    X-Real-IP          $remote_addr;
  proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
  proxy_set_header    X-Forwarded-Proto  $scheme;
}

I’ve already experimented with pretty much everything here, including using the new Forwarded field as documented here, even though I don’t think ShinyProxy uses it at all. I’m also using the $connection_upgrade setup as documented here, though it doesn’t seem to make any difference as opposed to just using the “upgrade” string for the Connection header. That value, by the way, as defined in the nginx http block, is:

map $http_upgrade $connection_upgrade {
  default upgrade;
  ''      close;
}

The basic question, of course, is if there’s something about my VPN and its configuration that’s making it impossible for the Shiny App and the client to fully communicate behind both the VPN proxy and my server’s reverse proxy. Again, there’s no issue if I just take the reverse proxy out of the picture, and set nginx up to allow direct access to ShinyProxy on port 8080 - and I’d be perfectly fine with that in operation, actually, if I could just make that connection SSL-secure.

Should also specify: I’m running Docker-containerized Shiny apps, with no SSL or anything between Docker and ShinyProxy (it’s a loopback interface), and I’m using an internal LDAP server for authentication, again on the loopback interface. I activated root DEBUG logging and replicated the error, but I don’t see anything relevant to the apparent failure of the Shiny server to get results back to the client (the actual R Shiny logs are clean as a whistle, just says it started up and was listening).

Can you see any http status codes when you try and load the application and look at something like Chrome’s developer tab with the console and networking. I had similar issues, and it’s helpful to know the status codes.

Also, what do your nginx logs say? Perhaps something along the chain is blocking a request, for instance the upgrade to a websocket connection that shiny requires.

Finally, in addition to the custom http block to map the upgrade logic, did you ensure you’ve updated your httpuv package?

1 Like

Thanks for helping. I finally figured out how to log those http status codes everyone’s always talkin about (thanks), and everything is 200 except for the websocket status 101. It says “pending,” but it says that whether or not the app is working, and my Googling suggests that it’s supposed to be always pending.

My nginx logs are clean, as well. Nothing in the error log, and nothing weird in the access log. The docker container logs are clean as well (they just say that the Shiny app is listening), and shinyproxy.log is clean as well, at the standard level of logging detail - it just indicates the authenticating in, the starting, and the stopping.

So that leaves httpuv. I stripped my app down to just a starting R image (I tried openanalytics/r-base and rocker/shiny-verse), some linux app installs/updates, installing Shiny and httpuv, and running shiny::runExample('hello_01', port=3838, host='0.0.0.0'). I’m still getting this behavior where it works on direct port 8080 but not through an SSL reverse-proxy (when the client is also on this VPN).

My dockerfile:

FROM openanalytics/r-base

RUN apt-get update && apt-get install -y \
    sudo \
    libcurl4-gnutls-dev \
    libcairo2-dev \
    libxt-dev \
    libssl-dev \
    libssh2-1-dev

RUN R -e "install.packages(c('shiny', 'httpuv'))"

EXPOSE 3838

CMD ["R", "-e", "shiny::runExample('01_hello',port=3838,host='0.0.0.0')"]

Relevant application.yml:

  specs:
  - id: debug_app
    display-name: Debug App
    container-image: debug_app
    access-groups: [admin]

Here is a nginx conf section that I’ve been using that works for a (chain of) reverse proxies that are in front of the shinyproxy component, for your reference. It is quite similar to yours but this is using $http_host instead of $host and attaches a trailing slash to the proxy_pass destination - not sure if any of that would make a difference and this setup might be different with regards to not using VPNs (which it doesn’t have any issues with though) and also the app is a “prefix app” and not in the root context:

	location /public/ {
		proxy_pass http://app:3838/;
		proxy_http_version                    1.1;
		proxy_set_header Upgrade              $http_upgrade;
		proxy_set_header Connection           "upgrade";
		proxy_read_timeout                    600s;
		proxy_redirect                        off;
		proxy_set_header Host                 $http_host;
		proxy_set_header X-Real-IP            $remote_addr;
		proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Protocol $scheme;
	}

For forwarding headers, do you already have this statement in application.yml? If not, does it make a difference if you use this setting?

server:
  useForwardHeaders: true

I do indeed have useForwardHeaders on, and I tried using $http_host instead of just $host and I also tried "upgrade" instead of the elsewhere-defined $connection_upgrade, but still no luck - the same behavior where the app starts up but server objects fail to load, only when behind this VPN and using a reverse proxy.

One thing that is interesting is the behavior with that location path. If I change it to /public/ and restart both nginx and shinyproxy, it breaks - it actually redirects to the /login/ path of my URL, and then 404’s out.

I’m also wondering about your proxy_pass path. 3838 is Shiny Server’s port, not ShinyProxy. I’m trying to direct this location to the ShinyProxy login, landing page, and apps, all with SSL security. Is that what you’re doing?

Responding to the question about the /public/ section, I was using this for having a both a non-authenticated instance of an app and also an authenticated variant available (using shinyproxy), but through another path, here is that nginx conf section:

	location / {
		proxy_pass http://shinyproxy:8080;
		proxy_http_version                    1.1;
		proxy_set_header Upgrade              $http_upgrade;
		proxy_set_header Connection           "upgrade";
		proxy_read_timeout                    600s;
		proxy_redirect                        off;
		proxy_set_header Host                 $http_host;
		proxy_set_header X-Real-IP            $remote_addr;
		proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Protocol $scheme;
	}

Outside of the proxy which has this config I have another proxy (jwilder/nginx-proxy:alpine) which deals with the certificates. So in some ways it is similar (I think :)) but I have a proxy chain with two nginx:es infront of the app. I’m not sure how you use the VPN in the setup, if you don’t go through it but access the reverse proxy, then everything works? If so, it sounds like the VPN might interfere with something? Not sure how to troubleshoot that, maybe wireshark to try to confirm at what stage mangling happens (headers being stripped or something like that)?