Problem with nginx in combination with development shiny version

[derborisk]

Hey all,

I’m facing a strange behaviour, which I can’t really classify.
I’m using shinyproxy in a dockerd environment with a nginx proxy for adding tls decryption (ubuntu server). In principle this setting is working fine for deploying shiny-apps.

My problem occurs when building the shiny app with latest development builds directly from GitHub instead of the stable CRAN version.

When building shiny by devtools from github the app is not running appropriate, because the browser is not able to load any js or css code.

Example from debugging console of the browser:

GET https://shiny.example.com/endpoint/8ea69e1c-3d20-4660-95ee-c433bab8a569/shared/strftime/strftime-min.js net::ERR_ABORTED 404 (Not Found)

The strange thing is, that the same app (build with github version) is working fine when you connect shinyproxy in raw http-mode on the bonded port, in my case http://shiny.example.net:8082

Can anybody reproduce that behaviour and classify the problem if it caused by the shiny-package, by shinyproxy or maybe by any other settings?

Here a short example.

I used “006-tabsets”-example in connection with the building instructions from the shinyproxy-template.

Dockerfile for the working container:

FROM openanalytics/r-base


# system libraries of general use
RUN apt-get update && apt-get install -y \
    sudo \
    pandoc \
    pandoc-citeproc \
    libcurl4-gnutls-dev \
    libcairo2-dev \
    libxt-dev \
    libssl-dev \
    libssh2-1-dev \
    libssl1.0.0

# system library dependency for the euler app
RUN apt-get update && apt-get install -y \
    libmpfr-dev

# basic shiny functionality
RUN R -e "install.packages('shiny', repos='https://cloud.r-project.org/')"

#RUN R -e "install.packages('devtools', repos='https://cloud.r-project.org/')"
#RUN R -e "devtools::install_github('rstudio/shiny')"

# copy the app to the image
RUN mkdir /root/tabsets
COPY tabsets /root/tabsets

COPY Rprofile.site /usr/lib/R/etc/

EXPOSE 3838

CMD ["R", "-e", "shiny::runApp('/root/tabsets')"]

Dockerfile for the not working container

FROM openanalytics/r-base


# system libraries of general use
RUN apt-get update && apt-get install -y \
    sudo \
    pandoc \
    pandoc-citeproc \
    libcurl4-gnutls-dev \
    libcairo2-dev \
    libxt-dev \
    libssl-dev \
    libssh2-1-dev \
    libssl1.0.0

# system library dependency for the euler app
RUN apt-get update && apt-get install -y \
    libmpfr-dev

# basic shiny functionality
#RUN R -e "install.packages('shiny', repos='https://cloud.r-project.org/')"

RUN R -e "install.packages('devtools', repos='https://cloud.r-project.org/')"
RUN R -e "devtools::install_github('rstudio/shiny')"

# copy the app to the image
RUN mkdir /root/tabsets
COPY tabsets /root/tabsets

COPY Rprofile.site /usr/lib/R/etc/

EXPOSE 3838

CMD ["R", "-e", "shiny::runApp('/root/tabsets')"]

application.yml:

proxy:
  title: ShinyProxy
  #hide-navbar: true
  port: 8080
  authentication: simple
  admin-groups: admins
  users:
  - name: jack
    password: password
    groups: admins
  docker:
    internal-networking: true
  specs:
  - id: tabsets_cran
    display-name: working version
    container-cmd: ["R", "-e", "shiny::runApp('/root/tabsets')"]
    container-image: tabsets_cran
    container-network: shiny-gate
  - id: 06_tabsets_github
    display-name: not working version
    container-cmd: ["R", "-e", "shiny::runApp('/root/tabsets')"]
    container-image: tabsets_github
    container-network: shiny-gate
logging:
  file:
    shinyproxy.log

[shrektan]

Your Nginx configuration may be incorrect. Have you noticed this page https://www.shinyproxy.io/security/ ? There’s a detailed example configuration for your Nginx .

[derborisk]

Thx for your answer. Yes i noticed that page and i think the nginx-configuration is not the problem because everything works fine if i build the app-image with:

RUN R -e “install.packages(‘shiny’, repos=‘https://cloud.r-project.org/’)”

Exactly the same app is not working anymore if i build the app-image with:

RUN R -e “install.packages(‘devtools’, repos=‘https://cloud.r-project.org/’)”
RUN R -e “devtools::install_github(‘rstudio/shiny’)”

I also tried the useForwardHeaders option in the application.yml, is does not fix the problem.

server:
useForwardHeaders: true

[shrektan]

I’m confused.

Why do you need to install packages in the apps? From my understanding, you are saying the packages fails to be installed in the Docker building stage when using devtools::install_github() rather than install.packages(). Am I correct? So it has nothing to do with ShinyProxy?

If you encounter problems when building the Docker image, it’s not related to the Nginx I believe… Instead, it’s because your computer is behind a proxy. You may need to pass “–build-args HTTP_PROXY=xxxx --build-args HTTPS_PROXY=xxxx” when you run docker build.

[derborisk]

Hey thanks again for your input, but can you explain the build arguments in more detail? I never used or needed it before.

But maybe I haven’t explained my problem clearly enough. As I mentioned before I’m not able to classify on which exact point the issue is located. I can just say it is linked to the chain “shiny – docker image – shinyproxy – nginx”.

There is no problem in building the docker images and I don’t install any packages in the app itself.

In both cases (“install.packages(‘shiny’)” & “devtools::install_github(‘rstudio/shiny’)” ) I get working images, that can be started in standalone modus. Both images are also running without issues via shinyproxy (self containerized), if I connect shinyproxy directly over the exposed port. The problem just appears when trying to start the “devtools::install_github()”-image via nginx and shinyproxy. The browser is just able to receive the raw html but no js oder css in addition, because of 404 error. Maybe there is a websocket-issue, but my skill level is not high enough to locate the source of the problem. Maybe somebody i able to reconstruct the issue.

In contrast the other “install.packages()”-container is running fine in exact that constellation.

That’s why I thought there could be a problem in the interaction of nginx, the proxy used in shinyproxy and the “devtools::install_github()”-shiny version.

As next step I will try to connect the started app-image in standalone modus via nginx to see if the problem can be located in the shiny-version.

In addition, here are my nginx-settings maybe there is something wrong?

server {
listen 80;
listen [::]:80;

    server_name shiny.example.com;

    include /etc/nginx/snippets/letsencrypt.conf;

    location / {
            #try_files $uri $uri/ =404;
            return 302 https://$server_name$request_uri;
    }

}

server {
listen 443 ssl;
listen [::]:443 ssl;

#CERTS
include snippets/letsencrypt_certs.conf;

ssl_protocols TLSv1.2;

# set max upload size
client_max_body_size 512M;
fastcgi_buffers 64 4K;

server_name shiny.example.com;

location / {
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_read_timeout 90s;

    # Connect to local port
    proxy_pass http://127.0.0.1:8082/;
}

}

[derborisk]

Hi,
I just tested to connect the both container directly via nginx to the bonded port and you are right my first assumption that the issue is linked to shinyproxy was not right because the same issue accurs without shinyproxy in the middle.

So there must be a problem in the github shiny version, the way I build the docker image or in my nginx settings.

The problem is I don’t get it

So once again the problem:

The target is to create an docker-image with a working shiny-server suppling an webApp (later in combination with shinyproxy). The docker image should ran onto an ubuntu server with a nginx-server in front of the app to secure the connection via tls.

Nginx-settings:

server {
listen 80;
listen [::]:80;

server_name shiny.example.com;
include /etc/nginx/snippets/letsencrypt.conf;

location / {
        #try_files $uri $uri/ =404;
        return 302 https://$server_name$request_uri;
}

}

server {
listen 443 ssl;
listen [::]:443 ssl;

#CERTS
include snippets/letsencrypt_certs.conf;
ssl_protocols TLSv1.2;
#set max upload size
client_max_body_size 512M;
fastcgi_buffers 64 4K;
server_name shiny.example.com;

location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection “upgrade”;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 90s;

# Connect to local port
proxy_pass http://127.0.0.1:8082/;

}
}

In this case I tried to build the 006-tabset example.

I copied the app.R into homefolder/tabsets/app.r and the following Dockerfile & Rprofile.site into my homefolder.

Rprofile.site

local({
options(shiny.port = 3838, shiny.host = “0.0.0.0”)
})

Dockerfile:

FROM openanalytics/r-base

#system libraries of general use
RUN apt-get update && apt-get install -y
sudo
pandoc
pandoc-citeproc
libcurl4-gnutls-dev
libcairo2-dev
libxt-dev
libssl-dev
libssh2-1-dev
libssl1.0.0

#system library dependency for the euler app
RUN apt-get update && apt-get install -y
libmpfr-dev

#basic shiny functionality
#RUN R -e “install.packages(‘devtools’, repos=‘https://cloud.r-project.org/’)”

RUN R -e “install.packages(‘shiny’, repos=‘https://cloud.r-project.org/’)”
#RUN R -e “devtools::install_github(‘rstudio/shiny’)”

#copy the app to the image
RUN mkdir /root/tabsets
COPY tabsets /root/tabsets

COPY Rprofile.site /usr/lib/R/etc/

EXPOSE 3838

CMD [“R”, “-e”, “shiny::runApp(’/root/tabsets’)”]

Then I created the first docker image “tabsets_cran”

docker build -t tabset_cran .

Then is changed shiny-source into the Dockerfile

RUN R -e “install.packages(‘shiny’, repos=‘https://cloud.r-project.org/’)”
#RUN R -e “devtools::install_github(‘rstudio/shiny’)”

into

#RUN R -e “install.packages(‘shiny’, repos=‘https://cloud.r-project.org/’)”
RUN R -e “devtools::install_github(‘rstudio/shiny’)”

and created the second docker image:

docker build -t tabset_github .

Starting the “tabset_cran” :

docker run --rm -p 8082:3838 tabset_cran

Everything is working fine via “http(s)://shiny.example.com” !

Now stopping the container and running the other one

docker run --rm -p 8082:3838 tabset_github

Connecting via “http(s)://shiny.example.com” is not working. There server sends only raw html and any js and css can’t be loaded because of error 404.

Here some example from the browser console:

shiny.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)

ion.rangeSlider.skinShiny.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)

bootstrap.min.css:1 Failed to load resource: the server responded with a status of 404 (Not Found)

That is really strange because I’m able to connect the same container directly via http://server-ip:8082 or http://shiny.example.com:8082 without the 404 errors.

What could be the problem? What am I missing?
I am happy about any hint.

[shrektan]

I’m able to connect the same container directly via http://server-ip:8082 or http://shiny.example.com:8082 without the 404 errors.

It suggests that the docker image is good. I believe the devil is hidden inside of the nginx configurations.

I notice that you are using https and the traffic on port 80 should be forwarded to port 443. However, I’m not Nginx expert either. Anyway, I think you may try to modify the forwarding part (server {listen 80; ...}) to something like this (Copied from https://www.shinyproxy.io/security/ ). :

server {
  listen                80;
  server_name           shinyproxy.yourdomain.com;
  rewrite     ^(.*)     https://$server_name$1 permanent;
}

[mnazarov]

I think it would also be useful to understand which exact change in the dev shiny code causes the issue. This may help to find a solution. Quickly looking at recent commits, this one - https://github.com/rstudio/shiny/pull/2280 - may be a culprit, from the related NEWS update:

Shiny now serves static files on a background thread. This means that things like JavaScript and CSS assets can be served without blocking or being blocked by the main R thread, and should result in significantly better performance for heavily loaded servers.

If you could test a version just before this commit, i.e.

RUN R -e "install.packages('remotes', repos='https://cloud.r-project.org/')"  # should be faster than installing devtools
RUN R -e "remotes::install_github('rstudio/shiny', ref = 'b606ba4dd7d75cfb96d346d7057a9a7dedeea9c8')"

and then try to understand what this change does under the hood that might not work well with nginx., you might get closer to solving the mystery…

[derborisk]

The problem and a solution is described here:

Shiny v1.3 known regressions and serious issues

[shrektan]

A quick answer is to use dev version of shiny and httpuv at the same time.