Shinyproxy in AWS EKS


#1

Hello,
Has anybody got ShinyProxy to work with EKS? The ShinyProxy I have won’t launch new containers and I just wanted to know if this is possible and if it is, can anybody share their app settings please?
Thanks!


#2

Hi Mark,

We have several different environments/deployments running on AWS EKS currently. Everything works quite well so far.

To what extent did you follow https://github.com/openanalytics/shinyproxy-config-examples/tree/master/03-containerized-kubernetes?

Please note that if you use containers that are stored in a private repository, you need to make sure to add an image-pull-secret to shinyproxy/application.yml. https://www.shinyproxy.io/configuration/ also contains extra information in the ‘Kubernetes’ paragraph.

Happy to have a glance at your yaml files.

Best,

Michael


#3

I thought I did copied it exactly. I have torn down my cluster and I am going to start again.
What is your apiGroup ? Is it meant to be rbac.authorization.k8s.io ?
Thanks
Mark


#4

I’ve followed the guide but this time I put built the docker container on each node and it is working. hurray!
I was using ECR before. I am going to try and use ECR again following your advice. I would have thought the nodes could have pulled from ECR using the node’s IAM role?


#5

We have been using DockerHub where I followed the Kubernetes documentation on secrets to create the proper secret (and then refer to it through ShinyProxy’s .yaml following the links I shared above). Not sure about ECR, but the documentation seems to agree with you https://docs.aws.amazon.com/eks/latest/userguide/managing-auth.html. Also note the ECR section here https://kubernetes.io/docs/concepts/containers/images/.

As a minor comment, I have created a similar ShinyProxy Kubernetes deployment on Google Cloud Platform through Kubernetes Engine, if you use their container registry you do not have to create a secret or something equivalent to an IAM role as long as your container registry and deployment are in the same project.

Best regards,

Michael


#6

Hi Michael!

Could you please tell a bit in more details how did you setup Shinyproxy through Kubernetes Engine?
I would like to try that too.

Best regards,

Anatoly


#7

Hi Anatoly,

The first step is to setup a cluster via the google cloud console using Kubernetes Engine. Once that cluster is up and running you can follow https://github.com/openanalytics/shinyproxy-config-examples/tree/master/03-containerized-kubernetes until you have a working demo setup based on those files. I can provide more detailed steps once I have a bit more time. Do you have prior experience with Kubernetes @Anatoliy_Vuets?

Best regards,

Michael


#8

Michael, thank you for your reply!
“Do you have prior experience with Kubernetes” - not much: I played with deploying containerized apps in Kubernetes Engine but have no experience with custom Kubernetes deployments.
The tutorial for containerized ShinyProxy seems to be for custom Kubernetes deployments and not adopted for cloud deployments with their infrastructure (or maybe I’m missing something) so I have questions here…
How can I use container registry here? Should my app (Shiny app) images sit there?
In Kubernetes engine you typically put your app container in image registry and make its deployment in your cluster. Also you can add nice features like auto-scaling and benefit from rolling update if you want to update your app without restarting everything. How does these work with ShinyProxy?
Also from the tutorial “Ensure the kube-proxy-sidecar image is available on all your kube nodes. E.g. by repeating the above steps on all nodes.” - what do I do if I decide to increase the number of nodes in my cluster? How does it work with auto-scaling?

Sorry if some questions are dumb, this is because I have no much experience.
I’m a bit lost here.

Best regards,
Anatoly


#9

Hi @Anatoliy_Vuets,

Your focus should initially be on shinyproxy-config-example/03-containerized-kubernetes/shinyproxy-example/application.yml. This yml file shows two applications 01_hello and 06_tabsets. I suggest you first leave this as it is and try to get ShinyProxy to work on Kubernetes. Once you have everything working, feel free to change the image names there to that of your own (this involves building the ShinyProxy image again and pushing it to your a container registry + modifying “shinyproxy-example” in spec.containers.name sp-deployment.yaml).

The deployment should be a few steps: start a Kubernetes cluster with Kubernetes Engine and then run kubectl create -f sp-deployment.yaml, sp-authorization.yaml and sp-service.yaml.

How can I use container registry here?

On GCS you can use Container Registry, if you use your own private images the sp-deployment.yaml and application.yml should use the path you see in the image below (eu.gcr.io/example/sp/…). I recommend you first leave the files on GitHub as they are and try to get ShinyProxy working on Kubernetes with the demo applications.

image

Should my app (Shiny app) images sit there?

You should have a ShinyProxy image in a registry as well as images (which are specified in your ShinyProxy application.yaml file) that contain your Shiny applications. If I remember correctly, using Kubernetes Engine and Container Registry within the same Google Cloud project allows you to not have to be bothered with secrets to specify registry credentials.

Autoscaling

I may come back on that once I have more experience with our autoscaling setup on AWS EKS, we have it setup but it has not been tested thoroughly.