Container-logs to s3

ShinyProxy
#1

I have multiple apps running in shinyproxy on an EC2instance. I am using an IAM role to give my apps access to other AWS resources (e.g. s3, dynamodb, etc.) with no problems. However, I would like to write my container logs to a s3 bucket but can not seem to get it working. Following the shinyproxy configuration docs I have added the following to my application.yml file:

container-log-path: s3://my-bucket/shinyproxy-logs
container-log-s3-access-key: mykey
container-log-s3-access-secret: my_secret
container-log-s3-endpoint: https://s3-us-east-1.amazonaws.com

For good measure I have also give access to the bucket to the host EC2 instance. But, while shinyproxy is working fine with these settings, nothing shows up in the s3 bucket. If I remove those settings and replace them with:

container-log-path: /etc/shinyproxy/container-logs

the logs do show up there. Am I missing something to get them to write to the s3 bucket?

#2

I just wanted to bump this question again. Has anyone on here successfully set up per-container logging to s3? The documentation on how to do it is very clear and simple, but it does not work for me.

#3

Here is my successful configuration:

proxy: # acording documentation: https://www.shinyproxy.io/documentation/configuration/#local-file-back-end
  container-log-path: s3://my-bucket/my-prefix
  container-log-s3-sse: true # !!important: S3 by default uses SSE - https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html
  container-log-s3-endpoint: https://s3.us-east-1.amazonaws.com  #S3 endpoints to all regions is here: https://docs.aws.amazon.com/general/latest/gr/s3.html
  container-log-s3-access-key: AKIAMYAWSIAMUSERTOS3
  container-log-s3-access-secret: 1TdISc8rzjE0YPTiSSpKzTu8UiGpKL88DwByQWNU

:point_right: It seems your s3 endpoint is invalid.

The documents on AWS revealed two important pieces of information:
1 - S3 endpoints: https://docs.aws.amazon.com/general/latest/gr/s3.html
2 - S3 SSE: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html

I hope this helps you.

1 Like