Container unesponsive

Sorry - I know this error has been posted many times. I have reviewed them all and can’t seem to find a solution.
I have a new installation of ShinyProxy on Ubuntu 20.04.1 LTS
applicationYML

`Error

Status code: 500

Message: Container did not respond in time

Stack Trace:
eu.openanalytics.containerproxy.ContainerProxyException: Container did not respond in time
at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.startProxy(AbstractContainerBackend.java:143)
at eu.openanalytics.containerproxy.service.ProxyService.startProxy(ProxyService.java:226)
at eu.openanalytics.shinyproxy.controllers.AppController.getOrStart(AppController.java:106)
at eu.openanalytics.shinyproxy.controllers.AppController.startApp(AppController.java:63)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:878)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:792)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:117)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)
at eu.openanalytics.containerproxy.util.ProxyMappingManager$ProxyPathHandler.handleRequest(ProxyMappingManager.java:160)
at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132)
at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)
at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)
at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)
at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)
at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:370)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:836)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2019)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1558)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1449)
at java.base/java.lang.Thread.run(Thread.java:829)`

Trying different things… I tried one sample with contain-network: and one without.
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
c9c3891fdd75 bridge bridge local
77873acb6686 host host local
a9bad54ec5a4 none null local
074bdc8c953c sp-example-net bridge local

sudo docker run -p 3838:3838 openanalytics/shinyproxy-demo R -e ‘shinyproxy::run_01_hello()’
Does not display an errors – comes up with listening on http://0.0.0.0:3838

I have tried different examples in the application.yml, but no success.

This is indeed one of the most frustrating errors because it can have many causes and the error message is mostly useless. I would suggest to start with cleaning up your application.yml file by removing all unneeded/unused options:

  • You’re not using https with docker so remove cert-path and container-protocol
  • Also try replacing localhost by 127.0.0.1

Hope this helps.

Thank you. I made those changes.

2021-08-13 20:56:21.514 WARN 203168 — [ XNIO-1 task-5] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (9/10): http://f41faadd386d:3838
2021-08-13 20:56:23.515 WARN 203168 — [ XNIO-1 task-5] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (10/10): http://f41faadd386d:3838
2021-08-13 20:56:25.732 ERROR 203168 — [ XNIO-1 task-5] io.undertow.request : UT005023: Exception handling request to /app/01_hello

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is eu.openanalytics.containerproxy.ContainerProxyException: Container did not respond in time
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) ~[spring-webmvc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517) ~[jakarta.servlet-api-4.0.4.jar!/:4.0.4]

Same error on both apps.

Where should my shiny-proxy-2.5.0.jar and application.yml files be located? I created a /shinyproxy folder and (for now) changed to rwxrwxrwx.

Hi Jay,

first to answer to direct questions.

Config file application.yml should be in /etc/shinyproxy, and .jar file in /opt/shinyproxy.

As you are mentioning that you have fresh installation on Ubuntu 20.04 I would first check how docker deamon is listening. On that version default should be on socket only (on that ubuntu version docker is version 20, so it is or ubuntu 20.04 behavior or docker 20 behavior).

Check it with:
ps -aux | grep dockerd
and if you do not see something like
root 780 0.0 3.0 1160296 123304 ? Ssl Aug09 4:15 /usr/bin/dockerd -H unix:// -D -H tcp://127.0.0.1:2375
than try to fix it with this manual:

Also, take deeper look at shinyproxy log file. It should be written if docker is inaccesible over tcp.

Thank you.
admin@shinyproxy2:/usr$ ps aux | grep dockerd
root 903 0.3 0.6 1187288 82524 ? Ssl 18:50 0:07 dockerd --group docker --exec-root=/run/snap.docker --data-root=/var/snap/docker/common/var-lib-docker --pidfile=/run/snap.docker/docker.pid --config-file=/var/snap/docker/796/config/daemon.json
root 2143 3.9 0.6 1384244 78032 ? Ssl 19:24 0:00 /usr/bin/dockerd -H unix:// -H tcp://127.0.0.1:2375
uadmin 2312 0.0 0.0 5192 724 pts/0 S+ 19:24 0:00 grep --color=auto dockerd

I am not getting a shinyproxy.log file created. I have that specified in my application.yml.

This says to put application.yml in the same folder as the jar file ( Configuration | ShinyProxy). Not sure where it creates the log file. Searched the entire system and don’t see it.

Jay,

dockerd looks good, but you have installed docker via snap and I do not have any experience with it (maybe does not make any change in this setup but I am not sure).

About log file, put absolute path eg. /var/log/shinyproxy.log but first create that file (eg. touch /var/log/shinyproxy.log) and check file write rights for shinyproxy user.

1 Like

Thank you for your help. I reinstalled Docker so it not via snap.

uadmin@shinyproxy2:/etc/shinyproxy$ ps aux | grep dockerd
root 7440 0.0 0.6 1679172 84340 ? Ssl 15:57 0:02 /usr/bin/dockerd -H unix:// -H tcp://127.0.0.1:2375
uadmin 9711 0.0 0.0 5192 672 pts/0 S+ 17:48 0:00 grep --color=auto dockerd
uadmin@shinyproxy2:/etc/shinyproxy$

thanks for the tip on the log file. I am getting a log file now. Would a log or debug log help diagnose the problem? I am not seeing any errors that are helpful.

2021-08-17 17:45:45.440  INFO 9505 --- [XNIO-1 task-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'

2021-08-17 17:45:45.445 INFO 9505 — [XNIO-1 task-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 5 ms
2021-08-17 17:45:49.785 INFO 9505 — [XNIO-1 task-3] e.o.containerproxy.service.UserService : User logged in [user: Test]
2021-08-17 17:45:55.555 INFO 9505 — [XNIO-1 task-2] c.s.docker.client.DefaultDockerClient : Starting container with Id: d1d3a414f3d196c43e61bef06bdc5ab1cdaca71bbdbc9f6fdda04eee32ea9411
2021-08-17 17:45:57.988 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (2/10): http://d1d3a414f3d1:3838
2021-08-17 17:45:59.990 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (3/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:01.991 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (4/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:03.993 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (5/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:05.994 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (6/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:07.996 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (7/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:09.998 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (8/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:12.000 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (9/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:14.001 WARN 9505 — [XNIO-1 task-2] e.o.shinyproxy.ShinyProxyTestStrategy : Container unresponsive, trying again (10/10): http://d1d3a414f3d1:3838
2021-08-17 17:46:16.229 ERROR 9505 — [XNIO-1 task-2] io.undertow.request : UT005023: Exception handling request to /app/01_hello

org.springframework.web.util.NestedServletException: Request processing failed; nested exception is eu.openanalytics.containerproxy.ContainerProxyException: Container did not respond in time
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909) ~[spring-webmvc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:517) ~[jakarta.servlet-api-4.0.4.jar!/:4.0.4]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584) ~[jakarta.servlet-api-4.0.4.jar!/:4.0.4]
at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) ~[undertow-servlet-2.1.4.Final.jar!/:2.1.4.Final]

Well,

it seems to me that in application.yml in docker section internal-networking is making mess. Try to remove it. Also, in same section container-protocol is strange to me… try to remove it also.

2 Likes

It is working!!! Thank you!

I had to remove the internal-networking line from the yml.

2 Likes