Today we upgraded the docker packages on our server that also runs some shinyproxies. After these updates shinyproxy was no longer able to spin up the app-containers and failed with
Failed to start app <<app name>>
After rolling back to the previous versions shinyproxy worked as normal, again.
These were the packages that we did upgrade but I don’t know exactly which one was the offending one.
In parantheses the version numbers are given in that order (working / not working):
docker-ce-rootless-extras:amd64 (5:24.0.7-1~ubuntu.20.04~focal / 5:25.0.0-1~ubuntu.20.04~focal)
docker-buildx-plugin:amd64 (0.11.2-1~ubuntu.20.04~focal / 0.12.1-1~ubuntu.20.04~focal)
docker-compose-plugin:amd64 (2.21.0-1~ubuntu.20.04~focal / 2.24.1-1~ubuntu.20.04~focal)
docker-ce:amd64 (5:24.0.7-1~ubuntu.20.04~focal / 5:25.0.0-1~ubuntu.20.04~focal)
docker-ce-cli:amd64 (5:24.0.7-1~ubuntu.20.04~focal / 5:25.0.0-1~ubuntu.20.04~focal)
The server is running Ubuntu 20.04.6 LTS.
Any hints why this is happening and any help is much appreciated.
Here is a shortened example log:
2024-01-23 09:06:57.476 INFO 1 --- [pool-1-thread-3] e.o.containerproxy.service.ProxyService : [user=xxx proxyId=yyy specId=zzz] Starting proxy
2024-01-23 09:06:57.488 WARN 1 --- [pool-1-thread-3] e.o.containerproxy.service.ProxyService : [user=xxx proxyId=yyy specId=zzz] Proxy failed to start
eu.openanalytics.containerproxy.ProxyFailedToStartException: Container with index 0 failed to start
at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.startProxy(AbstractContainerBackend.java:116) ~[containerproxy-1.0.2.jar!/:1.0.2]
at eu.openanalytics.containerproxy.service.ProxyService.startOrResumeProxy(ProxyService.java:455) [containerproxy-1.0.2.jar!/:1.0.2]
at eu.openanalytics.containerproxy.service.ProxyService.lambda$startProxy$4(ProxyService.java:279) [containerproxy-1.0.2.jar!/:1.0.2]
at eu.openanalytics.containerproxy.service.ProxyService.lambda$action$11(ProxyService.java:599) [containerproxy-1.0.2.jar!/:1.0.2]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_392]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_392]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_392]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_392]
at java.lang.Thread.run(Thread.java:750) ~[na:1.8.0_392]
Caused by: eu.openanalytics.containerproxy.ContainerFailedToStartException: Docker container failed to start
at eu.openanalytics.containerproxy.backend.docker.DockerEngineBackend.startContainer(DockerEngineBackend.java:153) ~[containerproxy-1.0.2.jar!/:1.0.2]
at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.startProxy(AbstractContainerBackend.java:108) ~[containerproxy-1.0.2.jar!/:1.0.2]
... 8 common frames omitted
Caused by: com.spotify.docker.client.exceptions.DockerRequestException: Request error: GET unix://localhost:80/images/myimage:mytag/json: 200
at com.spotify.docker.client.DefaultDockerClient.propagate(DefaultDockerClient.java:2840) ~[docker-client-8.16.0.jar!/:8.16.0]
at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2712) ~[docker-client-8.16.0.jar!/:8.16.0]
at com.spotify.docker.client.DefaultDockerClient.inspectImage(DefaultDockerClient.java:1509) ~[docker-client-8.16.0.jar!/:8.16.0]
at eu.openanalytics.containerproxy.backend.docker.DockerEngineBackend.isImagePresent(DockerEngineBackend.java:243) ~[containerproxy-1.0.2.jar!/:1.0.2]
at eu.openanalytics.containerproxy.backend.docker.DockerEngineBackend.startContainer(DockerEngineBackend.java:81) ~[containerproxy-1.0.2.jar!/:1.0.2]
... 9 common frames omitted
Caused by: javax.ws.rs.client.ResponseProcessingException: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `com.spotify.docker.client.messages.ImageInfo`, problem: Nul l virtualSize
at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 5203]
at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:884) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:98) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.JerseyInvocation$5.completed(JerseyInvocation.java:851) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:197) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.ClientRuntime.access$300(ClientRuntime.java:74) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.ClientRuntime$2$1$1.run(ClientRuntime.java:167) ~[jersey-client-2.26-b03.jar!/:na]
...
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.internal.Errors.process(Errors.java:267) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:303) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.ClientRuntime$3.run(ClientRuntime.java:209) ~[jersey-client-2.26-b03.jar!/:na]
... 5 common frames omitted
Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `com.spotify.docker.client.messages.ImageInfo`, problem: Null virtualSize
at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 5203]
at com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:2047) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:587) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:610) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:293) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromObjectWith(ValueInstantiator.java:288) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:519) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:2025) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1175) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:803) ~[jackson-jaxrs-base-2.13.4.jar!/:2.13.4]
at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:257) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:236) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:156) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1082) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:874) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:808) ~[jersey-common-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.ClientResponse.readEntity(ClientResponse.java:326) ~[jersey-client-2.26-b03.jar!/:na]
at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:881) ~[jersey-client-2.26-b03.jar!/:na]
... 33 common frames omitted
Caused by: java.lang.NullPointerException: Null virtualSize
at com.spotify.docker.client.messages.AutoValue_ImageInfo.<init>(AutoValue_ImageInfo.java:91) ~[docker-client-8.16.0.jar!/:8.16.0]
at com.spotify.docker.client.messages.ImageInfo.create(ImageInfo.java:97) ~[docker-client-8.16.0.jar!/:8.16.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_392]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_392]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_392]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_392]
at com.fasterxml.jackson.databind.introspect.AnnotatedMethod.call(AnnotatedMethod.java:104) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:291) ~[jackson-databind-2.13.4.2.jar!/:2.13.4.2]
... 51 common frames omitted