This is not something specific to shinyproxy, and can be achieved by using a custom CSS in your shiny app.
You would need to add something like:
content: "Your session has timed out.";
save it as
styles.css and include in your