code to gracefully stop provider-server and any dependent goroutines it started

[rewantsoni]

could you pls create a new issue to stop the server and also this cache? I see we don't have any stopping procedure and let's fix at the earliest before they become problematic?

Originally posted by @leelavg in #3466 (comment)