[HMA] Add explicit connection close for signal ops and clear temp refs

[juanmrad]

Summary

Attempts to fix memory leaks in the index lifecycle that cause worker RSS to climb steadily with each index refresh (see #1813).

Close raw_conn in load_signal_index() and commit_signal_index() previously these database connections were never returned to the pool, leaking on every refresh cycle.

Clear built_index reference and signal_list before calling trim_process_memory() so gc.collect() can actually reclaim them without waiting for python to clear reference.

[Dcallies]

Have we figured out a way to collect evidence on what the source of memory is for longer running processes? At least internally we have some tools that allow us to inspect the memory usage.

There are other things that we could do to flail at the problem such as rewriting index build to try and make it incremental, and only rebuild once the delta starts to get too high. In practice I think this reduces the need to rebuild the index at all to zero for most workloads.

[Dcallies]

I'm very suspicious this does anything, because it requires that the local variable is not properly garbage collected when it leaves scope, which stretches my understanding about how refcounting works.

Do we have any evidence that this is needed?

Random googling:

1. https://stackoverflow.com/questions/28324084/does-deleting-reassigning-a-python-list-allow-for-garbage-collection
2. https://blog.codingconfessions.com/p/cpython-garbage-collection-internals

[Dcallies]

This might work too: https://docs.python.org/3/library/contextlib.html#contextlib.closing which lets you use a with statement.

Reading https://docs.sqlalchemy.org/en/21/core/connections.html#working-with-the-dbapi-cursor-directly I think this is very likely missing the close