(Migrated) Good read on hot-reloading everything

(This message has been automatically imported from the retired mailing list)

A question that arises from time to time is if it’s possible to
hot-reload arbitrary Python modules, dependencies or classes such as
SQLAlchemy models.

While I would very much like for it to be the case the real deal-breaker
is updating references to code that has been already imported.

Graham Dumpleton nicely catches it in these recent articles in the
context of the wrapt module:

Another point is dealing with code of dependencies that acquires any
resources such as TCP connections of file locks.

What should happen exactly to already established connections and live
transactions if you import say, a PyMongo library, connect to the DB and
later on decide to up the dependency’s version?

In principle, seeing as this is all open-source, I don’t think this is
something that could not be overcome - it’s just mundane, time-consuming
and low-level stuff that needs quite some research and experimentation.

As for SQLAlchemy, yes, it will make sense to special case SA models
somehow so they can be uploaded without server restarts.

On 10/04/2015 10:01, Dariusz Suchojad wrote:

A question that arises from time to time is if it’s possible to
hot-reload arbitrary Python modules, dependencies or classes such as
SQLAlchemy models.

While I would very much like for it to be the case the real deal-breaker
is updating references to code that has been already imported.

My inclination here is to take a different approach by leveraging the
existing load-balancing infrastructure:

  • start up new workers using the new code/libraries
  • stop sending new requests to the old workers
  • when the old workers have finished their current jobs, let them die off

Am 10.04.2015 um 11:11 schrieb Brian Candler:

My inclination here is to take a different approach by leveraging
the existing load-balancing infrastructure:

  • start up new workers using the new code/libraries * stop sending
    new requests to the old workers * when the old workers have finished
    their current jobs, let them die off

Yes, this sounds like a clean and reasonable approach to the reloading
issue. You only need hot-reloading in production, and there you should
have enough servers.