Consider moving all imports to the top of the files

[JoanFM]

As found for this issue #2527 makes me wonder if we should move away from import in inner functions.

As said there, used with threads can lead to deadlocks:

• If we do the inner imports to avoid cyclic dependencies -> We should refactor the code to avoid them

• If we do to minimize the import of external dependencies except for specific features (like scipy for sparse) -> Have a global variable to know if dependency exists.

[hanxiao]

reject, as that will revert all the effort we reduce the loading speed #2510

[JoanFM]

Then we cannot support Threads and Processes Pods mixed up

[hanxiao]

https://stackoverflow.com/questions/6078712/is-it-safe-to-fork-from-within-a-thread/6079669#6079669

[hanxiao]

#2550 (comment)
this is fine to me