Provide Lwt.sleep as a Unix-less function

[raphael-proust]

Whilst discussing js-of-ocaml programs with @hhugo , we discussed the possibility of providing a platform-independent Lwt.sleep. The function Lwt.sleep would be to Lwt_unix.sleep like Lwt.pause is to Lwt_unix.yield. Below is a short description of a possible implementation. Please provide comments, feedback, ideas, etc.

(* temporary buffer of sleeping promises before the scheduler reaps them *)
let sleeps_waiting_for_a_scheduler : (float * unit wakener) = ref []

(* how to sleep when there is no scheduler *)
let sleep_no_scheduler f =
  let wt, wk = Lwt.task () in
  sleeps_waiting_for_a_scheduler := (f, wk);
  wt

(* how to sleep *)
let sleep_dynamically_picking_the_scheduler = ref sleep_no_scheduler

let sleep f = !sleep_dynamically_picking_the_scheduler f

And then when we call Lwt_main.run it installs a proper callback before entering the loop:

let sleep_in_scheduler f =
  (* code that is currently Lwt_unix.sleep *)

let run p =
  let rec run_loop () =
    (* unchanged *)
  in

  let sleeps = !Lwt.sleeps_waiting_for_a_scheduler in
  Lwt.sleeps_waiting_for_a_scheduler := [];
  List.iter (fun (f, wk) -> Lwt_engine.on_timer …) sleeps; (* injects all the waiting sleeps in the engine *)
  Lwt.sleep_dynamically_picking_the_scheduler := sleep_in_scheduler;

  let r = run_loop () in

  (* restore scheduler-less sleeping after run is done *)
  Lwt.sleep_dynamically_picking_the_scheduler := Lwt.sleep_no_scheduler;

  r

I omitted some details (e.g., exception management around the call to run_loop), but the gist is here.

A serious issue of the approach shown here is that the timers start ticking down only once the call to run is made. I'm not sure how to resolve this without introducing a call to a function to get time which would be backend dependent.

Maybe the engine needs to set the sleep-injection function before the scheduler starts. Basically, as soon as the current_engine is set.

Ideas? Comments?

[dinosaure]

Hi,

Your approach seems a bit invasive for our use case (MirageOS) where we implement our own scheduler with our own time function (which depends on the backend). Currently, you can see an example of our loop here. I don't have, a priori, remarks against that but we must be in sync about your plan and our current ecosystem to be sure that we still able to inject our time function and our "scheduler" regardless what you want to do.

More concretely, we must be sure that Lwt.sleep can be easily replaced by our specific syscall. Then, we will continue to implement our run function as we do currently (may be @hannesm/@avsm will be more advised than me about that).

[raphael-proust]

Thanks for the comment. The timing is what is giving me the most doubts and I'm happy to have a more concrete example to check. I'll have a look at it and see if I can refine the proposal to take it into account.

[avsm]

What's the actual problem you're trying to fix here, @raphael-proust @hhugo? It would be good to see a small example of something that needs to sleep in a platform-independent way for both jsoo and unix compilation, and then it'll be easier to figure out how Mirage fits into it. In most cases, the amount of time something sleeps for is pretty closely tied to the IO usage, which in turn is tied to the backend in use.