Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement single-binary concurrent workers #18

Open
3 tasks
robacarp opened this issue Sep 10, 2018 · 1 comment
Open
3 tasks

Implement single-binary concurrent workers #18

robacarp opened this issue Sep 10, 2018 · 1 comment
Labels

Comments

@robacarp
Copy link
Collaborator

robacarp commented Sep 10, 2018

This would also be a good opportunity to clean up the Runner class, as it's currently wired up like a pile of spaghetti.

Off the top of my head, here's a checklist:

  • reconsider the singleton pattern in Runner
  • take a runtime parameter for spawns (and forks?)
  • both runner and overmind have infinite loops...are both necessary?
@m-o-e
Copy link

m-o-e commented Jul 6, 2022

Fyi, I'm using this to have multiple Mosquito::Runner's in one process (works with Mosquito v1.0.0rc2):

class Swamp
  Log = ::Log.for("swamp")

  @@fibers = [] of Fiber
  def self.start(n=1)
    raise Exception.new("Swamp already started") unless @@fibers.empty?
    n.times do
      @@fibers << spawn { Mosquito::Runner.start }
    end
    Log.info &.emit("Swamp started (#{n} workers)", { ev: "swamp.started", workers: n })
  end

  def self.stop(timeout = 28.seconds)
    Log.info &.emit("Swamp stopping", { ev: "swamp.stopping", workers: @@fibers.size, timeout: timeout.total_seconds })
    t = Time.monotonic
    Mosquito::Runner.stop
    begin
      while !@@fibers.empty?
        fiber = @@fibers.shift
        while !fiber.dead?
          sleep 0.42
          raise IO::TimeoutError.new if Time.monotonic - t > timeout
        end
      end
    rescue ex : IO::TimeoutError
      Log.error &.emit("Swamp failed to stop", { ev: "swamp.stop_failed", reason: "timeout", workers: @@fibers.size })
    else
      Log.info &.emit("Swamp stopped", { ev: "swamp.stopped", workers: @@fibers.size })
    end
  end
end

@robacarp robacarp changed the title Implement fiber powered workers Implement single-binary concurrent workers Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants