Particle swarm sampler #17
Conversation
marcofavoritobi
force-pushed
the
particle-swarm-sampler
branch
2 times, most recently
from
69494d3 to
8e7992a
Compare
Implementation of the classical particle swarm sampler.
References:
- Shi, Yuhui, and Russell Eberhart. "A modified particle swarm optimizer."
1998 IEEE international conference on evolutionary computation proceedings.
- Eberhart, Russell, and James Kennedy. "A new optimizer using particle swarm theory."
MHS'95. Proceedings of the sixth international symposium on micro machine and human science.
AldoGl
force-pushed
the
particle-swarm-sampler
branch
from
8e7992a to
d5cec59
Compare
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #17 +/- ##
==========================================
- Coverage 99.44% 98.63% -0.82%
==========================================
Files 22 23 +1
Lines 1086 1171 +85
==========================================
+ Hits 1080 1155 +75
- Misses 6 16 +10
|
There was a problem hiding this comment.
Amazing work @marcofavoritobi, thanks!
I have added this sampler to the "overview" notebook in the example folder and I have also added an essential test.
I would have only some these small observations:
-
Is it the case that the PS sampler simply cannot run in combination with other samplers? If so, we could add some kind of check in the code to guarantee that this never happens.
-
I would remove the possibility of printing position and velocities of the particles as this seem too low level information. What do you think? I am open to change my mind here.
-
I would move your notebook "particle_swarm_sampler" to the examples folder and create a symbolic link to the docs folder, as done for the other notebooks.
black_it/samplers/particle_swarm.py
Outdated
| - r1 and r2 are two random numbers uniformly distributed in the range of (0, 1). | ||
|
|
||
| This sampling method has to be used alone in Black-it Calibrator, because existing_losses and existing_points | ||
| arguments of sample_batch are assumed to be computed by this sampler. |
There was a problem hiding this comment.
What happens if one uses the PS in conjunction with other methods?
There was a problem hiding this comment.
in fact, the docstring is not quite correct (my bad). _previous_batch_index_start will necessarily point to the index of existing_points/existing_losses at the moment of the previous call to sample_batch. Any other existing batch is ignored. Therefore, atm, the PS sampler works as if it were alone.
We could switch the behaviour by considering the global loss as the true global loss of the calibration task, and not just the global loss within the PS sampler.
There was a problem hiding this comment.
I kind of like this idea actually... It would allow different samplers to benefit from each other, and this is one of the nice features of Black-it I think
There was a problem hiding this comment.
But I think we should leave this as an option
black_it/samplers/particle_swarm.py
Outdated
| ) | ||
| self._previous_batch_index_start = len(existing_points) | ||
|
|
||
| if self._verbose: |
There was a problem hiding this comment.
I would remove these print statements for the moment as they seem useful only for debugging. We could maintain them if we had several level of verbosity (0, 1, 2, 3, ...) but I having only 0 and 1 printing positions and velocities of the particles seems to much
There was a problem hiding this comment.
sounds good, let's remove the prints and the verbose option
AldoGl
force-pushed
the
particle-swarm-sampler
branch
from
f2ca921 to
bb7e80e
Compare
Sounds good :) |
AldoGl
force-pushed
the
particle-swarm-sampler
branch
from
bb7e80e to
9808aab
Compare
…ate symbolic link to docs folder
…pler This commit adds a new boolean option to the constructor of ParticleSwarmSampler, namely global_minimum_across_samplers. This option allows to switch between two types of behaviours of the particle swarm sampler: If `global_minimum_across_samplers` is False, then the velocity update rule, for the global minimum part, uses the current position of the particle that in its history has sampled, so far, the best set of parameters. Else, if `global_minimum_across_samplers` is True, then the velocity update rule returns the point in parameter space that achieved the minimum loss. Note that this point could have been sampled by a different sampler than "self".
marcofavoritobi
force-pushed
the
particle-swarm-sampler
branch
from
6f4c5e7 to
ad11165
Compare
|
Thanks @marcofavoritobi for the new "global minimum" feature 👍🏼. I updated the docstrings and the notebooks, I think we can merge now provided that tests pass |
Proposed changes
This PR adds an implementation of the classical particle swarm sampler.
References:
1998 IEEE international conference on evolutionary computation proceedings.
MHS'95. Proceedings of the sixth international symposium on micro machine and human science.
Fixes
If it fixes a bug or resolves a feature request, be sure to link to that issue.
Types of changes
What types of changes does your code introduce?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply.developbranch (left side). Also you should start your branch off ourdevelop.Further comments
If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...