-
Notifications
You must be signed in to change notification settings - Fork 101
AMR14
- Tweaked PyBoxLib’s setup.py to support the
develop
command throughsetuptools
. - Created a
pyboxlib
HashStack. Works on the Mac machine (see below). - Created a PR for HashStack to get this up.
extends: - name: hashstack urls: ['https://github.com/hashdist/hashstack.git'] key: 'git:c5e2c7a8cad8751fe5a9ab130f267652bf2e09e0' file: osx.yaml parameters: fortran: true PATH: /Users/mwemmett/opt/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin packages: mpi: use: host-mpi blas: use: host-osx-framework-accelerate pyboxlib: package_dirs: - pkgs - base
Note that I set the PATH
so that HashDist picked up my local
installation of MPICH
. The pyboxlib.yaml
file currently contains:
extends: [setuptools_package] dependencies: build: [mpi, numpy] run: [mpi, numpy] build_stages: - name: enter_pyboxlib_dir after: prologue before: install handler: bash bash: | cd Src/Python sources: - url: https://ccse.lbl.gov/pub/Downloads/BoxLib.git key: git:9abcfa56867347305f3a0ad9674fb5ae1ec98ef6
- Reverted PyBoxLib’s
setup.py
file to it’sdistutils
state instead ofsetuptools
. - Talked with Kyle about domain/patch/grid design in PyClaw and how we will approach regridding.
- Kyle will create/import an AMR Solver in PyClaw.
- Regridding will be driven by a user settable routine that, given a
state, returns a
char
tagging array. This will be passed down to BoxLib which will take care of regridding. - Started working on mock
Amr
andAmrLevel
classes to bridge between PyClaw and BoxLib…
- Worked with Aron to get the PyBoxLib HashStack up to date.
- Merged Aron’s patch to PyBoxLib to improve the detection of libraries for gfortran.
- Made a mirror of BoxLib on github.com.
- Will continue to work on regridding…
- Slowly realising that the C++ version of BoxLib is fairly awkward to use as a library, whereas the Fortran version is much more lightweight. As such, I am going to see if I can swap out the backend of BoxClaw to use the Fortran version of BoxLib instead of the C++ version.
- Participated in morning discussions regarding the future and sustainability of Clawpack.
- Continued work on BoxClaw and AMR in the afternoon.
- Spent most of the day getting the Fortran version of PyBoxLib back in working order, and working on a Mac.
- Seems to be working… needs some tidying.
The current FPyBoxLib install creates two dynamic libraries. One
contains the Fortran BoxLib, and the other contains a small shim to
expose a NumPy view of a FAB. Ideally I could include the shim and
BoxLib into one library, but when I try to do that I get segfaults
when creating NumPy views. Apparently this is due to complications
around import_array()
, but I can’t figure this out.
- Continued working on regridding. Seems to be working!
FPyBoxLib is very fragile, and some aspects of it are not very Pythonic. In particular
- the
__getattr__
inbase.py
is very fragile; - the two stage constructors are silly;
- initialization (
fboxlib.open()
) is fragile.
These need to be addressed.
I think there is also something bad happening with respect to the NumPy views outliving the underlying FAB arrays. Is there a better API for getting and releasing these views?
Chatted with Aron about this and he suggested using Cython to do the
wrapping instead of ctypes
. Will try this out…
I have started creating a more robust wrapper and will be contained in one Python extension module.
Currently BoxLib assumes that a function called tag_boxes
is defined
to determine which cells on a given level should be refined. This
model doesn’t work well when trying to use BoxLib as a library. We’ll
have to write a tag_boxes.f90
specific to FPyBoxLib and allow this
to call back into Python.
I have created local regrid
, make_new_grids
and tag_boxes
routines. These now pass C function points (c_funptr
) down and
eventually call back to do the actual tagging.
Now to try attaching a Python routine!
MAN ALIVE! IT’S WORKING! THERE’S MEN ALIVE IN THERE!
Here is a sample script
import fboxlib
fboxlib.open()
ba = fboxlib.boxarray(boxes=[[ (1,1),(100,100) ]])
la = fboxlib.layout(ba)
mf = fboxlib.multifab(la)
fab = mf.fab(1)
print fab.array.shape
fab.array[...] = 1.0
fab.array[10:20,10:20] = 1.02
fab.array[50:60,50:60] = 1.02
def tag_boxes(mf, tb, dx, lev):
print "TAGGING FROM PYTHON", dx, lev
# ...
mfs = fboxlib.regrid([la], [mf], [0.5], tag_boxes)
- Reworked and tidied up the PyBoxLib wrapper quite a bit.
- Tidied up the
pyboxlib
HashStack and created a PR against Aron’sahmadia/hpc3_2014_stack
branch.
After a final push, witness the magic…
- Create a HashDist profile and bring in the
pyboxlib
package. - Run the following script (see
BoxLib/Src/Python/F90/tests/regrid.py
)…
We first create a 2d multifab
from (0,0) to (100,100) and fill it
with 1.0, except in two blocks from (11,11) to (21,21) and (51,51) to
(61,61) where we fill it with 2.0.
We then calls BoxLib’s regrid
routine and pass it the tag_boxes
routine. This tag_boxes
routine gets called by BoxLib to mark cells
for refinement. Currently we refine on the first level if the first
component of the original multifab is > 1.0 (which corresponds to the
two blocks that we set to 2.0).
The underlying layout are printed… note that after regridding there are two high-resolution boxes around the two blocks that we tagged!
import fboxlib
fboxlib.open()
ba = fboxlib.boxarray([[(0,0), (100,100)]])
la = fboxlib.layout(ba)
mf = fboxlib.multifab(la, nc=3, ng=0)
print "#"*80
print "# before regridding"
la.echo()
fab = mf.fab(1)
fab.array[...] = 1.0
fab.array[10:20,10:20] = 2.0
fab.array[50:60,50:60] = 2.0
def tag_boxes(mf, tb, dx, lev):
if lev > 1:
return
mf = fboxlib.multifab(cptr=mf)
tb = fboxlib.lmultifab(cptr=tb)
mfab = mf.fab(1)
tfab = tb.fab(1)
tfab.array[mfab.array[:,:,0] > 1.0] = 1
mfs = fboxlib.regrid([la], [mf], [0.5], tag_boxes)
print "#"*80
print "# after regridding"
for mf in mfs:
mf.layout.echo()
Here are the results…
################################################################################ # before regridding LAYOUT[(* ID = 1 DIM = 2 NBOXES = 1 PD = BOX[{0, 0}, {100, 100}] 1: BOX[{0, 0}, {100, 100}] 0 *)] ################################################################################ # after regridding LAYOUT[(* ID = 1 DIM = 2 NBOXES = 1 PD = BOX[{0, 0}, {100, 100}] 1: BOX[{0, 0}, {100, 100}] 0 *)] LAYOUT[(* ID = 7 DIM = 2 NBOXES = 2 PD = BOX[{0, 0}, {201, 201}] 1: BOX[{16, 16}, {47, 47}] 0 2: BOX[{96, 96}, {127, 127}] 0 *)]