AMR14

Matt’s notes on (HPC)^3 2014

2014-11-09

Overview

• Tweaked PyBoxLib’s setup.py to support the develop command through setuptools.
• Created a pyboxlib HashStack. Works on the Mac machine (see below).
• Created a PR for HashStack to get this up.

PyBoxLib HashStack

extends:
- name: hashstack
  urls: ['https://github.com/hashdist/hashstack.git']
  key: 'git:c5e2c7a8cad8751fe5a9ab130f267652bf2e09e0'
  file: osx.yaml

parameters:
  fortran: true
  PATH: /Users/mwemmett/opt/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin

packages:
  mpi:
    use: host-mpi
  blas:
    use: host-osx-framework-accelerate
  pyboxlib:

package_dirs:
- pkgs
- base

Note that I set the PATH so that HashDist picked up my local installation of MPICH. The pyboxlib.yaml file currently contains:

extends: [setuptools_package]

dependencies:
  build: [mpi, numpy]
  run: [mpi, numpy]

build_stages:
- name: enter_pyboxlib_dir
  after: prologue
  before: install
  handler: bash
  bash: |
    cd Src/Python

sources:
- url: https://ccse.lbl.gov/pub/Downloads/BoxLib.git
  key: git:9abcfa56867347305f3a0ad9674fb5ae1ec98ef6

2014-11-10

Overview

• Reverted PyBoxLib’s setup.py file to it’s distutils state instead of setuptools.
• Talked with Kyle about domain/patch/grid design in PyClaw and how we will approach regridding.

AMR

• Kyle will create/import an AMR Solver in PyClaw.
• Regridding will be driven by a user settable routine that, given a state, returns a char tagging array. This will be passed down to BoxLib which will take care of regridding.
• Started working on mock Amr and AmrLevel classes to bridge between PyClaw and BoxLib…

2014-11-11

Overview

• Worked with Aron to get the PyBoxLib HashStack up to date.
• Merged Aron’s patch to PyBoxLib to improve the detection of libraries for gfortran.
• Made a mirror of BoxLib on github.com.

AMR

• Will continue to work on regridding…
• Slowly realising that the C++ version of BoxLib is fairly awkward to use as a library, whereas the Fortran version is much more lightweight. As such, I am going to see if I can swap out the backend of BoxClaw to use the Fortran version of BoxLib instead of the C++ version.

2014-11-12

Overview

• Participated in morning discussions regarding the future and sustainability of Clawpack.
• Continued work on BoxClaw and AMR in the afternoon.

BoxClaw

• Spent most of the day getting the Fortran version of PyBoxLib back in working order, and working on a Mac.
• Seems to be working… needs some tidying.

The current FPyBoxLib install creates two dynamic libraries. One contains the Fortran BoxLib, and the other contains a small shim to expose a NumPy view of a FAB. Ideally I could include the shim and BoxLib into one library, but when I try to do that I get segfaults when creating NumPy views. Apparently this is due to complications around import_array(), but I can’t figure this out.

2014-11-13

Overview

• Continued working on regridding. Seems to be working!

FPyBoxLib

FPyBoxLib is very fragile, and some aspects of it are not very Pythonic. In particular

• the __getattr__ in base.py is very fragile;
• the two stage constructors are silly;
• initialization (fboxlib.open()) is fragile.

These need to be addressed.

I think there is also something bad happening with respect to the NumPy views outliving the underlying FAB arrays. Is there a better API for getting and releasing these views?

Chatted with Aron about this and he suggested using Cython to do the wrapping instead of ctypes. Will try this out…

I have started creating a more robust wrapper and will be contained in one Python extension module.

Tagging

Currently BoxLib assumes that a function called tag_boxes is defined to determine which cells on a given level should be refined. This model doesn’t work well when trying to use BoxLib as a library. We’ll have to write a tag_boxes.f90 specific to FPyBoxLib and allow this to call back into Python.

I have created local regrid, make_new_grids and tag_boxes routines. These now pass C function points (c_funptr) down and eventually call back to do the actual tagging.

Now to try attaching a Python routine!

MAN ALIVE! IT’S WORKING! THERE’S MEN ALIVE IN THERE!

Here is a sample script

import fboxlib
fboxlib.open()

ba = fboxlib.boxarray(boxes=[[ (1,1),(100,100) ]])
la = fboxlib.layout(ba)
mf = fboxlib.multifab(la)

fab = mf.fab(1)
print fab.array.shape
fab.array[...] = 1.0
fab.array[10:20,10:20] = 1.02
fab.array[50:60,50:60] = 1.02

def tag_boxes(mf, tb, dx, lev):
    print "TAGGING FROM PYTHON", dx, lev
    # ...

mfs = fboxlib.regrid([la], [mf], [0.5], tag_boxes)