progress tracked in docs/progress/11-1.md

heliumdatacommons · Nov 1, 2018 · f25e7ff · f25e7ff
1 parent 1ee9ef5
commit f25e7ff
Show file tree

Hide file tree

Showing 7 changed files with 353 additions and 13 deletions.
diff --git a/benchmark/README.md b/benchmark/README.md
@@ -62,10 +62,10 @@ performance drop)
 ### [Benchmark Tools](tools/)
 
 - Small-file I/O
-    - [smallfile](https://github.com/distributed-system-analysis/smallfile)
+    - [ ] [smallfile](https://github.com/distributed-system-analysis/smallfile)
 - Large-file I/O
-    - [fio](http://freshmeat.sourceforge.net/projects/fio)
-    - [iozone](http://www.iozone.org/)
+    - [ ] [fio](http://freshmeat.sourceforge.net/projects/fio)
+    - [x] [iozone](http://www.iozone.org/)
 
 ### Shared filesystem solutions
 
@@ -84,6 +84,9 @@ different use cases, which are likely to impact the performance, *e.g.*, the
 [GlusterFS volume type](https://docs.gluster.org/en/v3/Administrator%20Guide/Setting%20Up%20Volumes/), 
 distribution of backend Ceph OSDs. 
 
+### Findings
+
+
 ### Frequent Asked Question
 
 #### 1. Why do you use NFS Ganesha over GlusterFS but not Ceph?
@@ -105,4 +108,15 @@ mounting CephFS directly.
 [A user experience](http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-November/022474.html) 
 reported in the `ceph-user` mailing list proves my intuition to some extent. 
 
-
+#### 2. Why not use filesystems such as ext4 and XFS over Ceph RBD instead of NFS?
+
+Ceph RBD is exposed as a local *block device* rather than a file system to the clients. In other 
+word, it is unable to handle file and metadata management, which a file system is responsible for. 
+In the experiment, when mapping a Ceph RBD with a file system (*XFS*) on top on multiple hosts, 
+the content in the file system cannot be synchronized in time and requires manual mount/unmount to 
+force the synchronization. As reported in [this thread](), running *ext4* with `-m 0` (the mode in 
+which metadata is maintained in the file system instead of the OS) tends to result in 
+synchronization problems due to the inability of the local file system in handling concurrent 
+operations performed by multiple distributed clients. In contrast, NFS is a known shared file system 
+able to properly handle concurrent operations from multiple clients. Hence, it is advisable to use 
+NFS on top of Ceph RBD to provide shared storage to clients.  
diff --git a/benchmark/ansible/data/benchmark.ipynb b/benchmark/ansible/data/benchmark.ipynb
diff --git a/benchmark/ansible/group_vars/all.yml b/benchmark/ansible/group_vars/all.yml
@@ -3,7 +3,7 @@ ansible_ssh_public_key_file: ~/.ssh/id_rsa.pub
 ansible_ssh_user: centos
 client:
   dir: /
-  host: 10.52.100.9
+  host: 10.52.100.3
   n_parallel: 1
   opts: vers=4
   type: nfs
@@ -19,7 +19,7 @@ disks:
   size: 100
   type: pd-ssd
 image: ubuntu-1804
-machine_type: n1-standard-16
+machine_type: n1-standard-4
 max_retries: 5
 mode: server
 options:

diff --git a/benchmark/ansible/runner.py b/benchmark/ansible/runner.py
@@ -8,15 +8,15 @@
 from subprocess import Popen, PIPE, STDOUT
 
 exp_id = 'ff95ca6b7c3c0b37ffca953f541eaa3d9e6d749a'
-n_repeat = 5
+n_repeat = 10
 
 clients = {
   # 'rbd': dict(host='10.52.100.16', type='nfs', dir='/', opts='vers=4'),
-  'cephfs': dict(host='10.52.100.11', type='ceph', dir='/', opts='mds_namespace=alpha'),
-  # 'nfsv4': dict(host='10.52.100.22', type='nfs', dir='/', opts='vers=4'),
+  # 'cephfs': dict(host='10.52.100.11', type='ceph', dir='/', opts='mds_namespace=alpha'),
+  'nfsv4': dict(host='10.52.100.3', type='nfs', dir='/', opts='vers=4'),
   # 'ganesha': dict(host='10.52.100.23', type='nfs', dir='/', opts='vers=4'),
-  'gfganesha': dict(host='10.52.100.3', type='nfs', dir='/data', opts='vers=4'),
-  'rbdganesha': dict(host='10.52.100.9', type='nfs', dir='/', opts='vers=4'),
+  # 'gfganesha': dict(host='10.52.100.3', type='nfs', dir='/data', opts='vers=4'),
+  # 'rbdganesha': dict(host='10.52.100.9', type='nfs', dir='/', opts='vers=4'),
 }
 
 

diff --git a/benchmark/docs/progress/10-31.md b/benchmark/docs/progress/10-31.md
@@ -3,4 +3,4 @@
 
 - [ ] Test scalability in performance of CephFS, Ceph RBD + NFS Ganesha and GlusterFS + NFS Ganesha 
     - CephFS tends to crash with limited RAM (< 15GB)
-- [x] Fix bug of the `iozone` container
+- [x] Fix bugs of the `iozone` container
diff --git a/benchmark/docs/progress/11-1.md b/benchmark/docs/progress/11-1.md
@@ -0,0 +1,4 @@
+11/01/18
+========
+
+- [x] Create a Jupyter notebook to show the preliminary results of the benchmark
diff --git a/benchmark/docs/progress/README.md b/benchmark/docs/progress/README.md
@@ -9,4 +9,5 @@ Progress
 - [10/28/18](10-28.md)
 - [10/29/18](10-29.md)
 - [10/30/18](10-30.md)
-- [10/31/18](10-31.md)
+- [10/31/18](10-31.md)
+- [11/01/18](11-1.md)