AddCat Bitswap Integration Tests

[btc]

features

• adds delay/latency configuration to the datastore, Bitswap.VirtualNetwork, and mockrouting types.
• adds benchmarks to observe the effect that (each type of) latency has on AddCat performance.
• adds long-running integration tests. execute with env variable IPFS_EPIC_TEST

learnings

• of the three types of latency, routing has the largest adverse effect on performance https://build.protocol-dev.com/job/addcat-benchmarks/4/console
• 500 MB transfer with "NY to SF" network latency, "SSD" blockstore latency, and 200ms routing latency runs ~forever. We now have a reproducible test case. https://build.protocol-dev.com/job/gotest/168/console

status

• Still playing around with variables.
• Will squash commit history before merging
• we may want to rebase the new bitswap changes on top of this for testing @whyrusleeping
• nice-to-have: mock out the network at a lower layer
• I hereby dedicate this PR to Riff Raff. https://www.youtube.com/watch?v=TL0RqEsF_mY

[jbenet]

this is awesome. you are a god among mortals.

[btc]

RFCR @jbenet

• squash commits
• all commits pass tests (job in-progress) https://build.protocol-dev.com/job/ipfs_test_all_commits_on_branch/20/
• add benchmark job to CI https://build.protocol-dev.com/job/addcat-benchmarks/

BenchmarkInstantaneousAddCat32MB            4159322045 ns/op       8.07 MB/s
BenchmarkRoutingSlowAddCat32MB              40494165777 ns/op      0.83 MB/s # AHHHH
BenchmarkNetworkSlowAddCat32MB              4168220210 ns/op       8.05 MB/s
BenchmarkBlockstoreSlowAddCat32MB           6712107451 ns/op       5.00 MB/s

[jbenet]

you can use go-random for a Reader with much faster randomness generation. dont really need cryptographic strength for these tests, and randcrypto is slow.

[btc]

Ah. Wonderful. I was looking for a way to get determinism.

71d6e5c

The icing on the cake would be for random to provide an io.Reader. It would cut down peak memory usage and make it possible to create larger tests. Not critical yet. The blockstore will limit test size before that will.

random.NewPseudoRandomReader(count, seed)

[jbenet]

huh, i thought it did. oh well

On Sat, Dec 13, 2014 at 7:41 AM, Brian Tiger Chow [email protected]
wrote:

In epictest/addcat_test.go
#444 (diff):

+func AddCatBytes(conf Config) error {
+

• sessionGenerator := bitswap.NewSessionGenerator(

•   tn.VirtualNetwork(delay.Fixed(conf.NetworkLatency)), // TODO rename VirtualNetwork
  

•   mockrouting.NewServerWithDelay(delay.Fixed(conf.RoutingLatency)),
  

• )

• adder := sessionGenerator.Next()
• catter := sessionGenerator.Next()
• catter.SetBlockstoreLatency(conf.BlockstoreLatency)

• adder.SetBlockstoreLatency(0) // disable blockstore latency during add operation
• var data bytes.Buffer
• // FIXME replace with a random data generator that reproduces data given a seed value
• io.Copy(&data, &io.LimitedReader{R: randcrypto.Reader, N: conf.DataAmountBytes})

Ah. Wonderful. I was looking for a way to get determinism.

71d6e5c
71d6e5c

The icing on the cake would be for random to provide an io.Reader

random.NewPseudoRandomReader(count, seed)

—
Reply to this email directly or view it on GitHub
https://github.com/jbenet/go-ipfs/pull/444/files#r21788868.

[btc]

before

BenchmarkInstantaneousAddCat1MB       20      91798436 ns/op      11.42 MB/s
BenchmarkInstantaneousAddCat2MB       10     180849588 ns/op      11.60 MB/s
BenchmarkInstantaneousAddCat4MB        5     360574235 ns/op      11.63 MB/s
BenchmarkInstantaneousAddCat8MB        2     719927342 ns/op      11.65 MB/s

after

BenchmarkInstantaneousAddCat1MB       50      32412242 ns/op      32.35 MB/s
BenchmarkInstantaneousAddCat2MB       50      64622451 ns/op      32.45 MB/s
BenchmarkInstantaneousAddCat4MB       20     199377593 ns/op      21.04 MB/s
BenchmarkInstantaneousAddCat8MB        5     419447024 ns/op      20.00 MB/s

[jbenet]

I really like this setup. 👍

[jbenet]

LGTM. RTM So far, my favorite PR of the month. 👍 👍 🌟 🌟 👏 👏

[whyrusleeping]

This is awesome. Ill rebase my changes over this.

[btc]

In case you're not interested in wading through a bayou knee-deep in merge conflicts, I got my socks wet here: https://github.com/jbenet/go-ipfs/tree/bitswap-rounds-rebase-whyohmy