Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replaced read-pkg-up with cache-aware package locator #1017

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

hulkish
Copy link
Contributor

@hulkish hulkish commented Feb 8, 2018

No description provided.

@hulkish hulkish force-pushed the cached-read-pkg-up branch from a9fb1b0 to 09fa103 Compare February 8, 2018 00:43
// if not found, allopw search to continue
if (err.code !== 'ENOENT') throw err
}
} while (dirname !== (dirname = path.dirname(dirname)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way this can be written without conflating an assignment with an expression?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ljharb Yes, but this plays nicer with the goal of "fail fast". Meaning, if i were to replace with a typical condition-first loop such as for/while - it would require me to add logic which ensured the initial dirname was searched.


const response = super.readPackageSync(location)
this.store[location] = response.result
return response
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we only ever use the result, why return the entire response?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ljharb It's needed so that readPackageUpSync can report all locations which failed to yield a file - in the event that no fles were located.

@coveralls
Copy link

coveralls commented Feb 8, 2018

Coverage Status

Coverage increased (+0.2%) to 96.449% when pulling 14fdf06 on hulkish:cached-read-pkg-up into 402c60a on benmosher:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.1%) to 96.091% when pulling a9fb1b0 on hulkish:cached-read-pkg-up into 219a8d2 on benmosher:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.1%) to 96.091% when pulling a9fb1b0 on hulkish:cached-read-pkg-up into 219a8d2 on benmosher:master.

@hulkish hulkish force-pushed the cached-read-pkg-up branch from 04eab1d to efe80dd Compare February 8, 2018 12:40
@hulkish
Copy link
Contributor Author

hulkish commented Feb 8, 2018

@ljharp one other thing that occurs to me... is it safe for the cache to exist at module level as i currently have it?

Are there instances where I should be clearing the cache? I'd like to add some tests to account for this.

@hulkish
Copy link
Contributor Author

hulkish commented Feb 10, 2018

@ljharb Ok I think this is ready now.

One easy way to see the benefit here, is by running the following command in both master and the branch for this pr (note the --trace-sync-io):

BABEL_ENV=test NODE_PATH=./src ./node_modules/.bin/mocha --require babel-register tests/src/rules/no-extraneous-dependencies.js --trace-sync-io 2>&1 | grep -c "src/rules/no-extraneous-dependencies.js"

The difference is clear when u see that master produces 524 io calls originating from the no-extraneous-dependencies rule. This pr branch, using same command yields 25

@hulkish hulkish force-pushed the cached-read-pkg-up branch 5 times, most recently from 0a298c4 to 3c0aeb3 Compare February 10, 2018 18:47
Copy link
Member

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall

import os from 'os'
import fs from 'fs'

export default class CachedPackageLocator {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder if this would be better extracted to its own published package with its own tests - is that something you'd be willing to publish and maintain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm sure, I'll try to find some some for this either today or this week.

const options = context.options[0] || {}
const filename = context.getFilename()
const deps = getDependencies(context, options.packageDir)
const deps = packageLocator.readUpSync(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems kind of odd to instantiate packageLocator only once and then use it only once to get at the readUpSync instance method. Perhaps readUpSync could just be exported directly as a singleton function, or perhaps CachedPackageLocator could be a factory function instead?


before(function () {
sandbox = sinon.sandbox.create()
sandbox.stub(fs, 'readFileSync').reset()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would it need to be reset immediately after creating the stub?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't work without it. Some kind of sinon bug, I gathered.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when it doesn't work?

@hulkish hulkish force-pushed the cached-read-pkg-up branch 2 times, most recently from 154b3a7 to 14fdf06 Compare March 4, 2018 15:02

before(function () {
sandbox = sinon.sandbox.create()
sandbox.stub(fs, 'readFileSync').reset()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when it doesn't work?

})

after(function () {
sandbox.restore()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be done afterEach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, that's what I had originally also - it does not initialize the stubbed method without calling reset, though. Try it yourself!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll play with it; but in the meantime let's change all this stuff so the sandbox is created before but everything else is in a beforeEach/afterEach.

@ljharb
Copy link
Member

ljharb commented May 14, 2021

@hulkish are you still interested in completing this?

@ljharb ljharb marked this pull request as draft May 14, 2021 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants