Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Associate feedback files with unique submission attempts #1200

Merged
merged 5 commits into from
Aug 26, 2019

Conversation

jhamrick
Copy link
Member

This updates the logic for computing the feedback hashes so that the hash depends both on the feedback contents as well as the course, assignment id, notebook id, student id, and timestamp. Feedback files using the previous hashing scheme should still be detected, however, so this should be backwards compatible with previously generated feedback files.

This alleviates the issue in #1141 where it was possible to fetch feedback for only some notebooks in an assignment, which had feedback available because the notebook contents hadn't changed. It also fixes the confusing behavior whereby if a student submits an unchanged assignment, they immediately have feedback available---technically it will be the same behavior, but I found this surprising and confusing since I hadn't gone through the actual process of generating feedback yet.

In the process of doing this, I restructured a bit the integration between nbgrader list and the Assignment List extension:

  • nbgrader list now returns a list of submissions for each assignment, rather than a flat list of assignment submissions.
  • nbgrader list also now lists in the command line output what the status of feedback is (available, ready to be fetched, not available).
  • Fixed a few subtle bugs relating to if you have set custom values for the feedback or submitted directories, or the way the nbgrader directory is structured.
  • Also did some general code cleanup (there were some variables that were defined but not used, or two variables defined for the same thing).
  • In the Assignment List extension, I created a new Submission class to handle the view of each individual submission.
  • Separate submissions for an assignment are now listed indented under the assignment name:

Screen Shot 2019-08-26 at 5 33 18 PM

Closes: #1141

@jhamrick jhamrick added this to the 0.6.1 milestone Aug 26, 2019
@rkdarst
Copy link
Contributor

rkdarst commented Aug 26, 2019

One idea I had was to, at submission time, generate a random token and put it into the submitted notebook and some cache. Then, there is less coupling between a hash algorithm and retrieving the notebook later (which is probably a good thing)...

On the other hand, tracking tokens separately from the notebook makes more work. They could be embedded in the notebook metadata, but you have the same problem as the next paragraph.

All this actually this makes me think: if a student A shares their notebook with student B, is B then able to retrieve A's results and grade? It would be best to avoid this if possible...

@jhamrick
Copy link
Member Author

One idea I had was to, at submission time, generate a random token and put it into the submitted notebook and some cache. Then, there is less coupling between a hash algorithm and retrieving the notebook later (which is probably a good thing)...

Yeah, that's not a bad idea. Probably more efficient too as then you don't have to compute the hash of the entire notebook. Really I think you actually shouldn't need the hash at all; I think it should probably be enough to have the unique key of (course, student, assignment, notebook, timestamp)...

if a student A shares their notebook with student B, is B then able to retrieve A's results and grade? It would be best to avoid this if possible...

Yes, I think the current setup would allow this. But I'm not really sure of a way around this barring having better access controls (which we should have once hubshare exists...).

@jhamrick jhamrick merged commit 8ed0b93 into jupyter:master Aug 26, 2019
@jhamrick jhamrick deleted the unique-hash branch August 26, 2019 19:28
@jhamrick
Copy link
Member Author

@meeseeksdev backport to 0.6.x

meeseeksmachine pushed a commit to meeseeksmachine/nbgrader that referenced this pull request Aug 26, 2019
jhamrick added a commit that referenced this pull request Aug 26, 2019
…0-on-0.6.x

Backport PR #1200 on branch 0.6.x (Associate feedback files with unique submission attempts)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible to fetch feedback for only some notebooks in an assignment
2 participants