Skip to content

Commit

Permalink
Completely rewrite textbook module and its unit tests (#198)
Browse files Browse the repository at this point in the history 
* Completely rewrite textbook module and its unit tests

As discussed in the CSC Discord server, textbook.py has been found to be
broken. Parts of the underlying textbook API had changed, invalidating
most of the existing code. For instance, one major change to the
underlying API is that the API's request URLs now require CSRF tokens,
which I believe weren't required before. Major breaking changes like
this weren't caught earlier because textbook_test.py mostly didn't
validate any function outputs, only their return types, and thus the
module's unit tests merely gave the illusion of correctness.

I rewrote the module along with its unit tests to adjust for the changed
textbook API, while taking the liberty to make the code more readable
and robust based on discussions in the CSC Discord server. This includes
better error handling, input validation, automatic retrieval of CSRF
tokens, and much more comprehensive unit tests.

* Remove slots=True from @DataClass annotation for Python 3.9 compatibility
  • Loading branch information
@tianyizheng02
tianyizheng02 authored Aug 18, 2024
1 parent f9f3def commit 6d6b1ef
Show file tree
Hide file tree
Showing 9 changed files with 4,112 additions and 503 deletions.
580 changes: 214 additions & 366 deletions pittapi/textbook.py
View file

Large diffs are not rendered by default.

238 changes: 238 additions & 0 deletions tests/samples/textbook_base_page.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
<!DOCTYPE html>
<html lang='en-US'>
<head>
<meta content='text/html;charset=UTF-8' http-equiv='Content-Type'>
<meta content='width=device-width, initial-scale=1, maximum-scale=1' name='viewport'>
<meta name="csrf-param" content="authenticity_token" />
<meta name="csrf-token" content="1MTtTVOcQCCXDjKNKTqkfiwp0lmLWz1RvFy2ed65XeyGO4on-8zWsQpEAt4cjiH0glx9CIyjhAOKpXhIqDK_vg" />
<link href='https://vst-assets.verbacompete.com/assets/favicon_compare-08e4986cab90bd2622956d5b0303387270841228afe70b7c758b4a40843e8be8.ico' rel='shortcut icon'>
<title>Compare Textbook Prices Online | University Store on Fifth</title>
<link href='https://vst-assets.verbacompete.com/assets/compare_assets/pitt-pc/v3-4d870539649f9b1797d6631e05aa4843a8cf1ccd.css' rel='stylesheet'>
<link href='https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.css' rel='stylesheet'>
<script src='//use.typekit.net/vvv5bqy.js' type='text/javascript'></script>
<script>
try{Typekit.load();}catch(e){}
</script>
<script src="https://vst-assets.verbacompete.com/assets/compare/v3-249443292d69473ed7b0d046a70c502d5f024caca540a10137900808975af855.js"></script>
</head>
<!--[if lt IE 7]>
<body class="ie ie6 index" id="compare">
<![endif]-->
<!--[if IE 7]>
<body class="ie ie7 index" id="compare">
<![endif]-->
<!--[if IE 8]>
<body class="ie ie8 index" id="compare">
<![endif]-->
<!--[if IE 9]>
<body class="ie ie9 index" id="compare">
<![endif]-->
<![if (!IE)]>
<body class='index' id='compare'>
<![endif]>
<div id='header' role='banner'>
<div id='banner'>
<div class='logo'><img src="https://vst-assets.verbacompete.com/assets/compare_assets/pitt-pc/pitt_banner-13fb4a443f97e8c86f7b6f1b9020c6e74c190fdb.png" /></div>
<div aria-hidden='true' class='addthis_toolbox addthis_default_style addthis_32x32_style'>
<a aria-hidden='true' class='addthis_button_facebook first'>
<i class='fa fa-facebook-f icon-facebook'></i>
</a>
<a aria-hidden='true' class='addthis_button_email last'>
<i class='fa fa-envelope icon-envelope'></i>
</a>
<div class='clear'></div>
</div>
</div>
</div>

<script src="https://vst-assets.verbacompete.com/assets/framebuster-324047eac65b190ff34d8bdfd9a913905d5b70cf5c3e5d4234b9a6f203a0a515.js"></script>
<!--[if lt IE 9]>
<div class="ie_warning">
<div class="text">
<h1>This site isn't optimized for Internet Explorer!</h1>
<p>We highly recommend you access this site through a modern web browser for a fast, full-featured experience. Download one of our favorites (Chrome, Firefox, or Safari) by clicking the icons to the right.</p>
</div>
<div class="icons">
<div class="icons_bg">
<a href="http://google.com/chrome" id="chrome"></a>
</div>
<div class="icons_bg">
<a href="http://www.mozilla.com/en-US/products/download.html?lang=en-US" id="firefox"></a>
</div>
<div class="icons_bg">
<a href="http://www.apple.com/safari/download/" id="safari"></a>
</div>
<div style="clear: both"></div>
</div>
</div>
<div style="clear: both"></div>
<![endif]-->

<noscript>
<div class="ie_warning">
<div class="text">
<h1>This site requires JavaScript for full functionality!</h1>
<p>We highly recommend you access this site through a browser with JavaScript support for a full-featured experience. Download one of our favorites by clicking the icons to the right.</p>
</div>
<div class="icons">
<div class="icons_bg">
<a href="http://google.com/chrome" id="chrome"></a>
</div>
<div class="icons_bg">
<a href="http://www.mozilla.com/en-US/products/download.html?lang=en-US" id="firefox"></a>
</div>
<div class="icons_bg">
<a href="http://www.apple.com/safari/download/" id="safari"></a>
</div>
<div style="clear: both"></div>
</div>
</div>
<div style="clear: both"></div>
</noscript>

<!-- Allows render_partial to find both .mobile and .html templates -->
<div class='compare index' id='content'>
<div id='menu-flash'></div>
<div id='menu'>
<div class='menu-wrapper'>
<div id='standard-menu'>
<div class='header' id='choose'>
Choose Your Courses
<div class='arrow-box'>
<div class='arrow'></div>
</div>
</div>
<div class='header' id='course-list'>
Current Course List
<div class='arrow-box'>
<div class='arrow'></div>
</div>
</div>
</div>
<div id='responsive-menu'>
<div class='header' id='choose'>
Choose Your Courses
<div class='arrow-box'>
<div class='arrow'></div>
</div>
</div>
<a class="responsive-menu-button highlight" href="#left-menu"><i class='fa fa-align-justify icon-align-justify'></i>
Course List
</a><div class='clear'></div>
</div>
</div>
</div>
<div class='table-wrapper'>
<div id='search-structure'>
<!-- / TODO: Remove -size user.setting from v3? -->
<!-- / - size = locale_user.setting("pc.settings.course_selector.font_size") -->
<div id='search'>
<label class='sr-only' for='term-select'>
Select a Term
</label>
<div class='search-wrapper' id='term'>
<select id='term-select'></select>
<div class='spinner_box'>
<div class='loader'></div>
</div>
</div>
<label class='sr-only' for='department-select'>
Select a department after choosing a term
</label>
<div class='search-wrapper' id='department'>
<div class='spinner_box'>
<div class='loader'></div>
</div>
<select id='department-select'></select>
</div>
<label class='sr-only' for='course-select'>
Select a course after choosing a department
</label>
<div class='search-wrapper' id='course'>
<div class='spinner_box'>
<div class='loader'></div>
</div>
<select id='course-select'></select>
</div>
<label class='sr-only' for='section-select'>
Select a section after choosing a course
</label>
<div class='search-wrapper' id='eset'>
<div class='spinner_box'>
<div class='loader'></div>
</div>
<select id='section-select'></select>
</div>
</div>
<div class='selected-wrapper'>
<div class='selected-list'>
<div class='none-selected'>No Courses Selected!</div>
</div>
</div>
</div>
<div class='go-wrapper'>
<a href='#' id='go'>
<i class='fa fa-share icon-mail-forward'></i>
Compare Prices on These Course Materials
</a>
</div>
</div>
</div>
<script>
$(document).ready(function(){
var terms = new Verba.Compare.Collections.Terms([{"id":"78104","name":"Fall 24","inquiry":true,"ordering":true}]);
var view = new Verba.Compare.Views.CourseSelector({el: "body", terms: terms});
});
</script>
<script>
if (!localStorage.getItem('cookie-policy-accepted')) {
$(document).ready(function(){
var banner = $('<div id="cookie-notice"><p>We use cookies to provide you with the best experience on our website. By using our site, you are agreeing to our <a href="https://vitalsource.com/cookies" target="_blank" rel="noopener noreferrer">Cookie Policy.</a></p></div>');

var button = $('<button>Accept &amp; Close</button>');

button.on('click', function() {
localStorage.setItem('cookie-policy-accepted', true);
banner.hide();
});

banner.append(button);
$(document.body).append(banner);
});
}
</script>


<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-ZV3F11F7QR"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());

// Global Compare Tracker
gtag('config', 'G-ZV3F11F7QR');

// Client-Specific Tracker
gtag('config', 'UA-41807431-1');
</script>

<div id='footer' role='contentinfo'>
<div class='wrapper'>
<div class='left'>
© 2024
<a target="_blank" rel="noopener noreferrer" href="https://www.verbasoftware.com">Verba | VitalSource</a>
</div>
<div class='right'>
<a target="_blank" rel="noopener noreferrer" href="https://vitalsource.com/privacy">Privacy Policy</a>
<span aria-hidden='true' class='dot'></span>
<a target="_blank" rel="noopener noreferrer" href="https://vitalsource.com/cookies">Cookie Policy</a>
<span aria-hidden='true' class='dot'></span>
<a href="/cdn-cgi/l/email-protection#0d7965687863647b687f7e6479747e79627f684d7d64797923686978">Contact Store</a>
</div>
<div class='clear'></div>
</div>
</div>

<script data-cfasync="false" src="/cdn-cgi/scripts/5c5dd728/cloudflare-static/email-decode.min.js"></script></body>
</html>
1,595 changes: 1,594 additions & 1 deletion tests/samples/textbook_courses_CS.json
View file

Large diffs are not rendered by default.

Loading

0 comments on commit 6d6b1ef

Please sign in to comment.