Skip to content

Rails app that mines the U.S. Chess Federation website and presents interesting and hopefully helpful data.

Notifications You must be signed in to change notification settings

jtisaac/USCF-Stats

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

96 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

This is a Ruby on Rails application that will mine the United States Chess Federation website for data and present it.

When I was a more serious chess player, I would always do things like this for myself. I kept meticulous records of all my games and would keep track of how I did under various circumstances against various people, look at my trends, and try to use them to understand myself better. I reveled in my biggest upsets and best performances while being upset by my worst losses. I compared myself to my friends and rivals using various criteria and invented goals for myself.

I am hoping to apply these same ideas and experiences to build an app that a lot of people can use.

9/22:

-Initiated project
-Learned about Nokogiri and Mechanize via RailsCasts
-Built web scraper to parse the USCF website and get data using regular expressions and CSS selectors
-Create UscfWebsite class to hold common methods for accessing the USCF website

9/23:

-Learned about Ajax in Rails (annoying because a lot changed in Rails 3.1 so all
 the tutorials were outdated)
-Used instance variables in partials with Ajax calls to dynamically update the page using a Ruby object
-Wrote code to collect all the opponents someone has ever played and sort them by rating
-Made MySQL database, pushed project to Heroku, sorted out little bugs in transitioning
-Doing it all in one call caused Heroku to timeout for players with long histories, 
 so I went through and made the entire thing a collection of Ajax calls that pass JSON objects
 and arrays back and forth to collect all the data
-Added some jQuery to make the user interface a little more friendly and show a progress
 indicator with % complete and the date of the last tournament processed

9/24:

-Worked on the Stanford machine learning course

9/25:

-Worked on the Stanford machine learning course

9/26:

-Built a good user interface
-Wrote an intro to the app to show to people

9/27:

-Spent a long time fighting with the poorly-formatted USCF website
-Learned that Ruby regular expressions don't recognize some of the UTF-8 blank space characters

9/28:

-Talked to GoDaddy and learned how to properly redirect naked domains through the Heroku proxy to the main page.
-Put a link to the website on chess.stackexchange.com

9/29:

-Added a bunch of methods to the API
-Did some bug fixing
-Played in a tournament and invited people to visit the website

9/30:

-Redesigned the code for the demo to make it better
-Added a lot of new columns
-Made the user interface a lot better
-Let users choose between regular and quick history
-Put sorting on the columns

10/1:

-Identified problems in the current demo design
-Learned about Redis
-Implemented a Redis NoSQL cloud database and set up the application to work with it
-Rewrote various parts of the application to use the new database
-Fixed some bugs
-Set the type and USCF ID to be session variables

10/2:

-Fixed some small bugs

10/3:

-Worked on the Stanford machine learning course

10/4:

-Went to hack night with the Research Triangle Ruby meetup group
-Built the get_results_against_opponents_from_tournament method in the USCF Website API

10/5:

-Vacation to visit my father, didn't work on stuff

10/6:

-Still on vacation
-Poked at the FANN neural network library

10/7:

-Still on vacation
-Learned some D3JS
-Built draft of delta chart

10/8:

-Worked on learning D3JS

10/9:

-Added D3Js axis to delta chart
-Did some math to scale the locations and sizes of the axis and bubbles according to the specified width/height.
-Made the bubbles say who you played
-Updated the interface to make it a little better and let users input the ID of who to check out

10/10:

-Usability testing, someone noticed a major bug in the deltas chart in later versions of Firefox.
-Opened issue, found solution and fixed it.

10/11:

-Busy watching the debate

10/12:

-Busy doing other things

10/13:

-Did some research and discovered locally weighted logistic regression
-Read Ph.D thesis paper and tried to implement a library for the algorithm in Ruby
-Got irritated with poorly written paper
-Made the deltas example a collection of AJAX calls similar to the demo example
-UI/UX changes to the deltas example

10/14:

-Fixed a defect in the deltas page
-UI/UX changes to deltas page
-Looked up the mathematical papers behind the USCF rating system to get a better understanding of it
-Opened up the JavaScript for the performance calculator on the USCF website and read it
-Implemented a binary search algorithm similar to that on the USCF website in Ruby to calculate performances
-Learned how to do line charts in D3JS
-Learned how to do time scales in D3JS
-Built a line chart that shows performance ratings and regular ratings over time
-Added buttons to deltas and performance charts to add gridlines.

10/15:

-Found out there's a tiny book on D3JS, read about fifty pages of it
-Added some animations and hovers to the charts

10/16:

-Finished D3JS book
-Added cool D3JS things to the charts
-Better hovers on the performance chart

10/17:

-Made the performance chart example a collection of AJAX calls
-Used D3JS's algorithms to make a force-directed graph to represent the results of a tournament

10/18:

-Busy doing other things

10/19:

-Busy doing other things

10/20:

-Studied linear regression
-Built a structure to add a linear regression model to the performance chart using the normal equation
-Let users choose features
-Users can choose arbitrary data and use machine learning to get an expected performance

10/21:

-Fixed a bug, did some testing

10/22:

-Added a new method to the USCFWebsite class to count how many games someone played in a tournament
-Started building a chart to show rating/performance vs number of games played
-The USCF has blocked my localhost from accessing their website, so I'll have to develop from a VPN from now on.

10/23:

-Made some cosmetic changes

10/24:

-Made some more cosmetic changes
-Updating routing a bit

10/25 - 11/29:

-Took a month-long break
-Studied public speaking and developed foreign policy ideas for the U. Penn Model U.N. conference
-Visited Amelia in Massachusetts

11/30:

-Built a logistic regression class that implements a basic logistic regression algorithm.
-Added two logistic regression classifiers to the deltas page
-Predict what rating difference marks the point where a player has a better chance of winning/drawing by rating.

12/1:

-Built a linear regression class to implement a basic linear regression algorithm
-Added two linear regression lines, with features log(rating) and log^2(rating), to the performances page.
-Made various parts of the website work better and look better.

12/2:

-Found a problem with the logistic regression classifier.
-Spent a while exploring various solutions, couldn't find a consistent one, asked a question on StackOverflow.
-Made a lot of cosmetic changes to the site.

12/3:

-Updated what the description says on the homepage.

About

Rails app that mines the U.S. Chess Federation website and presents interesting and hopefully helpful data.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 56.3%
  • HTML 38.0%
  • CSS 4.0%
  • CoffeeScript 1.1%
  • JavaScript 0.6%