Monday, October 03, 2011

Tarski's World: When and Where Grade Grinder's Cheat Prevention Fails

For my semantics class, we have been using a "book" called Tarski's World to quickly learn, interactively, predicate logic. "Book" is in quotes because it's mostly a suite of software that is sold with the book for using predicate logic within a simplified world.

My feelings are mixed. Some of the exercises are interesting; others are tedious. What's come to bug me the most has been the grading software. It's very bad at giving decent advice, which, after banging your head against a computer keyboard for a while, is exactly what you need.

Sure, this process is frustrating, but in the end, I've learned a lot of predicate logic. On the contrary, what bothers me most is the false accusations the grading software generates--in particular, the claim that my solutions are "similar" to someone else's, effectively the suggestion that I'm cheating. As an academic, this is really insulting, and something up with which I will not put.

According to the link in the "you're a cheater" message, Tarski's World saves a 12-digit timestamp with every save. It claims the odds of two students saving a file at the same time is the same as winning the California lottery. This is accurate. When you upload a file, it looks at the other sets of timestamps that the file was saved with. If there are overlapping sets of timestamps at any point, the files are declared "similar."

However, the designers overlooked one thing. Some exercises require that you start with a particular world that Tarski's World has provided. Whoever wrote these files saved them at some point, and therefore, they are guaranteed to be similar by the above definition--the files originated at the same point in time and are therefore similar.

To verify this hypothesis, I opened up a Tarski's World world file. To my surprise, the file was nothing but plain text--no attempt of any kind to encrypt or obfuscate the file and timestamps. The dates immediately stood out as Unix time--that is, the number of 60ths of a second that have happened since January 1, 1970. However, using these whole numbers yields gigantic results--the standard time for one of the timestamps put me in the year 45000, which was clearly not the case.

I attempted to shorten the timestamps to find the Unix time within them; it's merely the first ten digits. The last two, I assume, are some kind of checksum. I tinkered with reverse engineering them, for fun, but I don't have that kind of time right now. More importantly, I knew where to find the evidence I was looking for and started piecing it together.

A "C" or a "D" precludes each timestamp. "C" stamps preclude "D" stamps. Sometimes, there are multiple instances of "D." What this looked like to me was "C" is some how a computer ID--maybe the time when someone installed Tarski's World on that particular computer. Checking one of my own C's, this turned out not to be the case, and it's probably the time when the file is opened. Either way, the precise meaning of the values is irrelevant. What matters is that I know the file was being tweaked around the time of that stamp.

I used Python's datetime module to handle the Unix times. The "D" and "C" values in the latter part of one of the files pointed at 2011. Checking out the "C" in one of the files, the Unix time portion of the stamp pointed right at 2004. This was a red flag and the cause of my commrades and I's woes.

So, to correct the Openproof Courseware website, it is not extremely unlikely that two files will be created at the same time. If those two files are meant to originate from the same file, they are guaranteed, by your definition, to be similar. Please rectify this, or at least mention the fact that "since some exercises were created by us you can relax blah blah blah..." instead of assuming that we're cheating because your work is sloppy. Then, I'll quit blaming you when my sentences are sloppy.

Also, you might want to look into not storing the timestamps in plain text. It would be incredibly easy to manipulate them, you could literally just... well, I won't get into that here. Someone who knows the files are plain text knows too much already.


Post a Comment

<< Home