AI graders get top marks for scoring essay questions

AI graders get top marks for scoring essay questions  – New Scientist

It seems like this has some pros and cons. Text analysis has advanced to the point where grammar and structure can be scored reliably, but meaning still gets left out. I would think you could do even better than this by searching wikipedia for correlations in concepts. Or perhaps ironically, scanning online essay databases people use for cheating. This could have the duel benefit of scoring the meaning somewhat as well as searching for exact matches which could indicate cheating.

But even if these tools are used in combination with the teachers grading essays manually it should save them lots of time. All the grammar markups could be thrown up on the essay so the teacher doesn’t waste time correcting those and tries to asses the orthogonal problem of how good the student’s ideas or comprehension are.

Advertisements

The Science of Solving Problems vs Football

You should read this great article by PhD student Joon Chuah at the University of Florida about the baffling plan to save money at UF by taking the Computer Science department apart while increasing the budget for sports. He explains the place of computer science in a university and society wonderfully. I’m reminded of a great line from a commercial (for BASF apparently) that relates to what he’s trying to say about why Computer Science is important, the tagline goes:

We don’t make the products you use everyday; we make the products you use everyday better.

Computer science is sort of like that. We don’t work out how the universe works, or how genes cause disease, or how to build stronger materials or how to manage the economy; we work out how to turn questions into computations, how to solve huge, complex problems efficiently, we invent the tools that scientists, engineers, economists and anyone else can use to solve their problems.

Sometimes in CS we assume everyone else knows this, that everyone understands the deal we have. I think of CS as a kind of meta-science, like mathematics that investigates common patterns that underlie every other field of inquiry.  We look at the problem of how to solve problems, the answers we find usually turn out to be quite useful to someone, even if it’s not always clear at the beginning quite how they will be useful.  But sometimes I wonder if maybe everyone doesn’t really get it. Maybe the reason people ask you to fix your computer is they don’t think about the difference between engineering, information technology and computer science. Maybe, this is something the field of computer science needs to work on making more clear to everyone who doesn’t know us that well as a field.

But that’s a big problem to solve, one step at a time.

If you want to help reverse this move at UF support this student protest site and let people know how crazy you think this is.

Light Reading: Space Elevators, Brain Uploading and more

I’ve been meaning to get this blog more active, “at least one post a week” I tell myself. But every time I want to write something it always end up being about Canadian Politics. I often start jotting down ideas and then get too picky about being sure and I end up never writing anything. So new rule, if I can’t think of a single new idea to write about I’ll post a short list of interesting articles I read this week and what’s interesting about them:

Reality is all Math : this is a really interesting side topic of philosophy of Math and Science that I think about in my spare time. What is the nature of the universe in relation the mathematics. Why is math so good at describing the universe? Is the universe a computer or is there some beyond computation in the way physics behaves? This article has some interesting news on the latest thoughts from quantum physics about the relation of information and computation to the nature of the universe.

It sounds like there is debate about how central information theory is to explaining the equations of quantum physics. The opinions seem to range from important to central to quantum physics is nothing but information theory. I like that last one, but we’ll have to see what they find. It looks like there is no risk of physicists completely explaining everything before we get workable large scale quantum computers. I am glad to see there is more discussion about why the universe we live in adheres to quantum weirdness rather than just accepting the highly accurate math without any explanation.

Upcoming Technology and Your Job : Andrew Leigh wrote this piece on his experience being that rare thing, a politician who pays attention to science (we could sure use more of those). He presents his list of five technologies that could revolutionize politics in the near future, but really they are disruptive technologies that would widely affect everyone in society. It’s a bit fast and loose but they are genuinely important technologies to keep an eye on, always being wary to look for wide agreement before believing any claims. I’m in favour of anyone who brings up space elevators as a viable technology. Apparently NASA has made some significant advances to powering remote devices with laser that could reduce the weight load for a space elevator cable dramatically.

His discussion of Machine Intelligence is also worth thinking about even if it is jumping the gun a bit. We aren’t exactly near to creating self aware machines or being able to upload our minds into computers. But it is becoming possible to think about computers with the complexity of a human mind so it’s worth thinking through the implications.

He makes the point that replicated minds would be a threat to many people’s job’s as a single person who’ve very good at what they do could farm themselves out to available in many places at once. I suppose this is true but I think there is a much more relevant short term concern of people being made redundant by technological advances before we get to the point of copied human consciousness.

This is one of the topics I’m hoping to blog about here in the future: understanding scientific and technological change from the point of view of the job loss metric. What is the long term viability of your current career? Could it be done by machines or through crowdsourcing the skills of many people? As Leigh points out, typists and human computers in the early 20th Century had a career which would not exist a few decades later. Many factory workers have already found that what they do is fully automatable. Many educators are now starting to wonder if everything they do really needs a live human being present. What is the value of repeating lectures when you could have videos of the best teachers in the world which can be reused over and over?

This is a simple way to make discussion of new advances concrete for the layman and also very relevant. Most people don’t really care about quantum computers, machine vision or robotics. But if you explain how what’s going on in these fields of research could affect people’s jobs down the road, or the jobs which may or may not be viable for their children in the future, then they’ll be more interested in gaining a high level understanding.

If you have other good examples of ongoing research that could make entire jobs obsolete that people should be more aware of let me know.

Women in Computer Science : this article is couple weeks old but it’s worth a repost. Maria Klawe used to be department head at my alma mater doctorum (yes, I made up that phrase, if you know latin then correct me) which has a strong focus on CS Education and making it accessible to womens. She’s done some amazing things changing the CS program at Harvey Mudd to make it more accessible and focussed on solving problems rather than programming for its own sake. They have got their graduation number of female students up to 40% which is stratospheric in Computer Science program terms.

The Chomsky/Norvig — Classical/Statistical AI Brouhaha

This post summarizes an fascinating ongoing discussion on the state of Artificial Intelligence research. It should be accessible to technical and non-technical but I’ll add more to it as the discussion heats up.

If you’ve come from my other blog Pop the Stack then you are probably of a bit more political bent,  you might be surprised to learn that this is a debate between Noam Chomsky (yes, that Noam Chomsky, everyone has their own Noam Chomsky and they’re all the same person, sort of like the Queen) and the head of Google research Peter Norvig.

No, Google hasn’t declared an extrajudicial assassination a victory or undermined democracy in favour of the Military-Industrial complex. Chomsky is a very famous in linguistics and a founding father of some of the concepts of computer science and artificial intelligence. Peter Norvig on the other hand wrote the book on modern AI and Google is the epitome of modern AI research which treats intelligence as advanced pattern recognition and  statistical analysis of huge amounts of data under uncertainty. It seems that some people disagree that this is the way to building a full artificial intelligence. And the dust is still flying.

I’m going to put up links to the parts I see and add commentary later, I don’t think this is a going to die off soon, especially with a cluster of AI conferences coming over the next few months, so stay tuned.

The Beginning

It all began with the MIT’s Brains, Minds, and Machines symposium on May 3,-5 2011. The initial review of the event from MIT TechReview. Hopefully there will be a video at some point, doesn’t MIT videotape everything?

One theme of the discussion that has garnered attention is summarized here in this statement by Marvin Minsky:

“You might wonder why aren’t there any robots that you can send in to fix the Japanese reactors,” said Marvin Minsky, who pioneered neural networks in the 1950s and went on to make significant early advances in AI and robotics. “The answer is that there was a lot of progress in the 1960s and 1970s. Then something went wrong. [Today] you’ll find students excited over robots that play basketball or soccer or dance or make funny faces at you. [But] they’re not making them smarter.”

Norvig’s Response

Sufficed to say Minksy’s statement is not how a lot of researchers would characterize the current state or the recent advances in the field. Peter Norvig explains the problem much more clearly than I or most people could hope to.  If you aren’t familiar with Peter Norvig then he literally wrote The Book on modern AI with Stuart Russell and he is also the director of research at Google. Be sure to read the comment section there is some real intelligent debate there as well as a little bit of flaming.

Commentary

Here’s a great commentary by Mark Liberman on Norvig’s response (he mostly agrees) and support for his characterization of Chomsky’s views.

Discussion

Or perhaps you’d like to hear what philosophy of linguistics people think of the discussion?

Oh, Reddit has a great debate going on.

More to come…

Some Thoughts on Teaching Math and Computer Science

Two unrelated thoughts about teaching math and computer science that I came across today.

Tau the line

First, today is π day, celebrate mathematics and the beautiful wonder of nature!  But could π be…wrong? π of course, is the ratio of the diameter of a circle to this circumference and a snazzy number to throw out in Divinci Code-esque thriller novels.  It has an air of magic and mysticism which mathematics is actually full of.  But only a few such numbers and equations break through the collective consciousness to the general non mathematical public: π, E=MC²,  imaginary numbers

So that’s why tau manifesto is so interesting. It’s a very well written and entertaining piece about geometry, history and π. It lays out the case that π is actually not the most natural number to use for circles. It’s not that π i

s wrong, its that its awkward and that in fact τ, pronounce ‘tau’, is actually more natural.  τ is simply π times two.  This simple change actually simplifies a lot of what was tricky about advanced geometry in highschool and it has implication for how formulas are written in many fields of science given that π shows up in a lot of equations about the working of the universe.  So, after reading the whole thing, I’m a believer, if only for how it might make teaching geometry a little less painful for students. So while Pi Day is nice, on June 28 I’ll be celebrating Tau Day too. And like the author says, if you enjoyed the baked pie on Pi Day, you’ll love Tau Day, it has twice as much pie!

Big-O No!

The other teaching topic I thought of today is probably one someone has already done.  If you are a computer/physics/math nerd you owe it to yourself to be addicted to the great XKCD webcomic. It’s really got some great stuff and almost all the jokes or comments require some significant knowledge of math, programming, statistics or science.  When teaching computer science it is very easy to get separated from the reason we are teaching various theoretical concepts, algorithms and analysis techniques. It usually can be grounded down to preparing students to be better programmers or modellers or to give them the grounding needed to understand other courses later on where they can get computers to do truly cool things.  However, it’s often difficult to make that connection for students before they actually know how those cool things work

But how about these learning goals? They are fun, easy to evaluate and provide clear goals to students in that they are total gibberish before the course and funny jokes afterwards.

1) After successfully completing this course on optimization you will…understand why this is funny.

And be able to explain why its funny, or at least supposed to be funny.  We won’t deduct marks if you don’t laugh, but we will judge you 🙂

2) After this course on complexity theory you will understand why the following comics are funny:

3) Unix Tools:

I’m there are lots more. If you can think of one put it in the comments <XKCDNumber> : <Course Topic>

%d bloggers like this: