programming and human factors
by Jeff Atwood
A bunch of people have linked to this academic paper, which proposes a way to separate programming sheep from non-programming goats in computer science classes – long before the students have ever touched a program or a programming language:
All teachers of programming find that their results display a 'double hump'. It is as if there are two populations: those who can [program], and those who cannot [program], each with its own independent bell curve. Almost all research into programming teaching and learning have concentrated on teaching: change the language, change the application area, use an IDE and work on motivation. None of it works, and the double hump persists. We have a test which picks out the population that can program, before the course begins. We can pick apart the double hump. You probably don't believe this, but you will after you hear the talk. We don't know exactly how/why it works, but we have some good theories.
I wasn't aware that the dichotomy between programmers and non-programmers was so pronounced at this early stage. Dan Bricklin touched on this topic in his essay, Why Johnny Can't Program. But evidently it's common knowledge amongst those who teach computer science:
Despite the enormous changes which have taken place since electronic computing was invented in the 1950s, some things remain stubbornly the same. In particular, most people can't learn to program: between 30% and 60% of every university computer science department's intake fail the first programming course. Experienced teachers are weary but never oblivious of this fact; brighteyed beginners who believe that the old ones must have been doing it wrong learn the truth from bitter experience; and so it has been for almost two generations, ever since the subject began in the 1960s.
You may think the test they're proposing to determine programming aptitude is complex, but it's not. Here's question one, verbatim:
Read the following statements and tick the box next to the correct answer. int a = 10; int b = 20; a = b; The new values of a and b are: [ ] a = 20 b = 0 [ ] a = 20 b = 20 [ ] a = 0 b = 10 [ ] a = 10 b = 10 [ ] a = 30 b = 20 [ ] a = 30 b = 0 [ ] a = 10 b = 30 [ ] a = 0 b = 30 [ ] a = 10 b = 20 [ ] a = 20 b = 10
This test seems trivial to professional programmers, but remember, it's intended for students who have never looked at a line of code in their lives. The other 12 questions are all variations on the same assignment theme.
The authors of the paper posit that the primary hurdles in computer science are..
.. in that order. Thus, we start by testing the very first hurdle novice programmers will encounter: assignment. The test results divided the students cleanly into three groups:
The test was administered twice; once at the beginning, before any instruction at all, and again after three weeks of class. The striking thing is that there was virtually no movement at all between the groups from the first to second test. Either you had a consistent model in your mind immediately upon first exposure to assignment, the first hurdle in programming – or else you never developed one!
The authors found an extremely high level of correlation between success at programming and forming a consistent mental model:
Clearly, Dehnahdi's test is not a perfect divider of programming sheep from non-programming goats. Nevertheless, if it were used as an admissions barrier, and only those who scored consistently were admitted, the pass/fail statistics would be transformed. In the total population 32 out of 61 (52%) failed; in the first-test consistent group only 6 out of 27 (22%). We believe that we can claim that we have a predictive test which can be taken prior to the course to determine, with a very high degree of accuracy, which students will be successful. This is, so far as we are aware, the first test to be able to claim any degree of predictive success.
I highly recommend reading through the draft paper (pdf), which was remarkably entertaining for what I thought was going to be a dry, academic paper. Instead, it reads like a blog entry. It's filled with interesting insights like this one:
It has taken us some time to dare to believe in our own results. It now seems to us, although we are aware that at this point we do not have sufficient data, and so it must remain a speculation, that what distinguishes the three groups in the first test is their different attitudes to meaninglessness.Formal logical proofs, and therefore programs – formal logical proofs that particular computations are possible, expressed in a formal system called a programming language – are utterly meaningless. To write a computer program you have to come to terms with this, to accept that whatever you might want the program to mean, the machine will blindly follow its meaningless rules and come to some meaningless conclusion. In the test the consistent group showed a pre-acceptance of this fact: they are capable of seeing mathematical calculation problems in terms of rules, and can follow those rules wheresoever they may lead. The inconsistent group, on the other hand, looks for meaning where it is not. The blank group knows that it is looking at meaninglessness, and refuses to deal with it.
Everyone should know how to use a computer, but not everyone needs to be a programmer. But it's still a little disturbing that the act of programming seems literally unteachable to a sizable subset of incoming computer science students. Evidently not everyone is as fascinated by meaningless rules and meaningless conclusions as we are; I can't imagine why not.
* which I hope to master sometime between now and my death
I have not seen it mentioned, but the US Gov has known about this since the 60's. They still use the same test for programmers, the ADPE ( Automated Data Processing Exam ). It consists of four sections, word association, numerical sequencing, memory, and dimensional correlation.
The failure rate for the tech training was around 80-90% as well.
SSgt on February 27, 2007 4:53 AMI think the test should be in some meta-language pseudo-code because intelligent people good at algebra and math but have never touched a programming language in their lives doesn't have to know that in something called C programming language "=" is an assignment operation. Maybe something like.
BEGIN PROGRAM
x - 10
y - 20
x - b
END PROGRAM
What are the values of x and y at the end of the PROGRAM.
AndresVia on February 27, 2007 4:58 AMPeople are going to lynch me for this, but those inventing pseudo-language are a bit silly. There's already a perfect language for this: COBOL
Granted, my COBOL is incredibly rusty, so I probably can't write code that compiles here, and it would be best to leave the division separators off, or, in fact, the whole data division, but:
MOVE 10 TO A
MOVE 20 TO B
MOVE B TO A
With a note that 'MOVE leaves the first value intact' if needed. (Or make it 'COPY' instead.)
In fact, this is basically what COBOL was designed for...having non-programmer readable programming language, and the reason it's such a joke is that, as this article points out, that doesn't actually work. Certain people simple cannot parse programming languages, no matter how simple the languages are.
It's not a matter of intelligence, I've known very very smart people who passed programming course by 'faking it' by copying and pasting code they knew worked, and I've known pretty dumb-in-other-respects people who could write code just fine.
'Sequence' is a big one that programmers who have not tried to teach simply won't believe, but there are people who simply do not order statements in their mind when evaluating them. They know to do it, they just look at blocks of code as a gestalt, and see a=10, and that's the right answer. This isn't due to intelligence or even knowledge level, it's something like dyslexia. Just like people with dyslexia can manually go letter by letter to get a word, they can manually run them in order, but that's very hard when loops or recursion or functions are involved.
There are, indeed, entire groups of people who *cannot* be programmers, will *never* be programmers, and really should be encouraged into some other line of work at the start of their programming education. Just like someone with dyslexia probably shouldn't be a copy-editor.
Now, if someone wants to star in their own made-for-TV movie about how they overcame hardship to do what they loved to do, that's one thing, but honestly, most of the people just heard their was a lot of money in computers, and should be firmly encouraged into law or something else where they aren't going to always be struggling just to understand what's going on.
Hrm. No preview. Well, let's hope this is right.
DavidTC on February 27, 2007 10:37 AMMr. Bauman,
I agree with what you and your wife say, however, I don't think you realize why it's not happening: Departments need numbers of students to get funding, building space, faculty, staff, etc. So they suck in every student they can and help them as much as they feel like whether they have a shot or not.
I think it's reticent of one of the biggest problems with bureaucracy: It puts money where it's needed in strange ways that have nothing to do with the the industry and everything to do with the number of people and their ability to fill out useless forms. And it rewards bad spending!
It's possible that not every school has this glaring problem and that some may actually deeply care about the students well being and education, but I'd bet most are more like this.
Also, to other commenters: You're not reading the study. Their conclusion was that the group that fairs the best is the group who finds a consistent model (whether it's C like or not) and sticks with it. They also found that not much changes after they understand the syntax.
And if you really want a familiar syntax you should use an algebraic syntax:
a=5
b=6
a=b
Unfortunately, this still doesn't make sense as algebra is formed without statements. This is more similar to formal proof language where you setup some variables early on (except you don't assign numbers, obviously).
However, back to your idea Mr. Bauman. I think the way to do this would be private industry placement exams taken between high school and your sophomore year of college (it doesn't make much difference if you take the wrong major in your freshman year, especially if you're in the right college -- as in college inside a university system).
So students could pay for exams that would say if they have aptitude for a given field. You could add a general aptitude test to give them some direction, but most students would likely be able to guess based on their personal history.
Charge $50 for a test like this and give nice results. I imagine you could get quite a few students on board! You might need to hire overpriced PhD's to "back" your tests though, in which case you can charge $150 for the exams.
I think it's interesting that so many respondents seem to see the mere possibility of a test that *may* reliably indicate programming talent or the lack thereof as being somehow threatening or pejorative. Have we really come to the place where an individual is lessened somehow if they don't possess all talents? I guess I should really reexamine my value to society since I'm a pretty good programmer but I can't carry a tune to save my life?
One of the things that intrigued me about the study is that it reminded me of discussions I've had with my wife. She is a top-notch educator with 30+ years experience and she has often mentioned that she thinks our institutions of higher learning should have an effective way to detect and discourage education majors who have absolutely no talent for the vocation. Her reasoning is that it is a disservice to the college students to allow them to spend so much of their time and money pursuing something they will never excel at and that it is also a disservice to the future students who wind up in their classrooms. I think the same reasoning can be applied to CS majors who have no talent for the trade. They suffer because they don't measure up in their jobs and those of us who come after them suffer when we spend our time cleaning up their mistakes.
I think it's important to strive for excellence in all areas but there are some areas where individuals will never achieve excellence because those individuals are not well suited for that particular type of endeavor. That doesn't mean we should prohibit them from pursuing those goals if they're set on doing so but it might be helpful to them to know that it's likely to be an uphill climb; we can leave certainly leave the final decision to them.
My take on it is that the Teacher is inexperienced. I've not seen that sort of results in the night classes I taught for Harnell college in King City, CA and surrounding areas. At least after the first 3 or 4 years of teaching.
I believe that most "intro to computer" classes are taught by graduate students. As a group they tend to assume that Everybody already knows the basics. By the time they grasp how to teach the novice, if they ever do, they’ve moved on and a new inexperienced teacher takes their place.
Bob Durtschi
Bob Durtschi on March 1, 2007 10:17 AMI think all the comments to the effect that "a = b is bad syntax" miss the point entirely. The test isn't about whether this represents good syntax or not - it's about whether the student will do well in CS or not. You can argue all you want about this syntax, but if this test is a good predictor, that's all that counts. If it's bad syntax, then apparently an intuitive understanding of this very bad syntax is a good predictor of success in CS - the fact that it's bad syntax is entirely off the point.
For the record, I never thought it was bad syntax. Any math person understands things like "let x = 15. Then x+5 is 20. Now let x=20. Then x+5 is 25". Nobody gets confused and says "Wait! You just said x=15 which means it can't be 20!". On the other hand "x=20+5. Then x=25. Oh, wait, now x=30." is confusing. It just depends on whether you're using "=" as a declarative statement about the permanent and necessary value of x or whether you're using it as a verb to consider x to have a value of 15 for the moment. Both are used in standard math without confusion so I never really understood people's confusion about the verb sort of usage in languages. Some languages us "let" or "set" to clarify, which I guess makes it a bit clearer that the verb form is being used, but it seems a bit unnecessary to me.
I like the guy's comments about meaninglessness. I remember trying to teach my mom a bit of programming. If I would write down "age=20" she would immediately pipe in "no, that's not my age!". I'd try to explain that it's just a name but no amount of explaining would do. Finally I'd change it to "x=20" in an attempt to divorce it from the meaning she was assigning it. At that point she'd ask what "x" was. I'd have to tell her we were using it to represent her age and we'd be back to "but I'm not 20!". She wasn't able to accept that "x" or "age" is meaningless in the context of the program and had to know what it "stood for" at which point all sorts of real life connotations would flood in making it impossible for her to understand what was going on.
This problem has puzzled me for years. I learned programming myself, with able tutoring from a friend. In turn, I've tried to teach others, some smarter than me, some not. There were some that picked up the concepts and then built on them. Others had great difficulty with the basic principles - and even when they'd gotten them, could not seem to carry forward with what they'd just learned.
I've always wondered at this difference. Looking back at my own history, I've also been curious why certain populations thrived in the school environment but never seemed to really grasp complex subjects - and vice-versa, some of the best minds felt totally out of place and suffered during their school career.
This essay really opened my eyes, and explained so much of why I felt out of place in school:
http://www.reciprocality.org/Reciprocality/r0/Day1.html
(if the comments don't allow links, google the words
mappers,packers,software and look for the link "Thinking about Thinking".
I don't attach a value judgement to mapper vs. packer, each have their place; and of course no one is all one or the other, it's a continuum. Still, it's a useful mental model when thinking about the issues this blog post has raised.
I have the advantage that I know Richard Bornat, the author of this paper. A lot of the comments on him here make completely false assumptions. In particular, he wrote his own programming text book many years ago which was quite well-known in its time, and he was an expert in functional programming, which he taught. So the idea that he's fixated on imperative programming, or just some dumb person trying to teach from a prescribed textbook is wrong, wrong, wrong.
Richard Bornat was Head of Department at the university Computer Science department where I teach when I joined it as a new lecturer (he's now moved on to another university for various reasons). He taught introductory programming, and I taught it for several years after him.
Every year we used to get into fits of depression - and I'm seeing it with the new guys who are teaching intro programming now - just WHY can't the students get it? We've tried everything - functional first, objects first, back to imperative first, use an IDE, don't use an IDE, use a language they use in industry, use a simple teaching language etc etc - and the SAME pattern still happens, a huge proportion of those on the intro programming course either fail or ought to fail (sometimes we just have to be lenient on the passing requirements since if we failed too many we might lose our jobs).
What's more, if you look at the big conferences on teaching Computer Science, you'll find people saying the same thing the world over. No-one seems to have hit on a way of teaching intro programming so that most people who take the course pick it up and end up competent in it. If there was such a way, and Bornat and me and everyone else who teaches intro programming are just bad teachers, wouldn't that way have become apparent by now?
Bornat's also a political lefty, the sort of person who naturally recoils from the idea that there are innate born abilities, with people naturally divided into categories. He's certainly not saying this because it suits his ideology. No, he's saying it because it reflects his long experience of trying to teach programming. It seems there really is a divide between those who "get" programming and those who don't.
The test he's using here is acknowledged to be really simplistic, an observation which might be taken further, but with statistics so clear it's obvious something is happening. If you read the paper, it's clear it's not a test on whether you've previously encountered C-style assignment statements as many of the detractors have suggested (did they actually read it before commenting?)
Matthew Huntbach on March 16, 2007 7:21 AMTwo interpretations, not necessarily consistent:
1 - Only one person even hinted at the fact that those who best perceive abstraction would vary their answers. Test takers presumably aim for the most correct answers. There is no sound reason to assign a particular function to the '=' operator (its resemblance to algebraic and programming usage is irrelevant and misleading). Thus one would have to be a gambler (most programmers are not) or a sheep to consistently assign it the same function. An actual rational actor would define the most likely interpretations - say, it either shifts the value right to left or vice-versa - and alternate between them, shooting for the highest possible score. The sheep, however, finds one answer that works, and sticks to it. Later, they will be told which one is correct and stick to that. They make good byte pushers because they uphold the banner of standards and conformity and ensure that everything is always "ready for the enterprise." Goats tend to wander off and explore flights of fancy (variables can not be assigned as values to other variables, so the value of the variable a is now the string "b".) Requires more intelligence and a more thorough insight into abstraction and the arbitrariness of formal systems, but they never get any work done.
2 - OK, so those who were consistent in their (incorrect) model of '=' over the first few WEEKS of this class did better than those who perceived the need for consistent interpretation, but changed their interpretation after they presumably learned (or got an idea of) how '=' works in nearly all commonly used programming languages?
NOTE: I'm perfectly ready to accept that there are people who can and can't program. And I thought this study was interesting, but that the reasoning and interpretation need some serious tweaking. As a sociological study, it seems to be about as rigorously and scientifically constructed as most of those done by sociologists. Now, you CS and Engineering guys wander down to the Sociology department and tell me if you're comfortable with that comparison.
Alex Gretlein on March 19, 2007 7:04 AMI'm with the people who vote that the teachers sound pretty bad. IMHO the entire metaphor for dealing with concepts on computers is changing, and more control is being given to users.
I have found some programmers to have fluid powers of conceptualization and good powers of communication. Others are fairly rigid. Luckily, there are programming languages and jobs for all different types of people.
I, on the other hand, am in the "oh jeez, that makes my pupils dilate and I must sleep immediately" category.
Kate on March 19, 2007 11:03 AMThis test was refuted by a trio of Australian computer science researchers. They found no correlation between the "consistent" group and the group that passed their intro-to-CS course.
anon on April 4, 2007 11:05 AMWhile I understand the motivation, this test is just badly designed.
After reading the paper I find that the results simply needed a lot of guesswork to interpret, even without considering that they had problems mapping the results from the first administration to those from the second administration, i.e, some subjects couldn't remember the pseudonames they had used for the first administration.
Besides, the Java model of assignment is simply not intuitive.
Maybe if the background of the experimenters where in ML, Python, Haskell or [insert name of well-designed language here], the results would be interpreted differently (not to mention the test would be formulated differently). Or at least maybe they would have the sense to administer the test properly so they could then actually interpret the results.
Perhaps they were on a deadline to publish something. I might be offbase here but Saeed Dehnadi is a graduate student at Middlesex. This might be his first crack at the problem. Don't take it as gospel.
His mentor's (Richard Bornat), background is in Java so that might contribute slightly to the bias in the test. Even though he appears to be proficient at it.
Also, and I'm nitpicking now, Saeed's webpage has no title. This might be oversight or might be choice, but it suggests one of two possibilities, either he pays less attention to detail than he should or he lacks creativity. Admittedly, it could also mean he has little time for such things given all the incredible code he writes, you know, with lots of formal content.
Okay, I was being sarcastic, but I agree with Andrew Shebanow above, the last comment on page 16 of the paper is extremely arrogant and elitist. I mean, how would they stack up next to a software engineer at Googleplex?
FernandoGonzalez on April 7, 2007 8:29 AMPhilosophy is similar in many respects. Many people can argue, but few understand the rules of logic to form a coherent argument.
In philosophy, much like programming, you have to have the intellectual integrity to succumb to a logical argument. An argument is composed of 1: givens(axioms) 2: Logic. In order to refute an argument you have to 1: disagree with the Axioms or 2 Find fault in the logic. If you agree with the axioms and cannot find fault in the logic, you must always agree with the conclusion. No matter what. Its hard to grok at first, much like understanding that the computer has no clue what you MEANT to do. It will only do what you SAID to do.
Philosophy students have a similar problem understanding an axiom vs. logical operand in their first year.
brian on April 26, 2007 11:38 AMI find it surprising that only two commentators mentioned that Intro to CS is usually a weeder course. Do they really *want* to teach the kids who don't know any programming, or have they set up the course (and this test) as a filter to find those who don't need an Intro to CS class.
Eli Gottlieb on May 12, 2007 10:00 AMI had a teacher who was once in charge of grading the AP computer science test back when they still used Visual Basic. He ran statistics between correct answers on the questions and between the actual score the student got.
He found a few questions that, had they asked only those, could have scored students with something like 80-90% accuracy. The question with the highest correlation was this (sorry, I only know VB.NET syntax if this is off):
What is the value of a Boolean x after the following line of code?
x = (x = false)
1) x is always true
2) x is always false
3) x flips
4) x stays the same
5) None of the above
Basically, if a test-taker got this question right, s/he succeeded on the exam, and if s/he got this question wrong, they didn't do well.
Michael on May 29, 2007 5:27 AMI think it's a similar number for mathematics. Some people just can't get it.
I went to a tech college for computers and remember all the moaning in algebra class. `It's worthless and a shouldn't be required computers come with a calculator. When am I ever going to need this junk.`
These where the same people who where lucky to do very poorly in the programming classes.
Matt on May 30, 2007 10:20 AMbefore i started learning to program, i would have been in the 8% who refused to answer. i would have been appalled at this test, would have stopped execution of the task right at seeing the string "int".
my logic just refuses to go further if there are unclear details. and maybe that's why only assembly could lure me into trying to learn programming. no other language could. i'm serious. :)
so, what does the paper predict about the 8%? i don't see mention of that?
now, i'm still just learning, started it recently, can't say if i'm going to be a good programmer or not (i love assembly but i need to learn some other languages obviously to actually make serious programs), but now i'm at least able and willing to answer this test question :) (in the way a c++ programmer would.)
so it isn't true a (GOOD) course can't change some students. i don't like the generalization drawn from this test. but i do agree that you need some abilities to become a good programmer. i used to help classmates with math.. they were really dumb, i can't imagine them learning programming. not saying there is a very strong correlation between math and programming skills, friend hates math but can do cool stuff in programming.
oh and i said GOOD course! i did have a programming course in elementary school and absolutely didn't learn anything from it. for the same reason i would have been in the 8%. we were just told to copy the program lines and then press F5 or whatever to run the program (it was qbasic and pascal in the old times). no chance for me there. and i felt stupid too, i saw some guys were playing with the computers in the class in a cool way, and i had no clue, further discouraging me, preventing me from looking at programming, for about 10 years! so yes social circumstances and teaching quality do matter too..
felix, you're right, you need to give time for practice (reading with little practice won't ever teach you to program). that got me thinking.. the paper on this subject could have gone wrong in one aspect: how much time did they give for students to get into programming / get comfortable with it? then lets assume the ones who'd been able to answer are the ones who'd already had some experience in programming.
(the other aspect is the quality of the teaching :) )
but, of course, one thing might eliminate this possibility: they did accept the solutions that were consistent even if not giving the "correct" solution. that doesn't require knowing any language, just having a strong enough sense of logic.
cmon_ on June 24, 2007 6:07 AMNew programming students need time to learn to program, and master programming very well. Generally many get a lot of programming difficulties at the start of their course. Many are discouraged, and are afraid of subjects/modules that deal a lot with programming. Teaching programming is also a problem in many universities.
Programming should be for people good at mathematics. People who get good grades at math can understand programming well (even if it takes some time). I have also realised that some IT students can never understand programming. These are generally people not good/skilled in logics/maths.
Felix on June 24, 2007 11:46 AMThis paper will never be published, seeing as it spends more time on petty insults directed at the people who will have to review it, than describing its findings.
What exactly is meant by a "meaningless" statement in formal logic? They're not meaningless statements; in fact, they're statements with a very definite meaning, assuming they're defined. The computer blindly executes statements, it is a machine after all, but the way we approach the logic is not meaninglessly. x y, say over integers, has a meaning, otherwise, we would have no idea what consequences it has or what it could imply in a particular context. When you read a computer program, x y has a very definite meaning. Tell a mathematician that proofs are meaningless, and you're likely to get a whack in the head with something heavy and potentially sharp.
How often does a programmer sit there frustrated or wondering why the hell something he intended to express in a program isn't happening as expected? That is after all what a bug often is; a gap between the behavior intended and the behavior experienced. There are a million reasons why this could happen, many of which boil down to reasoning in absence of some knowledge (either a simply oversight, or lack of knowledge of an error elsewhere, or a misunderstanding). And rest assured, as we should all know, programmers are first and foremost excellent creators of bugs!
If I understand correctly, a claim is being made here that there are people who are incapable of understanding that the intended behavior can somehow not be isomorphic to the behavior observed. I'm not sure I understand, much less believe that statement. Reasoning used in programming is not somehow alien to people. Matthias Felleisen, previously of Rice University and now of Northeastern University, would probably go into one of his famous military-strength fits if he read this "paper". Programming is a discipline, and like any discipline, it requires discipline. A particular mindset is necessary to work effectively, and mindsets can change.
A more trivial claim perhaps is being made that people don't understand why they need to, say, add a ; after a statement, or end an expression with a ). I find this claim absurd, especially since syntax generally implies something: the ; tells the computer that the statement is ended, just as the parenthesis make something unambiguous. If there are programmers that think this is some concept that is inaccessible to some people, then I really question their ability to reason abstractly and their ability to form a big picture. Or, they must really suck at teaching something so basic from algebra.
Correlation is not causation. The conclusion is only as good as the premise. Garbage in, garbage out.
Bob the Chef on July 20, 2007 10:33 AMI think Bob the Chef pointed out something important when learning to program.
ability to reason abstractly and their ability to form a big picture
Also the ability to take rational/business requirements and transform them into, mostly, concrete logic. For my last semester in college I was able to take a graduate course to fulfill my credit requirements. I was appalled at the number of people in that class that couldn't abstract out the core process/idea behind a group of functions or processes.
One of our projects was to create a class library(dll) that would encapsulate the logic for a tic-tac-toe game. When discussing it in class it was amazing the number people that would want to print to console or pop up a message box when an error occurred. Just as scary were the number of people that would test the same things in the UI and logic layer, ie testing for a tie in the UI layer when the logic layer would "tell" you if there is a tie, or at least it should of from the specs given to us.
Just as disheartening when I was asked to help debug some classmates programs I found huge 15+ if blocks for testing if there was a winner or a tie. Why not just loop through all the rows and call one method that checks if that row is a winner. It was almost as if they didn't understand the concept of abstraction or rather didn't understand how to put into practice the concept of abstraction.
It worries me when I, as an undergraduate, feel smarter than 90% of a graduate course class. They have already been through what I am going through shouldn't they be better/smarter than me or not there at all? I hope that didn't sound to cocky I like to think I'm a little more humble than that.
I may not agree with what all of the paper says and some of the conclusions it comes to seem to be coming from incomplete/skewed data, but I do think that there are some people that just don't have the ability to wrap their heads around the concepts of programming like abstraction and polymorphism and the like, or the ability to break things down into logical steps.
Matt on July 20, 2007 12:45 PMI read this before, it struck me as interesting. I came back to it and now it seems less interesting. It looks like yet another example of academics thinking that they are the gatekeepers to the world. That it is up to them to open the minds of the little people (those who are good enough to deserve their attention at any rate.)
I've gotten A's in some programming courses and F's in other ones. All of my teachers were quite sure of their own ideas about goto and comments and pointers and OO -- and I have seen since the same opinions and other opinions and opposite opinions in many different peoples mouths. All of them think they are saying something.
Everything I have learned I have learned on my own, just like English, just like sex. Stop taking yourselves so damn seriously -- you're just baby-sitters. Especially the college professors.
James D. Newman on September 3, 2007 2:04 AMWhat's the correct answer, anyhow? I racked my brain and all I could come up with was none of the above.
Brian on October 24, 2007 7:27 AMthe answer is 20/20 haha... :)
this is nice website! Thanks!
You know, I presented this to one of my programmer friends in this form:
a = 10;
b = 20;
a = b;
And he said it doesn't make any sense. I couldn't believe it until he told me that he interpreted it as a math problem and in math, that does make no sense.
Bad test design if you ask me.
If you had something like:
a - 10
b - 20
a - b
I think the test results would of been significantly different
name on November 15, 2007 9:54 AMBrian,
I'm not sure which question you were talking about, but if you were wanting to know the answer to the "x = (x = false)", then it is that x flips. If x was true before the statement, then true does not equal false. This makes the evaluation of the () part false. So, x gets assigned false. If x were false, then false DOES equal false, so x becomes true. Hope that helped!!
Everyone else,
But just to weigh in on the VERY old thread I stumbled upon here, I have a hard time going one way or the other. We all know those individuals who we have the damnedest time conveying the basics to. Those are often people who want as little to do with computers as possible. However, when it came to school, I think it is very clear to almost ANYONE who was a student that these two camps existed. But WHY did they exist? That is what the paper is trying to figure out.
There is one observation that I would like to make. Most of the students who did very well with little effort did so because they did much of this on their own. I know that this argument has been mentioned, but there is one correlation to this that i'm not sure has been mentioned. Students who are learning in his or her free time obviously have a greater passion for the subject. In the age of the internet, there is nothing that an individual cannot research on his or her own. If the student doesn't share drive of the better performing student, then maybe THAT is the reason the don't perform as well.
Basically, my point is the cowardly "agree with both sides" way of looking at it. lol. So maybe the one side is right that the test could potentially predict success. And maybe the other is correct that the test cannot predict who is CAPABLE of learning the material.
David on November 18, 2007 3:39 AMTo
Matt on July 20, 2007 11:45 AM
Since you can without a moment of thought, lash with little or no respect for those who are slowed in someway, i'll be seriously frank.
Im just begaining to step into the world of java programming and seeing your posts just irks me.
I've been thinking, a friend of mine have a "wired brain" just for programming, astonishingly, always the first to complete any homework within the given time frame.
And i noticed his other areas of study were completely wrecked, engineering maths etc.
So do i look down on his bad areas? Rather i look at how good he tries to coach us into this field.
Since i'm posting just to "bash", Matt you did best search yourself within and think over how much people you've insulted with those so called "harmless" comments. In the business model, i assume theres such a term called "win win situation".
Rather than posting how bad those people are, you could've help them those with sincere learning attitude but as you put it, zero ability to "pain abstract" out of nothing.
And for those who read and got to read by chance
A man with a beard with grey hair is my father, does that make all those with similar traits my god-father?
As said, there will be those with extraordinary gifted talents as well as those with it, struggled yet barely manage things.
However the latter always sees through hardwork.
So please,
if
either the teacher is willing to teach or the student loves programming.
Else
everything will be in Loop.
One does not learn how to walk, but one mimics.
Isn't that how most of us learn stuff? Memorizing the pattern until our internal minds sorts the pattern out to more logical thinking.
Therefore, if anyone here wana change the "programming world"
start with
Any Random Kindness, help those around.
I don't have time for this crap (to busy programming) so I'll get straight to the point.
1. Some one with no prior programming experience is not going to know that = is an assignment operator and it is not fair to assume that they will infer it from the context of the question.
2. The question is further flawed by the use of the assignment operator (=) in the answers where a comparison operator (==), or plain English, could have been used. Instead the author is relying on the subjects prior experience with mathematical problems in which the = operator is equivalent to a comparison operator (==). Therefore, in the context of this question, = has two alternative meanings.
However it is likely that those students with an interest and/or experience in programming will have knowledge of both usages and successfully navigate the semantic of the question.
Furthermore I think it's a reasonable assumption that those students with interest and experience are more likely to be successful.
HENCE THE CORRELATION BETWEEN STUDENTS WHO DID WELL ON THIS TEST AND THOSE WHO DID WELL IN THE CLASS.
In short your question is flawed and your science is crap. I believe this revised question would yield a surprising increase in the number of students able to answer correctly.
--
In the C programming language the = operator copies the value of one variable into another. Read the following statements and tick the box next to the correct answer.
int a = 10;
int b = 20;
a = b;
The new values of a and b are:
20 and 0 respectively
20 and 20 respectively
0 and 10 respectively
10 and 10 respectively
30 and 20 respectively
30 and 0 respectively
10 and 30 respectively
0 and 30 respectively
10 and 20 respectively
20 and 10 respectively
--
I can be contacted at the following email address memeticvirus [insert at here] gmail [insert dot here] com
zeropointo
Actually I lied, I deleted that email account...
zeropointo on December 27, 2007 8:55 AMI only have my own experiences to relate, which is that it took me a long time -- some might say an embarrassingly long time -- to understand concepts like recursion and indirection.
I think this research confuses aptitude with ability in an almost offensive way.
Joe Chung on December 27, 2007 9:53 AMThis is just an observation. It isn't tested significantly, and the effects of the environment are not checked rigorously. Please do lighten up and recognize this for what it is, just an observation that the first three weeks of an introductory CS class didn't manage to change the students' behaviour much. There is a lot of reasons that could cause this:
1) People are indeed goats and sheep, no way to help it.
2) Students don't attend the first few classes in any measurable way.
3) We are not teaching people properly.
4) Students do not trust their brains, they trust the test-passing techniques they were conditioned for throughout their education.
To separate these causes (show which ones are just theories and which are the cold hard reality), one would need to do plenty of work. Probably it's already happening, and the study was intended solely to indicate that the authors are working on this.
My personal experience is that everybody who can pass a university entry exam can learn to do practically any mental task adequately. The problem is only in how long will it take and how much tutoring it will require, and whether it's a good idea to sink more resources into bringing absolutely everybody up to the same level rather than separate those who seem to be more able early on (creating a risk of misidentifying the students' abilities).
Sex has nothing to do with the affinity to programming, social stereotypes and the little biological details of having a slightly different window of opportunity to reproduce and the level of commitment to reproduction ("women are pigs, men are chickens", if you wish) are what I think cause most of the apparent gender inequality in the industry.
David:
As far as I can tell, you are analyzing x=(x==false) flip x instead of x=(x=false) set x to false.
Tepsifles on January 21, 2008 2:22 AMWOW! A new site to troll!! :) Pretty good posts even though I had to skip some of the back log. Yes, it feels like some kind of matrix choice. Some people just like different color pills! There, is ofcourse, loads of scientific data to uncover with respect to all our predispositions, to summarize what exactly humans are and if we will ever be able to really classify human nature even if the taxidermy spawns iterations that out number the population.
Ezrad Lionel on February 10, 2008 7:14 AMI teach at a summer camp aimed at getting kids (ages 13-17) ahead in programming (as in they've never seen it before), or teaching them more than your average high school will (for the ones who have seen it before). I COMPLETELY agree with these results. Anyone who claims 'I can teach anyone this in 15 minutes' has obviously never tried. I have worked ONE ON ONE with people (albeit, many of them 13-14 and not college aged) to whom I explain these very basic assignment concepts slowly and clearly over and over and over, for a week of camp. They never get it. Ever. Then there are the other kids who when I explain it to the whole group the first time, get it perfectly and start asking me relatively high-level questions like "What do I do if I want to compare them?" before I've even covered if statements. Take into account, this camp is mainly for rich people, and all of these kids usually go to fancy private schools or very good public schools. A lot of them are really smart otherwise, but just don't seem to get programming. Obviously this study is in no way conclusive, but I still agree with it intuitively whole-heartedly. I believe that explaining to the students what the assignment operator does would help some, but really, in my experience, it makes far less of a difference than you would expect.
Alex on April 21, 2008 6:13 AMI find these results to largely be an abdication of actual teaching, sadly it is all to common in computer science departments. The teachers don't teach, and only the self-learners whose minds may be more predisposed to programming can pick it up on their own. Then the teachers point to the results. I have seen this happen again and again. Sigh. While predisposition is obviously a factor, it is real teaching that is needed, and that is rarely a simply going over of the concepts. It requires really translation of the underlying features of programming into easily graspable metaphors, analogies, and allegories. But that takes work, and dedication to teaching, which is not actually taught to doctoral students. So in the end they are not held accountable for teaching, but merely for knowing, and students continue to fail to learn.
Jeff McNeill on April 23, 2008 11:30 AMIn my experience of teaching introductory programming, I notice that many students do *not* see the = sign as an assignment operator. In fact, throughout their public school education, they learn that this is, instead, an "is equal" operator.
In ordinary math, the = sign means that both sides are equal. Another corollary to this in math is that the left-hand-side (LHS) and right-hand-side (RHS) are interchangeable.
In computer programming, it *never* has that meaning. For example, in Visual Basic, the = sign might mean one of two things:
1) Assignment - This is where the right hand side is "boiled down" to a single value, and that value is then stored into the single variable on the left of the = sign.
2) Comparison - This is where the left and right sides are compared and a question is asked, "Are these values equivalent?" The answer, of course, must be a True or False (aka "Yes" or "No").
Another interesting item to note is that this difference is seldom explicitly pointed out to beginner programmers. They are left to "infer" that the "mathematical meaning" isn't ever used in programming. At some point, we as educators fail to take into account the fact that we are "re-defining" symbols and constructs that students already have definitions for (largely through Math, but also through Chemistry, Physics, etc.).
Another item pointed out in the documents by Saeed is the problem that students have with thinking that all the statements somehow occur simultaneously. The idea of "sequence of operations" is not always easily perceived by students. Again, that can be seen in math statements. Consider these statements:
a = 45
b = 55
c - b = a
This is a "set" of "equations" which are all true at the *same* time. Their order is irrelevant. Sometimes their order is changed, but the meaning, it is emphasised, is the same:
c - b = a
b = 55
a = 45
The problem, as it is put to the student, is that they are to "solve for c".
When these students are put into a computer programming course, where they believe that the computer knows how to solve math problems, there is a fundamental set of assumptions and definitions (mostly false) that they bring to the area of computing.
Notice: These students aren't "dumb" or "stupid". Sometimes these are very bright students! They are simply employing a form of logic, but it is not *programming logic*. What we can do as educators is to be explicit about how we are going to re-define what they currently know or assume.
Dan Gilleland on April 25, 2008 8:52 AMActually, I have to say I have witnessed the same thing in both my programming classes and my logic classes.
Some students simply could not do logic. No matter how much everyone here howls in protest, some people can't do logic. Period. It's a fact.
Some of the students who lacked the ability in my classes were those who studied the hardest, asked many questions and really really tried. I felt bad for them. They really tried. They just lacked some kind of innate ability.
There is a middle group who can't do it yet but can learn. But there really is that group that is incapable. I am sorry if that offends you, but it's the truth. Sometimes life isn't fair.
Practicality on July 14, 2008 2:25 AMhttp://www.secretgeek.net/camel_kay.asp
Alan Kay's comments on The Camel Has Two Humps. Found on Leon Bambrick's blog, thanks to reddit.
Joe Chung on August 24, 2008 2:48 AMFor example, does anyone know of a framework author/architect
that is female? The people that come up with Jini, or Ruby on
Rails, or Hibernate, or the like. I can't recall a single
female author of groundbreaking ideas, either theoretical or
practical.
There are profound differences between men and women in world view and mode of thought. These are evident from the literature they create, the literature they consume and the way they comport themselves over the spans of their careers.
The archetypal chick flick – Gone with the Wind – is described in its own advertising as a searing tale of passion in a world gone mad. Essentially, it's about the feelings of the protagonist in a world that is utterly beyond the protagonist's control. If a Mills and Boon novel has a happy ending, it's provided by the intervention of a man. At no point does a woman attempt to change her world. She adapts to it, cries about it, or waits for a man to change it for her.
Men, by contrast, write about almost nothing but taking control of their world, and the mechanics by which this is attempted.
Another fundamental difference is the list thing. Men teach one another the mechanism, the distilled principle, because there is less to remember and it has to be taken in context anyway. Women want a fixed context and rote instructions. If you try to teach them the principles instead, they don’t listen and they get angry, saying “I don’t care why, I just asked you to tell me what to do.” If you give them a list of steps it must be exhaustive like a computer program because (also like a computer program) if context changes breaking the procedure or if anything has been omitted, blame is ascribed to the writer of the procedure.
A direct consequence of this intellectual inflexibility is that women do not create tools. They can be taught to use them, often very well, provided that the use of the tool can be described as lists of steps - programs!
Visit a craft shop like Spotlight. It will be crawling with women who think they are creative. In fact all they ever do is stick glitter to boxes, or cut cloth according to a plan that was almost certainly created by a man, before stitching it together using a sewing machine definitely both invented and made for them by men.
Some of them will vary the patterns, but creation ex nihilo is a behaviour exhibited almost exclusively by men.
I suppose you could say that women play god using the thing between their legs, whereas men use the thing between their ears. Probably this is enculturated behaviour. Possibly it is an artefact, in men, of the inability to play god the easy way; certainly many of us see our creations as children of sorts.
Peter, I hope you are not serious because that was the most downright stupid thing I've read in a month.
Chris on November 12, 2008 2:29 AMI'm 50, started learning electronics at 9 and programming on a PDP 8e using BASIC 8 in 74. I don't have a degree but have worked at two colleges. This is not about programming (there is no gene) it's about conceptual thinking. The particular problems on this test are related to transitive functions. These same people had a hard time in math class. They have poorly developed skills in two areas, they do not think symbolically nor do they understand levels of indirection. I would predict that the binary distribution would match their performance in their math classes as well.
My personal view is that these skills are 'naturally' developed in the early to mid-teens. It has some similar characteristics to language learning in that you can learn languages after that cognitive development window closes, but it's a lot harder.
It's great to post a comment on a post more than 2 years old now...
I think it's interesting that almost everyone has concentrated on disagreeing with the fact that the test was given using C like syntax. To me, being a novice programmer learning C related languages, I got the assignment right away and had no trouble with the question.
I think the more interesting fact is the idea that those who failed the test, biased or not, never improved. Isn't this the point?
One of the earliest commenters mentioned that the educators must be to blame and that he could teach anyone what's required to pass the test in 15 minutes. What this paper is getting at is that there may be those who, whether it is that they can't or simply don't want to, will not improve.
Having an insight like this before committing to 4 years of study might be valuable to the student.
The interesting thing here is not the test itself, rather it is what this type of test indicates about how the subject is taught. The problem if you will is not the student but the apparent refusal of the learned practicioners to realize that human beings do not all learn in the same way. Most of the people here claim that programming is like mathematics, I would argue that this is in reality quite false and that most programming is actually more like natural language rather than mathematics. Rather than worrying about who had the consistent model when answering the questions (Which really is just a way to avoid realizing that the typical method of treating programming like a mathematical endeavor is probably not the best way to handle things for the human beings that will be writing the programs.) one should be looking at how the students interpreted the question(s.)
When we are in grade school we are taught what one is, what two is, what A is, what B is. Programming seems to be the exact opposite for many they are given complete words and then expected to extract the individual sounds of the letters. Students are not taught to speak the language, rather they are taught specific phrases. So they end up like the tourist who really only knows four or five basic phrases in a language trying to ask for directions to a given destination.
Can any of you imagine learning to speak English without knowing enough to even begin to read a dictionary? This is what students and others must go through when it comes to programming. I think that this test may provide an excellent spring board for figuring out the best possible approach rather than an indicator of who or who will not be able to learn to program.
It doesn't matter how supposedly simple a programming language is to learn if those learning it are never taught the alphabet so to speak.
Shadow Skill on December 13, 2008 10:03 AMInteresting. When I was 15 I'm sure I'd fail the first test with variable assignment. We had an introduction to programming in high school and I barely passed it, because I refused to grasp exactly those 3 concepts mentioned above, probably for the exact same reason mentioned above. But by the time I was 18 I passed the second programming course in high school and introduction to programming at university with highest scores without much effort. The only reason I can think of is that something sparked in my head that I should do whatever it takes to learn programming after my initial failure.
Vasil on December 23, 2008 12:59 PMSorry Middlesex University isn't a good university. They don't get good student in the country. More on that have a look at Times Good University Guide: http://extras.timesonline.co.uk/tol_gug/gooduniversityguide.php?AC_sub=Computer+Sciencesub=18x=52y=6
Student on December 24, 2008 3:41 AMJust to note -- the problem at my university does not have much to do with the kinds of people coming in or the methods of teaching. It has a whole lot to do with the fact that they are forcing non-majors (Japanese and Math, as two random examples) to take a Java programming course. I can see no way for this to be helpful -- especially not to their GPA! -- and it makes the first class a joke for those of us who are Computer Science majors.
Marble Host on February 18, 2009 11:23 AMFrom the page: Evidently not everyone is as fascinated by meaningless rules and meaningless conclusions as we are; I can't imagine why not. When I began programming, back in the olden days of punched cards and paper print-outs (70's), I observed that some programmers were able to write code that was complete crap. I found blocks of code that would receive an input value and arduously convert it into the same value it already was.http://www.marblehost.com
Initially I was reluctant to believe my own mind; there had to be something I was missing. Slowly, I began to realize that many programmers had no idea what they were doing, and they were in the industry! How could this be? I saw this situation many times through out my career. For example: SELECT last_name, first_name FROM emp_name WHERE 1 0 That chunk of code will separate the sheep from the goats. I forgot - which ones were the programmers?
orange!!!!!!!!!!!
Diplodocus on March 17, 2009 5:03 AMI'm sorry but that is just rediculous - and it goes to show how crap modern day schools are at teaching, and understanding pupils.
Instead of disregarding non-programmers you should be seeking ways to communicate to both types of student in their own language.
Zoe on March 23, 2009 11:40 AM@Shadow Skill Exactly! I completely agree!
Zoe on March 23, 2009 11:47 AM@Peter thats has got to be one of the most elaborate trolls ever - because it is simply impossible for an intelligent person to believe that nonsense.
Zoe on March 23, 2009 11:53 AMyaa its true that 30-60 % students can't program most of them fial in thier first test.But everyone wants to be a programer so everyone id putting there hands into this without knowing there scope or anything.I think today lot of programmers are available and daily new one's are also coming.My suggestion for programmers will be to shift towards the mobile world because a lot mobiles are being manufactured everyday.and in turn public demand is increasing.
http://www.funmahol.com/
so there will be a huge chances of success in this field also chances of succes are lot more than in any other programming field comparing to programming for mobiles.
It was two years ago, but Bob the Chef's post is a bit amusing; statements in formal logic are meaningless by the definition of formal. Yes, we may choose to interpret those strings of symbols as programs and/or data and call some correct and others buggy, but none of that is in the realm of formal logic and when it comes to reasoning about what the computer does, you have to leave all that interpretation stuff at the door. Unless your computer is sentient, everything it does just involves meaningless manipulations of meaningless strings of symbols - it's not going to know any better. It's the *programmer's* job to deal with that fact - that statements of formal logic are meaningless. (And definitions only go so far and no further: Euclidean geometry never defines what a point is so point is a meaningless concept.)
Like I said, it was two years ago - what prompted me to post was that I gave the full test to my brother (he's a panelbeater with zero programming background). His first complaint was that he doesn't know any programming (as if I didn't know).
I told him it doesn't matter; just see if you can make any sort of sense out of it.
He was wrong in every answer (assuming conventional C-like semantics), but he scored in the consistent group. Maybe
I should suggest he take a few night classes in programming?
It's very interesting, if i was young, I would fail in the test. When you go to college, you must learn the programming because you have no choice.
evening shoes on May 26, 2009 11:52 AMThis should not take any programmer more than 6 seconds to figure the answer.
What are those people doing who are getting it wrong?
DarkNight on May 27, 2009 4:27 AMit's all personality
you have to like to solve puzzles, coding is all about solving puzzles.
those who have very analytical and problem solving type personalities succeed.
those who do not are more at risk of failure.
check out this website http://www.bizet.com/ava.php
they know why.
david on June 12, 2009 2:15 AMVery nice post.I agree mostly but every person is diffrent.
Egipat Letovanje on June 20, 2009 11:43 AMI agree with Steve.
Egipat Letovanje on June 20, 2009 11:45 AMJesse, I read your thoughts on the double hump and I felt like I was reading my biography :D
"I'm just supposing here, but I bet those students who need some practice and attention (perhaps some help that would go towards learning a more consistent mental model) don't get the attention and encouragement they need. They are thrown into the midst of cryptic syntax, alongside a bunch of folks who already seem to understand it naturally, and they get discouraged very quickly because they don't pick it up as fast. The classroom is huge, the teacher has way too many students, and doesn't have time to bring the stragglers up to speed. Especially when the stragglers may become quickly demotivated in an environment where the subject matter comes so easily to some students. Just my opinion, but I think this phenomenon comes much closer to explaining the double-hump."
Sorry to quote this whole thing again, but this really rang true for me, so true that I'm actually posting something on the internet. I was one of the stragglers in my CS 101 class, I felt so confused the first day when our professor asked if anyone had prior knowledge and 3/4ths of the kids raised their hands, and even more confused later when it seemed I just couldn't get it. I ended up failing and having to retake it, and I felt truly miserable about it.
Well to put a long story short... somehow it just sort of "clicked" for me. I don't know why, but I've since caught up with the kids in that 101 class I failed and have risen to a 3.8 GPA in a year. Maybe some people are inherently better at programming, but that's life. Some people are inherently good at piano, but it doesn't mean anyone can't become decent at playing piano. Just because you aren't going to play and be the star of a symphony orchestra doesn't classify you as a bad pianist.
peter on June 30, 2009 12:55 PMThis article is wrong. Programming is about taking a basic set of instructions, and making something complex out of it. You can't ask questions about basic instructions, and if a student answers it wrong assume they can't program.
It does take time for the logical parts of the brain to develop. If you are trying to teach someone to program who can't grasp the logic, don't assume that the'll never ba able to program. You just have to help them develop the logical part of the brain.
Lastly programming requiers problem solving ability. Just becouse a student is not good at this, does not mean they will never be good at it. Things like project euler can *quickly* get someone better at problem solving. If not, than more gradual approches are availible.
This article seems to discurage people who can't program "in the early stage" to continue programming, they will never get better. That is BS. Everyone can get better at programming. Just becouse you fail your first test does not meant you'll fail all of them.
Hiroshe on July 10, 2009 8:05 AMI just have to wonder:
If we accept the premise that assignment + MeaningVsRules is both the first hurdle and the failure point for so many student, could we instead just spend a little extra time dealing with those points for those students who need it?
Joel Coehoorn on July 22, 2009 12:19 PMThe question does seem biased towards some programming experience. If the idea was to identify an ability to somehow detect rules consistently, when they really want to exclude any previous familiarity with the language, they should have picked symbols which aren't so C-ish. After all, they say the test takers have to detect a consistent connection, not know C. And that = sign in both the question and the answers, is also complicating the matters.
I wonder how similar the answers would have been if the questions were written like:
} a*10 %
} b*20 %
a*b %
The rules are the same rules. Just replacing some unknown symbols with other unknown symbols.
Plus, I don't like the idea that they think they can tell what represents a consistent outlook, and what isn't. It's perfectly possible that someone new will see a consistent rule which the researchers didn't. It could be a seemingly silly connection, one that a programmer will never make, but it may still be consistent. Someone who doesn't know the rules can come up with very wacky, yet totally consistent, ones.
The language, as mentioned in the comments above, is actually something I don't have a problem with. The values are new values, even if they're the original values. The values were just assigned, so they're new.
On the technical side, the original value may even have been the default uninitiated memory. Without exact knowledge of the language, the compiler, and the context (maybe a and b have been defined before, and this is a re-definition? Maybe something else? Remember, this isn't officially C, and doesn't necessarily comply with any specific C standards) the values may very well be new.
And on the semantic side the variables didn't have a value before they were assigned, so this is the new value. Otherwise you justify sentences like "My new car? Oh, this isn't my new car, I just got my driver's license and bought it yesterday, so it's the first car I ever owned."
Yaron on February 6, 2010 9:46 PMQuibbling about the language is missing the point. I think part of what this test seeks to identify is the ability to determine these things heuristically, without necessarily knowing how syntax translates into semantics. That's mostly what you end up doing in production code - you don't write clever academic programs from scratch, you use libraries and frameworks and other people's code that has either minimal or no documentation, and try to make sense of what it means and how to use it. If you can't do that, and you need to have everything spoon-fed to you, then as the test states, you have no business being a programmer.
Remember that the test is looking for *consistency*, not necessarily *correctness*. Even if someone got the syntax wrong (I'm not sure how), they're looking for the formation of a mental model, which is a required element of problem solving, which is what programming is really about. If you can't form a consist mental model on your own, you can't code. When you write real production code, you're most often working in an unfamiliar problem domain and you have to do the same thing - solve a problem you don't understand!
So it's using C syntax, or C# syntax, or the syntax of any of the other 50 languages which is exactly the same. So what? It's very close to the algebraic syntax you ought to have learned in grade 9 or earlier, much closer than a language like VB or Pascal.
Blame the teachers if you want, but there are lousy teachers in every discipline and you don't see the double hump in their classes. What's so special about programming?
Aaron G on February 6, 2010 9:46 PMInteresting discussion.
A few observations from my 23 years of programming and 3 years of teaching high school programming.
Sexism: In my prior life as a programmer/manager, about half my team was female. Quality of programming was indistinguishable between the sexes. In some teams the best programmer was female; in others it was male; often it depended on the business knowledge of the programmer rather than on the mechanics of programming.
Language to teach: I opted to teach Java as my introductory programming language. It provides a safe language for beginners: they can't accidentally do something that trashes their system. It can also be written and run on any of the major high school platforms: PC, Mac or Linux. And yes, all three platforms are represented in my student population. And no, knowledge of multiple platforms does NOT mean that the student is a better programmer!
Beyond that, Java allows all the basic strategies of programming: procedural approach, sequential processing, compiling (OK, so it's byte code and not a "true" compile. They're still exposed to the thought process of translating written code into something machine readable.), logic statements, loops, recursion, data types, arrays, sorting, trees, linked lists, file IO, searching through documentation for the code they need. I also require documentation (the bane of many "true" programmers), requirements gathering, testing (string, unit full application), and good design. For my advanced class I also get into project life cycle and maintenance by taking a single application through several cycles of improvement.
Weeding out: My first couple lectures are completely from hand-outs, and the first assignments are based on that information and the class discussion. I then review the programs in class using student code (from that class or another, with the victim's name removed) so we can see examples of good and bad design and code approaches. I also put my own code up for review, including bad code, so we can go through trouble shooting and debugging. That is the most useful teaching technique I've found based on the students' "ah ha!" response. After several weeks and assignments I talk with all the students about continuing in programming or finding another area of interest. Most don't need the conversation.
BTW, my favorite first program is to write the directions for making a peanut butter and jelly sandwich. Then I'll pick one and misinterpret as many of the directions as I can to illustrate the stupidity of the computer, the importance of clear instructions.
English language: Having come from the corporate world of application programming and working with the user community, I emphasize to the students and to our English teachers the importance of clear writing skills. Writing is the most important part of working with users: meeting minutes, email, user documents - if they are unclear to the users, you are just wasting your time and theirs, and your project is doomed. Unless you work for someone who has those skills, but you won't ever be in charge of many successful projects. But maybe you only ever want to be a bit twiddler, and those of us in management appreciate the bit twiddlers who work for us. (Is that food for flaming, or what?!)
Quality of teaching: There is no doubt that there are good teachers and there are bad teachers. And while there is pressure on elementary teachers, and to a certain extent on high school teachers, to use better methods and techniques, I don't see the same urgency in the college and higher levels for teachers to keep up with current teaching techniques. In many cases they have there PhD so know what is best for us, and if we can't get it, well, then we just don't belong in the field! I've been fortunate to have had good or excellent teachers. Not that there are things I wouldn't have changed! And for the money, there are few better college classes than evening classes at community colleges in the DC area: the students are pros in their fields, looking to improve their skills or expand their knowledge base (no trade mark infringement intended) by taking evening classes. They know what they want and are very demanding, so the best teachers are assigned to their classes.
And yes, I will play with the test and see how it does in my classes. And I'll watch out for follow up studies, because this has said, "There is something interesting here."
Well, this has certainly gone on much longer than I intended, but there were a lot of points in the discussion that I wanted to respond to based on my experience.
I find it curios that only one poster commented on the imperative nature of this test.
It would be interesting to see what results would follow if they'd choose a purley functional model of programming.
I can even imagine that the dichotomy isn't between ability to program or not but rather affinity towards functional programming v. affinity towards imperative programming.
John Nilsson on February 6, 2010 9:46 PMI don't believe coding is pretty much decided from the start. There is of course the gifted programmer, who rarely has any problems. The one that doesn't seem to understand anything, and the one struggles at first, but eventually becomes better because they don't actually understand what's going on at first.
I personally think using a language like java cripples the programmer with all its features. I started out in C++, passed the first course of programming with a C, then struggled with the second semester and got a D. I retook, and still only ended up with a B, then took a advanced course of C++ and got a A.
I actually enjoyed learning new things, even though they were pretty difficult. However, when I transferred, and the school that I go to now teaches java, everything seems oversimplified, and they don't like to go into technical details, but like to talk in abstract terms.
I believe the test you described was kind of off-topic. Someone can easily read that statement from right to left and get the opposite answer ( it's not impossible to make a language's assignment operator in the opposite direction).
In general I would have to say that a programmer's ability is partially decided by their initial state, but it's not impossible to become a good programmer though experience.
Animehiro on April 7, 2010 9:36 PMYou say this test is "intended for students who have never looked at a line of code in their lives". How, then would they even know that "a=b" is changing the value of a or b at all?! If they learned algebra, they'd think that you're saying "a and b are equal". And how are they supposed to know what the hell "int" means? That test is bogus.
Colin W on April 11, 2010 8:35 PMIn the foolish belief that someone might actually care about reading this comment, I find this article extremely sad.
You have identified the very thing that needs to be taught in order for someone to program. You have determined a way to test whether it has been learned or not. It is beautiful. The logical next step is to figure out a way to teach that. Why do you stop there and assume the rest is pointless? Why not dig deeper and figure out how to teach that particular point?
Programming is no more difficult than reading. At one time, only the special few, the priests and scholars, were able to read. As time progressed, our teaching methods improved, and more and more people learned to read, until today, even little kids learn to read. The techniques to teach programming will likewise improve, until eventually we are able to teach it to anyone. You have made an excellent advance in this area: why did you stop there?
Andrewthompson10 on August 28, 2010 10:54 AMAs woman, someone who did poorly in algebra, and never programmed (well I had BASIC in grade school, but it was just copying pre-existing programs), some of the comments here are why I initially stayed out of the computer science field. Fortunately I met some men in my department who were just really encouraging and supportive so here I am in my first year of starting programming, and I discovered that I love it. I should have known, considering that 2 years before I decided that I wasn't happy about my lack of mathematical abilities and decided to teach myself and ended up loving it despite my performance in high school. What's interesting is that the book I used was from the "Art of Problem Solving" series which is designed for gifted students and students taking part in math competitions. However it teaches math in a way that I love and enjoy. I wish I had it during high school. Anyway, I found computer programming to be the same. I love it, and when I find resources that explain it to me in a way that I understand I excel. I also am very fortunate to have a professor that believes that everyone can get it and hard work determines your success. I also find it interesting that when there are women in my CS class, they are usually of South Asian and Middle Eastern descent. They don't seem to have any confidence issues, and don't seem to buy into the theory that because they are women they are disadvantaged. I've learned a lot from them and try to think as they do.
(Warning: Gross generalizations ahead!) There is a real preoccupation with superior/inferior and smart/dumb in Western culture. People are believed to be inherently superior or inferior and there's not much they can do about that. Contrast that with the Asian model which posits that your superiority in a subject is a result of how much effort you put into it.
I think the conclusion of the study represents the split in these two schools of thoughts. The authors of the study seem to accept the Western model, so they conclude that the students who don't form a consistent model can never be taught, rather than conclude that those who don't form a consistent model should be taught how to form a consistent model.
I would suggest using the assessment not to weed out people, but as a placement for different Intro to CS courses. Most universities should have Intro to Programming courses that separate those who have had some programming experience and those who have absolutely none. I know this is not feasible for many departments but I think to assume that all students step into the the Intro CS course with an equal footing is quite wrong. In my experience this has never been the case, and those with more experience tend to drive the level of the course upward and make it miserable for those with now programming ability.
I have to say I'm pretty glad I don't have the authors of these articles as my prof. I'd probably be pretty disadvantaged.
Kmj Reg on September 12, 2010 2:27 PMI can tell the programmers from reading the comments.
You're not reading the rules, or blindly ignoring them. Sure you can teach them to pass the initial test, but that's useless -- it invalidates the test.
I could go further, but if you don't get it, you won't get it. Why waste the time? I respect the rule.
Eloi Eloi on September 23, 2010 6:03 PMInteresting article, I enjoyed it.
Extending on what you said, if the a=b test is for assignment and sequence, I would suggest that coding a solution for the Knight's Tour (http://en.wikipedia.org/wiki/Knight's_tour) would be a good test for recursion / iteration. To me this begs the question, what would be the concurrency test?
Alex Bethke on April 28, 2011 3:38 PMAn intelligent, long-form, highly referenced treatment of the subject is in the book "making software" by O'reilly. It's chapter 7, "Why Is It So Hard to Learn to Program?" by Mark Guzdial.
Aŭto Didakto on May 10, 2011 2:10 AM 44% of students formed a consistent mental model of how assignment works (even if incorrect!)
39% students never formed a consistent model of how assignment works.
8% of students didn't give a damn and left the answers blank.
Oddly, approximately 9% of computer science students are unable even to perform basic tasks such a comprising a portion of 100%.
PSR on October 13, 2011 10:53 AMThe comments to this entry are closed.
Subscribe in a reader
Subscribe via email
|
|
Traffic Stats |