Uhm ... you've fumbled the variable assignments in the first question.

Aw, c'mon, it was a one-character typo. Ain't there no love in the 'hood?

It's only tangentially related to the above entry, but I found this great article, "Johnny can so program", here:

http://news.com.com/2010-1007-5700858.html?tag=tb

I wonder if there's another variable at play. Perhaps the students who scored with a consistent view had already been exposed to programming. When I was in college, I remember the students in my classes who had trouble were ones who were entirely new to programming. The rest of us had either been in programming classes previously or had been coding for fun since we were kids.

The interpretation of the results of the above assignment test make me feel angry. I am not a teacher but I am confident that I can teach (in less than 15 minutes) any person smart enough to be admitted to a computer science class how to pass the above test. The results speak volumes about the utter ignorance of educators regarding the nature of human knowledge and how to acquire it. It is a tremendous act of injustice to blame the students and not the teachers. Now if a student is *unwilling* to learn the material then that is another question. But to blame the poor education methods on students willing to acquire the knowledge is just ridiculous.

"unteachable"? How is this teaching being done, would changing the teaching method affect the results?

Anibal, yes changing the teaching methods would definitely change the results somewhat. Maintaing 100% accuracy with humans is nigh on impossible.

It is a good paper, I was at its presentation. I look forward to the follow up work that will be done on it. Unfortunately most computer science departments are not in the position to turn away students, so I think the best application of this(if it turns out to work well in all institutions) would be for streaming the class into honours students and pass students.

Generally in CS Education Research you'd need a multi-institutional, longtitudinal study to have fully verified results, so I guess we'll wait and see what comes out of it.

I liked the paper, but the acedemic snobbery in Section 5 pissed me off no end:

"The consistent group seem to be much easier to teach. We speculate that they divide into two groups: the very successful ones find programming easy and may perhaps be those who ought to follow a conventional computer-science education, with lots of formal content; the moderately successful perhaps are the software engineers, those who can program but can’t imagine that they will ever enjoy it, and are content to submit to management discipline and to be drowned in UML (ugh!)."

Poor software engineers, doomed forever to management discipline, UML, and a lack of formalism, all in a job that they'll never enjoy!

I would add 'indirection' as hurdle number 3 between recursion / iteration and concurrency. Unless you treat indirection as part of recursion? Anyway, I observed of my fellow students that, whether expressed as explicit pointers or references, or as reference semantics, many of them simply never got the concept of indirection.

Programming ability is not known to be correlated with age, with sex, or with educational attainment

Uh, it is known to be correlated with sex. In fact, it is glaringly obvious. Are these people jokers?

I'm by no means an expert but I've been teaching high school computer science for 14 years. Was a "real" programmer before that.

I did link a comment to this post and to the orginal study, because I find both fascinating.

Over the years, I've had actually three groups of students.

1 group vows very to get involved with programming again. 1 group absolutely loves it (I was in the group myself), and can't imagine doing anything else. A third group can do it, but doesn't like it much.

The good news, since I'm teaching high school, it gives them an opportunity to find out which group they belong in without shelling out any money.

The funny part is the high number of parents who are upset that their student isn't in the 2nd group. I keep telling them that their kids are normal but they don't always believe me.

And yes, the first group is the most prevelant.

I get both gifted and talented students, regular students and special ed (not many of those), many of them who have no clue what computer science is, but think it might be cool to sit at a computer for an entire class period. Many of my second group have come out of the group that didn't have a clue if they would like it or not.

Another good article that's food for thought. This site always has good stuff!
I always wondered about that glazed stare I sometimes get back from some smart, though non-techie people when I try to explain something. Am I not communicating well?
Too much jargon?
Have I not simplified the question or explaination enough?
Do I have something between my teeth?
It usually happens when I'm trying to gather requirements or when explaining what happens when they do a particular action within an application and how that might effect other things in the system.
It's sort of like that Friends episode when Phoebe tries to teach Joey French.
I'll have to keep this article in mind and change how I explain things or ask question when I get the "look." Instead of trying to explain why the computer behaves as it does and getting myself and the user more frustrated, take a different tack completely.

"Programming ability is not known to be correlated with age, with sex, or with educational attainment

Uh, it is known to be correlated with sex. In fact, it is glaringly obvious. Are these people jokers?"

Unless you mean correlated with *having* sex (which in inversely related, I think...), you are wrong. Fewer women *attempt* to become programmers, but those that do they are just as capable as the males. The ACM and IEEE are trying to engage more women in the the field, but the are hitting a wall with the sterotypes of the field, not the ability of women to code.

I wonder how many of the people who fail programming classes also fail algebra classes?

It's an incorrect question. B has NO new value, only an old value, ergo all the answers are wrong.

This is what happens when you let subliterate engineers design a test: they screw it up. Right there, in the directions.

There are plenty of 17/18/19 year olds insecure enough that they are going to read the question like an actual English sentence (which it is not, it DOES NOT COMPILE given the correct answer, but you coders seem to have ZERO respect for the actual language us humans use) and believe that B HAS CHANGED, MUST HAVE CHANGED.

Such total crap.

Hey, I didnt do to well in algebra or physics, but I got a 3.99 GPA in programming, I did however such in English ("but you coders seem to have ZERO respect for the actual language us humans use") I guess that statement might be true.

That should be "however SUCK in English".

I haven't read the paper yet, only the test, so excuse me if I've overlooked something. But at first sight I find that assignement operator is a (unfortunate) convention of certain programming languages that we just have to learn, but it has no logic as seen by anyone that has learned the meaning of "=" in maths.

I wonder what would have happened using a different notation like "<-" or ":=" that risks no confusion with equality operator. I think it's called "usability". Maybe that explains why most programmers are unable to design decent user interfaces.

The question does seem biased towards some programming experience. If the idea was to identify an ability to somehow detect rules consistently, when they really want to exclude any previous familiarity with the language, they should have picked symbols which aren't so C-ish. After all, they say the test takers have to detect a consistent connection, not know C. And that = sign in both the question and the answers, is also complicating the matters.

I wonder how similar the answers would have been if the questions were written like:

} a*10 %
} b*20 %
a*b %

The rules are the same rules. Just replacing some unknown symbols with other unknown symbols.

Plus, I don't like the idea that they think they can tell what represents a consistent outlook, and what isn't. It's perfectly possible that someone new will see a consistent rule which the researchers didn't. It could be a seemingly silly connection, one that a programmer will never make, but it may still be consistent. Someone who doesn't know the rules can come up with very wacky, yet totally consistent, ones.

The language, as mentioned in the comments above, is actually something I don't have a problem with. The values are new values, even if they're the original values. The values were just assigned, so they're new.

On the technical side, the original value may even have been the default uninitiated memory. Without exact knowledge of the language, the compiler, and the context (maybe a and b have been defined before, and this is a re-definition? Maybe something else? Remember, this isn't officially C, and doesn't necessarily comply with any specific C standards) the values may very well be new.

And on the semantic side the variables didn't have a value before they were assigned, so this is the new value. Otherwise you justify sentences like "My new car? Oh, this isn't my new car, I just got my driver's license and bought it yesterday, so it's the first car I ever owned."

hi jeff

the example given seems very tilted toward people with experience in a C-like language.

if instead of:

int a = 10;
int b = 20;
a = b;

it was in a more verbose format:

Variable a is integer
Variable b is integer
set a = 10
set b = 20
set a = b

then clever kids with no programming experience might have a fairer shot at it.

otherwise you need to use the 'code-cracking' part of your brain to try and infer a meaning from an unknown syntax.

i know that the biggest barrier to learning in the earliest CS subjects i took was the quality of the lecturers, not the material itself.

interesting article still. best of luck.
lb

I think what most commenters are missing is that the results of the test don't change after 3 weeks of the students being taught programming, by which time they would have been taught the syntax etc.

I'm a latecomer to this discussion, but in my experience as a late-blooming 30 year old CS undergraduate senior, I've found the programming classes to be useless, and less badly taught as un-taught. I've only been to a community college and then the University of Illinois in Chicago, but the introductory programming classes were:

1. Object-oriented, which left students with little or no understanding of procedural methods, and

2. Weed-out classes. The classes consisted primarily of descriptions of different types of problems and the mathematics behind them, rather than ayntax and structure, for which people were told to just read the book.

Code was barely directly acknowledged until the Data Structures core, and then it still depended on which instructor you got, some being very code light and some being nicely code heavy. You could tell that it was a big temptation for teachers to be code light at this point, because if they concentrated on code, they would also have to concentrate on teaching students who had been in a computer science course for two years how to program.

Since coding is a hobby for many young people, I think that educational institutions have relied on that to establish their expected learning curves, leaving people who had little to no experience programming when they entered school no choice but to cheat like crazy, spend all of their spare time studying code, or switch majors. And it isn't a necessarily a deficit in abstract thinking in my experience, because everyone I know who dropped out of CS ended up in Electrical Engineering, which is nothing to shake a stick at on the abstract front. They still don't know how to program, while doing math that I can't make heads nor tails of. Most absurd memories:

1. Java as the required programming language. I'm not going to bash Java here, but wouldn't it be nice for students to have to learn their own garbage collection? And wouldn't pointers be a nice thing to learn, even if we never decided to program in a language with them again?

2. Taking a core class on operating systems theory, after being being deluged with Java, and finding out it was in C (of course) without even one C class on campus?

Of course, I'm thirty, and also one of those people who always programmed, so I had no problems, but I saw plenty of people who I knew were better at abstraction than me (from Calculus, DiffEQ, and physics classes before) and their total agony at trying to finish a program that had been stacked against them.

It might be worthwhile for people to mentally review what exactly a single study means in the realm of science.

It is neither more nor less than evidence. It may have been misinterpreted. It may be the beginning of a solid proof. As yet, though, you can't know which is which; it is merely an interesting tidbit.

Preparing to use this test in a real college environment or decrying the study as rank elitism are both premature.

I'm prepared to believe that perhaps the standard teaching methods are questionable and that with other methods the currently "unreachable" students might yet learn what is meant. In fact, this test is useful simply by virtue of providing a discriminator; much of psychology works that way. We can't actually know whether a person has "schizophrenia", because we don't even know exactly what "schizophrenia" is, but we can study how two populations, divided by a standard "schizophrenia battery test". Without that tool study is borderline impossible.

On the other hand, I'm prepared to believe that the ideas hinted at by the study are true and there are people who just can't get it. I saw senior-level students who I seriously wonder if they could pass that test, and after four years of experience I daresay teaching methods should be of minimal consequence; the teaching method of the compilers is both constant and rigorous.

It's worth further study. If it does turn out that there is a segment of people who literally can not understand computer programming, and it turns out we can identify them with high reliability, then we *owe* it to those people to so identify them before they spend years going down the wrong (for them) path.

"Encouragement" is not an intrinsic virtue. "Encouraging" somebody who has been mountaineering for a week to take on Everest is not a positive thing, it's postively *murderous*. Encouraging somebody who can't learn to program to learn to program is not a positive thing, it's *evil*. If this study does reveal something, then the authors are not evil elitists, they are the heros of every non-programmer the test identifies.

Hmm, so the key is to never work with the 44% of students that formed an incorrect, consistent mental model of how assignment works.

That explains a lot.

3 weeks of programming classes taught me *nothing*, except what textbook the lecturer had helped write and wanted us all to buy.

The problem in a current educational setting is deciding which languages (of the thousands out there) to teach students. Majoring on .Net and Java will teach them useful work skills, but some time spent on Ruby, Python or other languages might be more useful in teaching them that there is more than one way to solve a problem.

I was taught COBOL, Miranda, Assembler and SQL - except for the last one, a more useless set of languages you would be hard to find. My best learning experience was on the job learning the actual languages I needed to use.

While the question is indeed vague and the asignment vs. equator could be an issue, I don't think it really is. Given that the answers are not "true" or "false" the function of = is fairly obvious.
The C style syntax is not a problem either. What they are looking for is a consistant mental model. If it's some crazy language where = moves the value from b into a and leaves b empty, then that should be reflected in all of their answers.

I wonder how many students would have answered false in the question if it had been there.. mind you, I taught a guy a language after he had read the teach yourself language in 21 days, and been on a 2 week course and had a degree in computing.. still didnt know what an integer was.

having been taught programming as part of an engineering course, where the course book was along the lines of 'C for idiots' (which was actually a good choice as it turned out :-) ), by a lecturer who understood FORTRAN and didn't have a clue what 'C' *was* let alone how to use it...

I spent many an hour with the only other person on the course who understood anything (far more than I did) running side classes to explain things. ironically the lecturer 'attended' these classes as well.

these tests would have been a very good idea. put the people who pass into one stream and the people who fail into another, teach to peoples abilities, then a bit further the stretch peoples minds.

there is not point baffling people, programming isn't hard once you get the basics. its just that 'getting' the basics can take time and needs to be taught well, by which i don't mean the all to common answer during my education of "you don't need to know that to pass the exam" which really annoyed me...

int a = 10;
int b = 20;
a = b;

Is C (based) syntax. You have to learn C to know the correct answer. If someone does not know C (or any other programming language) they could easily not know that = in this case is an assignment.

Assuming the student being tested to know such syntax is ridiculous (after all whats the point in the int keyword? or the semi-colons?). The person who designed the test obviously forgot that they themselves know how to program. Either that or they were just lazy to think up a more generic language.

All the same, interesting stuff. Might be good to see the test conducted on actual so called professional programmers as well (not just students). I wonder how many "professional programmers" have no aptitude for programming? ;-)

Quibbling about the language is missing the point. I think part of what this test seeks to identify is the ability to determine these things heuristically, without necessarily knowing how syntax translates into semantics. That's mostly what you end up doing in production code - you don't write clever academic programs from scratch, you use libraries and frameworks and other people's code that has either minimal or no documentation, and try to make sense of what it means and how to use it. If you can't do that, and you need to have everything spoon-fed to you, then as the test states, you have no business being a programmer.

Remember that the test is looking for *consistency*, not necessarily *correctness*. Even if someone got the syntax wrong (I'm not sure how), they're looking for the formation of a mental model, which is a required element of problem solving, which is what programming is really about. If you can't form a consist mental model on your own, you can't code. When you write real production code, you're most often working in an unfamiliar problem domain and you have to do the same thing - solve a problem you don't understand!

So it's using C syntax, or C# syntax, or the syntax of any of the other 50 languages which is exactly the same. So what? It's very close to the algebraic syntax you ought to have learned in grade 9 or earlier, much closer than a language like VB or Pascal.

Blame the teachers if you want, but there are lousy teachers in every discipline and you don't see the double hump in their classes. What's so special about programming?

It's important to note that:

* 44% of students formed a consistent mental model of how assignment works (even if incorrect!)
* 39% students never formed a consistent model of how assignment works.

They didn't just look for "correct" answers, they looked for consistency. So, if you don't know that a=b means the value of b gets assigned to the value of a, then the group in the first bullet would create a mental rule for what it means, and consistently apply that rule throughout the test. The folks in the second bullet would 'change' that mental rule for each question.

Now, the question is, if you didn't know, and you were guessing, did the folks from the second bullet alter their rules in order to get some of the questions right, instead of applying the same rule throughout the test. The whole, "well, I don't know how to do this, but I want to get some of these right" attitude.

A double hump is not limited to programming. In elementary geometry, there is a classic problem called the Pons Asinorum ("Bridge of Asses"). Some students handle it swimmingly, others require extensive handholding. It has also been said that regarding advanced mathematics, only about 15% of secondary students really "get it" and the other 85% just follow along. Knuth once remarked that in Stanford CS classes, only about 1-2% of the students were at all fluent when it came to programming. Gerald Weinberg in "The Psychology of Computer Programming," noted that unlike most fields of endeavor, where the difference between the best and worst practitioners might be a factor of 2, in programming, even after being measured with a wide variety of metrics, the difference can be as much as a factor of TEN.

I remember looking at code before attending my first "Introduction to Programming" class and not understanding how something was "equal" to something else. This was one of those "aha" moments where the pieces came together. I would have failed the question before the class but that simple stepping stone opened worlds to me.

While I agree that the language specifics isn't really a problem, they are not meant to get it correct, only show a consistant model the C style asignment operator is confusing to everyone I've seen exposed to it for the first time. ":=" is a little better, and then something like "let x = 10" next, and then something like "let x be 10".

I'd note that not only in programming does ability vary by a factory of ten. I've also observed a similar range in teachers.

It's kind of obvious which end the authors of the study fall into, since they assign the blame for the problem to the students.

And yes, you can teach the "I don't get it"s to program, but you have to have some understanding of teaching, not simply a degree in programming.

As an individual who has gone through many MANY programming courses, it's well known to most people that these two groups exist. I was dismayed in my first year of engineering study when 90% of the class aknowledged that they had already learned Java (program we were studying) and many of the topics covered by the class. I was not one of them. By the end of the term, the class average was somewhere around 50% (tough university as well), with most students failing and struggling to even understand many of the basic concepts taught in class. Regardless, without much effort, I managed to pull out one of the highest marks (sorry if it sounds like i'm bragging).

So why does this divide exist? Obviously it has nothing to do with how smart an individual really is, but more so how they think. In my opinion, actual programmers are clearly able to see the big picture, and how little bits and pieces relate and affect everything else. Sadly, with how simple programming is to teach/learn, the industry will continue to churn out huge amounts of incompetent programmers.

So the answer was a=20 b=20 ?

Exactly the same thing happened in the Kalmar Laszlo Computer Sciences School (Or whatever is the english name of it) in Hungary. Btw it seems to be true: "The two most common elements in the universe are Hydrogen and stupidity."

I think the previous comment on prior programming exposure is a good one. I remember first 'getting' assignment from a Spectrum BASIC program I had entered from a book. I have had similar experiences in 'getting' Prolog, C++ pointers, threading and, more recently, Ruby closure blocks, so I feel it is not a matter of either understanding, or not, these concepts, but in how long it takes for one's brain to make the comprehension leap.

If people fell into two camps on understanding assignment, then would not those assignment-impaired individuals never understand alegbra also?

This not c syntax. The answer could be 20 or 10 as long as the rule is applied consistantly by the person taking the test.

QUOTE:
"So why does this divide exist? Obviously it has nothing to do with how smart an individual really is, but more so how they think. In my opinion, actual programmers are clearly able to see the big picture, and how little bits and pieces relate and affect everything else. Sadly, with how simple programming is to teach/learn, the industry will continue to churn out huge amounts of incompetent programmers."
Daiken on July 17, 2006 10:46 PM

*right u are!
From the beginning I would like to say that a programming course does teach openminded students something..
I think the test is a good tool both for teachers and the students. The students receive a signal that they might pretty much be wasting their time and money for soemthing they-re not suited for, or, in case, let's say..they weren't yet showing interest by the time of the second testing, they should start doing it, if they really want to be in this business..My ideea is that there are many bright students that "wake up" pretty late during faculty and start seriously working as programmers and maybe be miscathegorised by a test like this, and maybe a "wake up call" early in the process of learning would be good..I sure could have used one earlier :)

I think this problem is a reality, I sure know some of the "ones that get it" and some who don't get it, no matter how much I try to explain to them... Whether this is because I'm a bad teacher or because they won't ever get it is left to study, but there sure is an hell of a difference between those two kind of people in the amount of effort they need to attain comprehension of the basics !
However I wouldn't say this is specific to CS, I encountered the same phenomenon in Maths (though the ones who are good in Maths aren't always good programmers). Of course this doesn't appear as much in college because Maths courses start way earlier and the Maths student are all in the "get it" part...

I think that the syntax is a minor problem. Except perhaps for the int declarators, it is fairly obvious what happens. What might not be as obvious though, is the fact that the program is a sequence that should be read top to bottom. I.e. the statement "a = b;" happens *after* the initializations of a and b. If that does not occur to you, determining the values of a and b is not straight forward.

I failed my first programming course in college, but I didn't give up. Now (some 28 years later) I am an accomplished and successful electronics engineer with about 20 years of successful embedded programming experience.
Like I've posted before, (1) immerse yourself and your way of thinking will gradually align itself to one that is compatible with programming (2) practice, practice, practice, practice! (3) if you stop trying before you succeed, you've failed.
I firmly believe that many people who fail programming at first could succeed if they kept working at it.
I also think it may be more fun to the a professional musician on the road, but there's an upside and a downside to everything!

~{;o)

Hi,
Maybe i am a fool, but i do not read the results the same way. In my opinion, the got a test before the course, and on after. so the test will allow to qualify of the course, not the students.
Obviously if 44% answer in the right way to the question they have some previous knowledge in the topic (and maybe that's why they were interested in the course).
(I haven't read the related academics papers, because I think we just need to change the computer science to something better).

Peace,

Steve

I am a third year computer engineer and work as a student lab instructor for my university. I would like to mention that from my experience, I agree somewhat with the conclusion of the study. I feel that tests like this could and should be used to determine which students MAY need more help understanding the basics. Through my quarters of teaching, I have seen the divide of students who see the code and understand right away...I have also seen students who are too shy to ask for help and as a result, never get a solid foundation of the basics. There is also several students in each group that I feel will just NEVER get it, no matter how many different ways I attempt to explain it (but I NEVER stop trying). My recommendation would be to give a test like this a week into the course, after the assignment operators have been discussed. Using the results from this, I would highly urge the students who had trouble to visit me during my office hours. As I tell all of my students...programming isn't really hard, you just need for it to "click". I am a big fan of student lab instructing, just because I know exactly what the students are going through; I've had the terrible professors, I've had the terrible books, I'm now able to adjust my teaching style to what I feel will work best for the students.

I think a lot of you may be missing the point here. The correct answer did not matter at all. After reading through the draft, I gathered that the test measured whether or not a student could hold a consistent rule set in his/her head or not. The students who did not answer consistently were most likely to be the students that did not understand that computers are dumb and will only do exactly what you tell them...following the exact same rules every time. This is a huge disadvantage when programming, because the rules do not change. Unlike the relativism most high schools teach, where there is no right answer and each problem should be looked at in its own way, where everyone can have his or her own set of rules and values, programming is very strict. The people who comprehend this are much more likely to be good at programming. That was the point of this article, not that knowing the correct answers to these questions makes you a good programmer. I think there was even a list of what all the answers meant the student was thinking at the time, most of which, were perfectly acceptable thought processes, if self-consistent. I'm just worried that something like this may be used as a deterrent for those who really don't get it at first, but are able to adapt quickly.

A large number of people commenting above fail to realise that the students were not being assessed on their actual answers to questions like the one above: what they were being assessed on was how CONSISTENT the mental model was of what they were being asked.

a=30, b=0 is an acceptable answer, but to be consistent, the student must then answer the remaining 11 questions using the same rules. If they do so, they are "consistent", and are demonstratably more likely to get higher scores in the actual course.

I'm disappointed by the number of software engineers who read this article and fail to realise this!

"but you coders seem to have ZERO respect for the actual language us humans use" ....sorry, but this coder can't resist: that should be "language WE humans use."

Great article, this explains why so many people failed out in College.

On a side note, anyone else think that this site's grey text is terrible for reading? A darker font would be much easier to read.

I've found that people who do well in high school geometry have a better chance of clearly understanding programming... Geometry proofs are pure logic. I did not do well in algebra, but got deans list and merit list in college while doing computer programming (merit list is a deans list for part time students).

I had the pleasure of tutoring many students in beginning programming courses. I am definitely a firm believer that some people can never learn how to program. It's a simple matter of brain function. Some people just don't have a brain that is suited for logic design.

Many years ago, after teaching myself to solve engineering problems in BASIC, I tried an undergraduate programming course in hopes of sharpening my skills. I made it through the first assignment, then dropped the course when I realized everyone else's program listings were 10 times as long as mine.

I never tried another class, but I've been writing working code as part of my career ever since, learning to use a variety of programming and scripting languages along the way.

I guess I'm an intermediate programmer, in the same way that I'm an intermediate
skiier. To me, programming is both a useful skill and healthy mental exercise, like writing poetry or playing chess or a musical instrument, even if you never get really good at it.

Maybe some people just don't like it or don't get it - some of the same people who can't seem to construct a grammatically correct sentence in their native language.

But it's still depressing to me that so many bright people I work with have been indoctrinated to believe they'll never be able to do anything with a computer outside of a GUI.

It is unfortunate when so called "studies" like this come out to make speculations sound like truths. This page might be better served in a Wiki and accepted as "thoughts and streams of conciousness without conclusive proof". I agree that it appears there are groups of people who just "get it", but I won't easily forget about all the social factors and variables at play here. I think the authors should consider other factors as well such as: teachers, class material, language and syntax, prior exposure to similar topics, left/right brain usage. Even though this industry has been around for some number of years, the formalities of teaching the subject are still in their infancy.

"dropped the course when I realized everyone else's program listings were 10 times as long as mine."...you're probably a really good programmer then, assuming your programs worked.

We've got a lot of old code where I work that's probably about ten times as long as it should be. Drives me crazy. Some people just don't seem to have the knack of figuring out a concise and elegant way of solving the problem.

I find fault with the belief that you can test someone's aptitude on a subject before they even begin their studies. You can test anyone's logic skills, which is what variable assignements do. They utilize the critical thinking of the subject. However, that same method can apply to any subject matter and doesn't really show an natural ability at one particular subject. It merely shows that individuals ability to apply reason across any material.

You test likelihood of success, but that denegrates those who come into it knowing very little and end up learning a lot. Perhaps even more so than those who did very well on the test. My fear is that if this theory is utilized throughout the academic community, then only those who show a natural aptitude for the subject matter will be allowed to take the course. And that even those who complete the course with a high grade, will still be hindered because they didn't show a natural aptitude for the material. Whereas someone who just barely passed the course will get higher consideration because they scored high on the tests. (Remember, potential and the results are two different things.)

The great thing about education, is that a person can come into their studies knowing nothing and walk away much wiser. It is the pursuit of knowledge that is most important, not who has a natural ability at the subject or not. By testing students for their natural ability, we are going to create a system where by people are only able to do what they show an ability for, not what they want to do. That sounds a lot like the former Communist regimes of the USSR and China.

Interesting discussion.

A few observations from my 23 years of programming and 3 years of teaching high school programming.

Sexism: In my prior life as a programmer/manager, about half my team was female. Quality of programming was indistinguishable between the sexes. In some teams the best programmer was female; in others it was male; often it depended on the business knowledge of the programmer rather than on the mechanics of programming.

Language to teach: I opted to teach Java as my introductory programming language. It provides a safe language for beginners: they can't accidentally do something that trashes their system. It can also be written and run on any of the major high school platforms: PC, Mac or Linux. And yes, all three platforms are represented in my student population. And no, knowledge of multiple platforms does NOT mean that the student is a better programmer!

Beyond that, Java allows all the basic strategies of programming: procedural approach, sequential processing, compiling (OK, so it's byte code and not a "true" compile. They're still exposed to the thought process of translating written code into something machine readable.), logic statements, loops, recursion, data types, arrays, sorting, trees, linked lists, file IO, searching through documentation for the code they need. I also require documentation (the bane of many "true" programmers), requirements gathering, testing (string, unit & full application), and good design. For my advanced class I also get into project life cycle and maintenance by taking a single application through several cycles of improvement.

Weeding out: My first couple lectures are completely from hand-outs, and the first assignments are based on that information and the class discussion. I then review the programs in class using student code (from that class or another, with the victim's name removed) so we can see examples of good and bad design and code approaches. I also put my own code up for review, including bad code, so we can go through trouble shooting and debugging. That is the most useful teaching technique I've found based on the students' "ah ha!" response. After several weeks and assignments I talk with all the students about continuing in programming or finding another area of interest. Most don't need the conversation.

BTW, my favorite first program is to write the directions for making a peanut butter and jelly sandwich. Then I'll pick one and misinterpret as many of the directions as I can to illustrate the stupidity of the computer, the importance of clear instructions.

English language: Having come from the corporate world of application programming and working with the user community, I emphasize to the students and to our English teachers the importance of clear writing skills. Writing is the most important part of working with users: meeting minutes, email, user documents - if they are unclear to the users, you are just wasting your time and theirs, and your project is doomed. Unless you work for someone who has those skills, but you won't ever be in charge of many successful projects. But maybe you only ever want to be a bit twiddler, and those of us in management appreciate the bit twiddlers who work for us. (Is that food for flaming, or what?!)

Quality of teaching: There is no doubt that there are good teachers and there are bad teachers. And while there is pressure on elementary teachers, and to a certain extent on high school teachers, to use better methods and techniques, I don't see the same urgency in the college and higher levels for teachers to keep up with current teaching techniques. In many cases they have there PhD so know what is best for us, and if we can't get it, well, then we just don't belong in the field! I've been fortunate to have had good or excellent teachers. Not that there are things I wouldn't have changed! And for the money, there are few better college classes than evening classes at community colleges in the DC area: the students are pros in their fields, looking to improve their skills or expand their knowledge base (no trade mark infringement intended) by taking evening classes. They know what they want and are very demanding, so the best teachers are assigned to their classes.

And yes, I will play with the test and see how it does in my classes. And I'll watch out for follow up studies, because this has said, "There is something interesting here."

Well, this has certainly gone on much longer than I intended, but there were a lot of points in the discussion that I wanted to respond to based on my experience.

Two items to note based on the conversation so far

> I had the pleasure of tutoring many students in beginning programming courses.

1) Bear in mind that "the camel has two humps" observation comes from people in the trenches teaching computer programming to novices. The idea that there are two populations, those that "get it", and those that don't, isn't a an experimental concept-- it's professional observation based on years of hard experience by seasoned, professional teachers.

> If someone does not know C (or any other programming language) they could easily not know that = in this case is an assignment

2) The authors specifically noted that they used the = operator because it was simpler to understand. Probably because it looks more like standard maths. However, this does make me wonder if a more explicit, verbose syntax..

> Variable a is integer
> set a = 10

would have helped at all. Probably not, but worth a shot.

Surely the question is a logic problem, it has nothing to do with C, semantics or otherwise. As posted previously, some people just don't have logical minds - and they would certainly have trouble learning programming, but the mind is a mouldable thing and I firmly believe that they could learn, given the correct tutoring and the will to learn.

Also interesting to note is that people who are good at programming also show aptitude at picking up other languages and vice versa.

It's an interesting question, but the 3rd line contains an error, a=b is clearly illegal.

To be honest I am very fascinated by the study being done here. It quantifies some of what I experienced in a beginner computer course in High School. As part of the course we were introduced to a programming called Turing which was developed as beginners programming language at the University of Toronto.

Prior to taking that course I was already exposed to logic and programming courses, and would consider myself to an analytical thinker. What amazed me about my peers was their inability to grasp logical concepts. Being done my assignments very early on, I assisted the teacher in help students to debug their program errors. These programs were littered with a large number of logic errors like:

1. Not dealing with assignments properly.
2. Not able to transcribe the requirements into the language constructs.
3. Not understanding the actual project requirements.

I think it would be even more interesting if these test could be applied to a high school level when students take their first programming course at a point where they have been taught the basics of the language... I think the results would still come out the same way because there was a group in my class that would not be able to understand the problem even with additional programming teaching.

Just to reitterate the language problem, I showed the test (out of personal curiosity) to several people who have never programmed, and they all got very confused, aggitated and thus reluctant to continue with the test over the fact they failed to understand that the = was an asignment, and thus the question made no logical sence in any form. After I explained this, they were alot happier.

One thing that no one has taken into account is learning style. Almost all public school and college instruction is geared to ONE kind of learning style (this is one of the problems with mass education). This "study" completely ignores this factor. Another factor that this study ignores is that most Computer Science programs do not teach the process of programming, but rather syntax, flow of control and encapsulation methodologies. The result is that students know "what," but not "why."

Forget teaching students syntax right off the bat, teach them how to solve problems, teach them how to break down complex systems into groups of interactive, simple systems. For instance, my oldest son wants me to teach him how to program, but he is still too young. He is not yet able to reason from the complex to the simple. He also does not have enough math to understand how to make logical comparisons. So should I teach him how to write "Hello World" on the screen? NO! There is no value in that way of teaching; it is pragmatic and therefore useless. Instead, using real life situations, I am teaching him (many times without him knowing I'm "teaching" him) how to think logically about things and how to see a bigger problem as simply a sum of parts. If you want to learn how to draw a realistic looking tree, start with understanding how to draw a leaf and even then leaf drawing can be broken down into simpler parts.

If a student does not like this kind of problem solving, he will never be able to look at a problem and write a program that really solves the problem (because he does not truly understand what the problem is). That same student may "get" assignment and recursion, but they if they do not know how to apply it, what good is the knowledge? The reason why there are so many bad programs out there is because the market is flooded with this kind of programmer.

In my own education, I had this same problem. I would have failed the test that is being put forward in this study. I found my first year of comp. sci. in high school boring and I did not really understand what was being taught. Why? Because I did not learn in the same way that I was being “taught.” The only reason I passed the class, is because I had friends in the class who mostly understood what was going on and they let me study (not copy verbatim) their programs (yes, this is still cheating and it was wrong as well as pragmatic – I should have asked for help from the instructor which probably would have done like those doing the study and told me to quit the class because I was not predetermined to be a programmer). My point is that I learned programming by pouring through programs and deciphering them. By looking at how several people solved the same problem, I was able to understand the problem and the best way to solve it. By year 2 of comp. sci. all of my friends (who were all smarter than me – at least according to the public school system’s grading methodologies) were coming to me for help. And now (16 years later), as far as I know, I’m the only one still programming.

Am I the world’s best programmer? No, not even close. There are many reasons for this many of which are beyond the scope of this reply, but one of these reasons is lack of specialization. To be really great programmer, one has to be able to specialize in a small set of programming languages and then learn every nuance of the language so that he can apply the appropriate features of the language in developing the best solution to the problem at hand.

A comment to those who think you need to know programming to answer the first question on assignment... What a joke.

This is basic math. NOT programming. You have 2 variables and you are assigning one to another. Simple as that. Grade 7 math taught me all I needed to know for that question!

And yet, there are companies out there that would rather hire someone who has graduated college with a computer science type of degree, irregardless of how skilled of a programmer they are.

Would you rather hire someone who barely squeaked by a C++ class, or would you rather hire someone who has had to learn how to program 100% on their own for many years and has a proven track record?

For a lot of places, it is still the college graduate that gets the preference. Maybe it's a CYA move...I dunno.

Readers should note that the dedicated professional teachers all recognise the two humps.
I have attended many teaching conferences where young aspiring teachers wonder what they are doing wrong. Eventually, they recognise the two hump problem. Some students just don't get it.
To all those who are insulting the article's authors' teaching ability, please consult the literature on teaching introductory programming.

And point of the test was not to find a way to discriminate between goats and sheep; as pointed out in the article (please read it) this was a complete accident. This was discovered by subsequently analysing the results. So this "so called study" is simply pointing the way. The lovely thing about this paper is it presents what they discovered and offers entertaining insights and opens up a very useful approach for studying potential programmers with a view to improving teaching. It is far more useful than so many of those so called "real" studies.

I completely agree with Scatman Dan, "I'm disappointed by the number of software engineers who read this article and fail to realise this!"

============================================

Whats the answer to the question anyway? -BG

============================================

or...

Those who can, do.
Those who can't, teach.

As a linux hobbyist / non-programmer, I find this discussion interesting; especially in that no one apparently posted an answer to the question.

As a non programmer I would have checked off:

a=10; b=20

In other words, the 'rules' of this particular program are such that 10=20. I know that 10=20 wouldn't make 'sense' in a basic math world, but I'm assuming that in programming, you can define your own rules, even if it wouldn't be 'true' outside of a programming environment. So...am I correct? Do I have the 'programmer's mind'? ;)

Ben,
If you have a ruleset that you would then apply to another similar problem, then you 'pass' the test and have the 'programmer's mindset'.

Your rule set doesn't match mine.

I read it as follows:
Set A = 10
Set B = 20
Set A = B (ie. move the value of B, to A)

Thus, both A and B are now equal to 20.

While I have been programming for 20+ years now, I've never used C (learned in Apple Assembler and Basic and Pascal, then moved to the AS/400, which is RPG)

> Would you rather hire someone who barely squeaked by a C++ class, or would you rather hire someone who has had to learn how to program 100% on their own for many years and has a proven track record?

I'd hire someone who did well in school and did some self-study. Believe it or not, those people exist. I've hired a few. Those are the people with passion - the ones that care about what they do with their life. Everyone else is a damned hack.

> Those who can, do.
> Those who can't, teach.

I've met plenty of people who can't, try to do.

I've also met many really good programmers/analysts that teach.

I agree with some that algebra, although important is not a necessary requirement. I actually found that I was better at logic and syntax of the given programming language. The mathematical formulae will usually be supplied for the programmer anyway. So in my case I found logic and syntax to be important. I found I was very adept at debugging programs. No I found I was a programming sheep. It also didn't hurt that at the age of 16 I graduated from a Unit Record School, I am now 55 with a degree in BDP/CIS. I love programming.

That test was ridiculous! It is so biased towards the imperative programming!!!
It does little to test logic.
Maybe the problem aren`t the students, but the whole "think like you're a fSCing Von Neumann machine." Of course, so many C++, Java, C, Python programmers out there won't even understand what I'm talking about.
So instead of identifying the disease, they choose to name students "uncapable." F******G CRIMINALS! MORONS! It seams the mental disease called imperative programming has caught everyone, except the few of us who choose to reamin sane and inside Computer Science, but outside marketing and corporations intrduing in on the field, and taking over...with their substandards...But so many people in academic departments are seriously braindamaged.
LOOK AT THE F******G RESEARCH DONE IN THE LAST 20 YEARS AND YOU'LL SEE...EFFECTIVE COMPILERS FOR NON-IMPERATIVE FUNCTIONAL LANGUAGES...VIDEO GAME COMPILER WRITTEN IN LISP...OR CLEAN...SML ALMOST CATCHING UP WITH C...YOU CHOOSE TO STAY STUPID!
A functional proagrammer would fail it, because of assignment.
Maybe Math students would fail it too.

Might be interesting to substitute
a == b;

He-he.
One of the typical pitfalls usually encountered
in conditional statements -
if (a = b)
versus
if (a == b)

Roger

a and b both equal 20 as the operations are executed in sequence

1. The integer of a - presumibly 11 > a > 10 is a valid assumption is calculated to be 10

2. The integer of b - presumibly 21 > a > 20 is a valid assumption is calculated to be 20

3 The result of a is then made equal to the result of b thus rendering a = b = 20

I think it depends on the person at the end of the day - some can and some can't - also depends on the language that one has been exposed to . . .

Steve

Looks like a paper describing the failure of professors to teach the subject they know (which is likely not education).

Perhaps if more of them (and I'm sure some do) focused more on conveying the material instead of carbon copying their favorite textbook to a chalk board, more students would know important things like what programming language the test is in, and what the symbols mean in that language as well as on the hardware of the machine it's instructing.

Who did well? The kids that learned something on their own first, beacuse thats obviously all they come out with!

RE: programming sexism.

There is a theory that women are smarter on average, but that men have more variability/larget standard deviation. Women cluster around a higher average, but men have a higher chance of being extremely smart.

This has beared out in my experience as a 34-year-old programmer. Women are generally good, but I have never met a superstar female programmer.

For example, does anyone know of a framework author/architect that is female? The people that come up with Jini, or Ruby on Rails, or Hibernate, or the like. I can't recall a single female author of groundbreaking ideas, either theoretical or practical.

I agree with many people that the given C example is riddled with hidden lessons/assumptions that actual programmers have learned. The '=' as an assignment is confusing since equality is a test, not an implication of transferrence or copying.

About the only thing missing here is ... Hitler!

Related to comment that implied 'No good woman programmers' Dr. Grace Hopper (COBOL and other early work). She as also coined 'debugging' since she found a moth in a relay switch on early computer. How about Ada Lovelace [The Ada language was named for her] the very first computer programmer ! Do google search on each for details.
I have been a computer programmer for 48 years [I still work for clients] and enjoyed the various comments etc.

"I agree with many people that the given C example is riddled with hidden lessons/assumptions that actual programmers have learned. The '=' as an assignment is confusing since equality is a test, not an implication of transferrence or copying."

I couldnt agree more. The test is already biased towards people that have programming knowledge, (especially C related languages). It isnt doing non-programmers any favours, not exactly a level playing field of a test.

PS Jeff - shouldnt you get a few words to enter when posting? i'm sure a spammer could work out "orange" is the word quite quickly :-)

"I agree with many people that the given C example is riddled with hidden lessons/assumptions that actual programmers have learned. The '=' as an assignment is confusing since equality is a test, not an implication of transferrence or copying."

I think the programming language is lesson important to be concerned about than the ability to just let things stand for other things. Choosing a language where 'a' is the assignment operator and other symbols are chosen compeltely randomly would be bad, but using = at least gives some people a clue of what is going on. I think it is key to note people using the = consistently is what is being looked at(I think) not necesssarily them using the = operator correctly.

(let ((a 10) (b 20)) (let ((a b)) a))

Gives new people less of a clue as to whats going on that the c code does. It would be interesting to see what would happen with a test like that.

You could even go with something like pascal assignment using
int a := 10;
int b := 20;
a := b;

But i think the slight ambiguity in the = operator actually tests whether or not a person will use that symbol consistently(even though it is tempting not to) which is what is really important.

int a = 10
int b = 20
a = b

In my mindset, a = b = 10, but then again I'm a basic programmer anyway. (Even tho that's sort of like me blaming almost failing English 10 on the position of Mars)

I enjoy coding. I enjoyed algebra but did not like geometry. My response to geometry was, "Prove it? What do you mean, prove it. It is intuitively obvious."

Maybe further instructions would help? Although I think the test would work, perhaps something like:

"where the equals sign is an operation of some kind. Decide what operation you want this to be an apply it consistently across all the following problems."

Or would that be explaining too much?

What pointless emphasis on the mechanics of software! Writing software is a creative process, much like writing a story. Like any other subject taught in schools, you love it, are indifferent to it or you hate it. Grasp of a language and programming concepts are the required underpinnings for being a software engineer, but do not begin to describe the experience of creating successful programs, any more than a comprehensive grasp of English will make you Stephen King. It is facile to try to identify a successful formula for churning out talented logicians, mathematicians, writers or any other human endeavour. It simply identifies the limitations and prejudices of the so-called "educators" conducting the study, and their fears of being caught out if they depart from the line of pretending to teach the self-motivated and ignoring the unmotivated.

How do people with no capacity to grasp basic programming concepts function in real life?

Programming is modelling; It's a map / territory thing. Someone who can't learn to program likely can't read or draw a map of any kind. How do they engage the real world? A human needs a sufficient understanding of real world processes to draw a map of those processes with enough truth to predict the consequences of their actions. A grasp of the rules of gravity, for example, is quite useful.

Interesting - the last I read on the the nature/nurture debate splits things at 50/50. Looks like there is some hardwired circuitry for programming that is either there or not.

The naysayers should realize that the article is based on experience not an educational theory.

What an odd question. Actually, I have no background in programming, and will start my first programming class in a month or so. Logically, a cannot equal b, so you have to look for other options. Without any context and without any history to base your decisions on, the answer could be any of them, because it is impossible to determine the value of =. = could very well be a modifier for the word 'int', then create a completely different meaning. If 'int' meant 'multiply this by 0', that changes the whole context, and a=b is no longer a logical difficulty. I'd say they're insinuating that there's a definition of what 'int' and '=' mean, and expecting you to know that. Or... what if ; meant stop doing everything here. A would = 10, and b would never occur, and the closest answer for b would b 0.

My background is actually in psychology, and I'm going back to school for UI design. That's a straight up invalid test, similar to the 'IQ' tests that favor one culture over another because of the language used. They're assuming that the person has at least seen a programming language before. Personally, I wouldn't want them as teachers.

Bring back cobol

01 A PIC 99.
01 B PIC 99.
MOVE A TO B.

Its not rocket science.

As someone who has been programming for a while, I'd like to say that that is a pretty bad question. Sure, it's glaringly obvious to us, but it doesn't just test assignment to variables; it also tests the ability to comprehend syntax by just looking at it.

Lvalues and rvalues are a trivial concept, but it's pure convention that says "the left side gets the result of the right side." It's implied to some extent when the variables are initialized, but this tests the ability to immediately comprehend foreign syntax correctly. If I had never seen code before, I'd say that that either they're both 10 or that they're both 20 - but I wouldn't be at all sure.

I think it's certainly true that some people have brains that are "wired" for programming, and some don't - like anything else, it's a talent that people have to a greater or lesser degree. However, the concept of assignment is simple enough that it can be taught to almost anyone with not too much difficulty.

I find it mind-boggling that people couldn't learn the concept of assignment after 3 weeks of class, to the point where I think the error is as much in the instructor as the students. Assignment is as simple as putting an object in a box - that's it.

I've had to take a "beginning computer programming" course before, already knowing the language, and I've found that instructors, even the good ones, can be pretty obtuse to beginners. On the first day, students don't need to know that a variable represents a specific location in memory, how it's possible to overflow a variable, or the representation of a floating-point number. Instead, they should get a solid idea of assignment in the abstract: you've got boxes that can hold one piece of data, and each box has its own name. If you put a new object in a box, you've got to take the old one out first.

Once the students have CONCEPTUAL understanding, you can talk about implementation, syntax, and everything else. Talking about the stack and the instruction pointer on the first day just makes students' eyese glaze over, and reinforces the idea that "nobody but a supergenius could ever learn this stuff!" (direct quote from a person whom I helped in a programming class, who went on to understand the basic concepts pretty well).

Teach the concepts, and all else will follow.

"I find it mind-boggling that people couldn't learn the concept of assignment after 3 weeks of class"

That is not what this study is about. At the first test, the students had no experience with programming. That meant that to solve the questions, they had to create their own model/ruleset about the way assignment works. Students who were able to create a model and use that model consistently for all the questions in the first test, scored on average better on the course exam, than students who didn't consistently use the same model.

The key word here is consistently. Did the student create a model and stuck with it, or did he use different models for different questions (or was he just guessing). What particular model he used didn't matter.

The course exam probably didn't just test understanding of assignment, but also if else constructs, loops, and other subjects. The point is that consistency in the study's test is a good indicator of how well the student would do in the exam.

So, where's the FRIGGIN test???

And why is the captcha always Orange? Is this blog based in Florida?

Some comments here (and elsewhere) attack the test's validity on the basis that it requires the taker to have some prior understanding of a programming language or at least programming concepts.

I'd like to support Thijs by reiterating that this is NOT what is being tested - the marking criteria, as can be seen at http://www.cs.mdx.ac.uk/research/PhDArea/saeed/ are based on the *consistency* of response and have nothing to do with the "correct" answer under a given programming paradigm.

It is not culturally specific (or at least the culture that it specifies is incredibly broad and hardly exclusive); all that is needed is an understanding of the relationship implied by an equals sign, which anyone with any sort of background in mathematics can deduce in this context.

Okay, just finished reading all the comments and the actual paper and results. Just to give you some background, I am someone who made it about half-way through a CS degree before dropping out due to a divorce, I have been a professional programmer for about 8 years, I've written successful products, and programming frameworks from scratch, and these days I can keep up just fine with PhD CS folks (not trying to brag, just trying to keep my background accurate and short).

Let me see if I have this right: The results indicate that there is a correlation between people who can derive a consistent mental model for answering these questions, and people who make it through an introductory programming course.

First of all, I think I appreciate the objectivity of the paper better after reading it--at first, like many posters, I saw this experiment as extremely biased towards people with a previous understanding of the assignment operator. Now I see how that is eliminated that by allowing multiple "right" answers so long as a consistent model is applied.

All that said, I think there are several really dangerous assumptions that this paper draws from these results, and which it implies in its conclusion.

1) That the "consistent mental model" required to learn programming is something inherent, that learning to think in the consistent manner necessary for programming cannot be taught.

2) That the 61 students across only TWO courses surveyed at TWO universities is sufficient data for drawing such conclusions.

3) That these two courses (or any intro CS course) provide suitable instruction for students to learn the "consistent mental model" necessary for programming.

Now obviously this last point happens to line up with already established evidence (based on far more more data) that apparently very few intro CS programs succeed in teaching basic programming to all students who enroll (the double-hump phenomenon). But there are a lot of other factors at work here:

1) Most universities don't have the facilities to support getting everyone who enrolls in their CS program through the whole program.

2) Most intro CS classes are over-crowded.

3)The previous programming experience of students in intro CS classes is extremely diverse, with some students already being semi-skilled programmers, and others having no exposure at all. The lowest common denominator is REALLY far away from the highest.

While one could argue that all this is irrevelant to the research, my point is that these conditions are prevelant (though not univeral) in colleges across the world, and that it affects the way that intro CS teaching gets done. This study rushes to the conclusion that these results point to something inherent about the students, without considering it may have something more to do with widespread conditions placed on the classrooms and teachers. Just because a lot of teachers in the trenches have experienced the double-hump phenomenon and couldn't seem to beat it doesn't prove it is inherent to some wiring in the students. If you live in the Arctic, you'll quickly assert that all bears are white--that doesn't make it true.

I'm just supposing here, but I bet those students who need some practice and attention (perhaps some help that would go towards learning a more consistent mental model) don't get the attention and encouragement they need. They are thrown into the midst of cryptic syntax, alongside a bunch of folks who already seem to understand it naturally, and they get discouraged very quickly because they don't pick it up as fast. The classroom is huge, the teacher has way too many students, and doesn't have time to bring the stragglers up to speed. Especially when the stragglers may become quickly demotivated in an environment where the subject matter comes so easily to some students. Just my opinion, but I think this phenomenon comes much closer to explaining the double-hump.

Of course, I do not submit all this not as a counter-theory--I have no data--but it is an argument that this study is pretty anecdotal and not good science. Many extremely subjective statements are presented as fact and the author's bias is made obvious in the introduction.

"We point out that programming teaching is useless for those who are bound to fail and pointless for those who are certain to succeed."

That's a pretty broad statement, don't you think?

So here are some anecdotes of my own (and at least I present them as such):

I taught two semesters of programming in a continuing education program (voluntary, small classroom, no grades, no pressure) and everyone could do basic programs by the end of it. No drop-outs. Of course, the make-up of my student population was biased because no one was trying to meet "core requirements"--everyone wanted to be there. Nonetheless, one student had an extremely difficult time, and required a great deal of one-on-one attention. As another teacher described, after enough practice, one day, it finally just "clicked" for her.

What is the difference between this student and the thousands that drop out of intro CS courses every year? I'll tell you I think it has nothing to do with some hard-wired gene for programming. That is just old-fashioned bigotry.

Personally I think there are two simple factors:

-The teacher's skill and experience with teaching, their patience and motivation, and their time-capacity for encouragement and one-on-one attention
-The students attitude and motivation to learn the material

Those two things added up, and I believe there is very little that cannot be learned.

Hello, before I lay down my views on the subject, a few things about me. I'm 25 yo, and I studyed programming and computer science for 12 years, starting from fifth grade. From 5th to 8th there were 2 hours of programming a week, on old Z80 / BASIC machines - very "light" (so to speeak) stuff - basic programming, basic control structures etc - and if memory serves, the course was suspended during 7th and 8th grade. From highschool(wich was CS) began the real heavy teaching, pascal, structured programming, intensive etc. And then there was the university - electrical engineering and CS. I am currently working as a programmer, employed by a software company - and I am very successfull in my work. Here are my opinions, based on my personal experience as a student of various institutions (I never taught):

1) There is definitely a programming gene, a natural, inborn ability to program. And yes, those who lack this will never be able to program computers. I still remember the first programming course - I had seen and played on computers before, but had no ideea about programming. From that very first course I was like: "hey, I understand this perfectly, this makes perfect sense, this is so easy - this is NATURAL". And I was perplexed that the others couldn't understand programming. After only a couple of courses, I was one of the two or maybe three "freaks" in the class, those who could understand programming perfectly. Thoughout highschool and university, I have seen bad teachers, excellent teachers, good and bad teaching, but the double hump was allways there, and nothing could be done about it. There were those (few) who could, and those who couldn't (well eventually, after years of hammering, they could write basic programs, and had a general grasp of programming, but not enough to become programming professionals - most of them don't even work in IT). And another true thing - the teaching is too slow, tuned to the rythm of the "goats". The "sheep" were devouring the books, learning several lessons in one session, were 1-2-3 semesters ahead of schedule, reading 10th grade manuals in 9th grade, languages, general algorythms etc. No other course was like this, there were no students lo learn math ahead of schedule, or physics, or whatever. Meanwile, the rest were struggling and going through all imaginable hell. I was perplexed - why can't they learn, what is so difficult, what is their problem? In the end, many were relieved to have finished highscool, and never have to see programming ever again.

2) About sexism - nobody can tell me there is no link between gender and programming ability. I once watched a documentary about the human mind, basic psychology and stuff, and one ideea was this: men have an inborn ability towards exact, mathematic, and engineering fields, and make "excellent nerds", while women are far better at psychology, human behaviour and interpersonal skills. This is clear as night and day. CS highscools were almost all male with as few as 2 girls out of 30 in one class. CS and electrical profiles in the faculty were all male. Languages, psychology were almost all female. I recently went to job interviews to several software companyes. The programmers/technical were all males, the same "nerds" and "geeks" types I saw in highscool and faculty. Human resources were all female, all of them.
As a sidenote, both my parents are programmers, my father is an expert, my mother barely strugles with some basic database software.

3) I am not shure about the test. The results sound interesting, but I am not convinced. I tryed to imagine how I would have fared back then, before any programming course - and frankly, I don't know. I do believe that programming is special, set aside from all the other objects, and needs specialised testing and evaluation - and specialised aproach of all aspects. As I said, I am not convinced that some test for some basic mind patterns could be the panacea. What I sugest, before admitting students, teach some basic programming, logic, algorythms - just a couple of courses - and then do lots of evaluations. As I said earlier, the sheep are identified very quikly.

4) For a teaching, introductory language, I swear by Pascal. People say it's a "comunist language", but it forges a coding discipline and clarity that I personaly value greatly. I consider myself lucky to have studied pascal, before moving to c/c++. Hated JAVA...

These I my personal and highly subjective opinions. Please excuse my english...

Hi, I'm a professor of physics. And a woman. There's definitely another factor at work here: confidence. If you were unconfident you would probably switch up the rules you were using to answer the questions just in the hopes of getting it right some of the time. That says nothing about your innate intelligence, just your confidence. I've given a lot of oral exams to students in a 101-level course and have found that the unconfident ones often know the material but it needs to be coaxed out of them because of the barrier that lack of confidence presents.

I'd also like to add a few other things that need to be said:
1. This experiment does not do a thorough enough job of testing the possible parameters. These profs should collaborate with a wide range of other CS profs at different colleges who teach classes of varying size and use different teaching techniques. I know a woman CS prof at U of Maryland who can teach anyone to program.
2. If the authors of the study really want to investigate this fully they should do several case studies where they do in-depth assessments with students from all over the double hump. Ask the students what their background is with computers, how comfortable they feel using them (ie, confidence level), etc.
3. Dragos, watching a documentary does not make you an expert. Any and all "facts" on television carry a bias. Part of the lack of women in your CS classes has been the slow-to-change phenomenon of available mentors. As more women become the teachers of CS classes more women will take them and feel like they belong. Also, in many cultures (including US culture) the patriarchal way of life keeps women down and discourages them from achieving more than their male counterparts. These seemingly small effects add up to a society that lacks women programmers. It is a social commentary, not a genetic one.

-Kristine

What a weird choice of "sheep" and "goats" as an analogy, and what a weird way to assign which were the sheep and which were the goats. Anybody that knows anything about sheep and goats knows that sheep are really really not very smart at all, and being called a sheep is a bit of an insult, whereas goats are much more clever and resourceful, much harder to keep confined, impossible to herd, devour anything and everything.

Given that managing programmers has commonly been described as "trying to herd cats", mapping good programmers to sheep seems particularly inept.

in response to dragos:

1) There is definitely a programming gene
re:
The problem i see with this opinion is that it is based on experience, but it doesn't take into account nurture. Perhaps in your first programing course, you had already been introduced to logical relationships in one way or another, and understood some basics about giving instructions and potential pitfalls; whereas the individuals who couldn't grasp it, had never really been introduced to these concepts. It could be that we are trying to teach a baby to run before they know how to crawl... which would only result in lots of skinned knees and a few concussions. Since you already knew how to crawl, it was easier for you to stand up and run.

2) gender differences
re:
Documentaries can be wrong, and a lot of them have been. I could argue, in the same manner, that since most girls play with dolls, it is the doll, not gender, that causes a lack of interest in programming.
Could it be that because as a culture, we assume girls will like pink and dolls, and then go ahead and stick them in a pink baby room with doll, that they like dolls because of the situation, and not because of a predisposition? How many countless families perpetuate this view, and thus perpetuate the situation in our culture. The classic case of self-fulfilling prophecy. Because we assume girls have this predisposition to psychology, we (as a net sum) unknowingly force them in that position.
In my highschool, when a conflict arose, boys were given gym, and girls were given home-ek... is it no wonder that most of the boys at my school assume girls liked home-ek and vice versa. It really had nothing to do with their predisposition, but instead having to do with the presumptions of faculty. Is it that far off basis that, as a whole, most schools do the same thing when it comes to psychology and comp-ed courses; thusly creating this mass assumption that boys are better at it than girls?
Correlation does not equal causation.

3)
programing is the act of instructing a computer to do things, and using the application of logic to make it do things that you want it to. Instructions is a form of communication, in essence, you are coaxing a computer to do something for you. And logic is an abstract form of looking at how we solve problems and arguments. It typically takes a lot of logic to preform well in math. And the instructions usually require an understanding of how a computer works (ie, writing to the appropriate audience).

4)
I bet your code isn't very object-oriented.

I've seen a comment here that complained about programming teaching being biased towards the imperative paradigm, and making it the supposed reason for the situation where so many people just can't get it.

I'll admit that I'm a novice at functional programming, and I know almost nothing about logic programming, but anyway...I still can't understand something: Is it really easiest to understand everything in computing in terms of functional programming?

Say, how can you explain a data structure such as a hash table, or say, explain how a garbage collector operates at runtime, with the functional programming mindset?

Another thing, why is it that pseudo-code, the preferred formal method for describing algorithms [at least in academic papers that is], is distinctively "imperative"? I'm just talking out of my own experience here ... Maybe that has changed in recent times? Almost all of the programmers I know are more comfortable with imperative programming, though there is no argument functional programming is vastly more expressive/powerful in many cases. The best programmers can think in whatever paradigm suits the problem best, but I can't think of a single programmer who can easily understand functional programming, and still struggles with imperative programming.

I remember my ( Stage 2 ) lecturer talking about some code a student was having trouble with :
--
int i = 0;
while(i<100)
anArray[i] = 0;
--

Now, this was in a second year course, with 2 pre-req papers, both of which cover a C-like language.

Personally, i think that these crap^W bad programmers exist because of the flawed teaching system. The stage 1 courses cover OOP, indentation, drawing flowcharts, comments, GUI buttons, etc but miss one important thing : teaching students about what code is. E.g., what happens when you run "a = b;". Students who don't instinctively get this fail because they never learn it properly and instead spend hours messing around in a circle. Also, at a higher level i see students who abuse recursion, or write f-ing massive switch statements when a simple bit of bit-or/and-ing, etc would have made it 2 lines.

Once i saw the source to this chess game a student was writing, he had used a separate array for each piece for each side. So he had a blackPawns array, a whitePawns array, and so on. Yep. So you can imagine what a mess the code to draw the board with these 12 arrays was. He hadn't done the code for the simple A.I. the computer side was going to have, because (surprise) it was going to be very hard. So i recommended the he use a 8x8 array of Pieces for the board, and have the Pawn class extend the Piece Class, and have white/black flag(s), empty spots are null, etc. Right ? (Hopefully) This is a pretty simple and standard way of writing it. His first response was "How do you check if a spot is empty?", i was about to say that you check if the element is == to null. His way of doing it ? "Catch the NullPointer exception." He didn't take my advice to use a 2D array for the board, and as far as i know is still trying to write the huge amount of code to handle the movement checks or A.I. To give you an idea of the extra work it would need :
His code :
bool isEmpty(int x, int y){
for(int i = 0; i < ...;i++)
if(writePawns[i].getX() == x && whitePawns[i].getY() == y) return false;
[ copy + paste * 11 ]
return true;
}
My Code ?
return board[x][j] == null;

I find it curios that only one poster commented on the imperative nature of this test.

It would be interesting to see what results would follow if they'd choose a purley functional model of programming.

I can even imagine that the dichotomy isn't between ability to program or not but rather affinity towards functional programming v. affinity towards imperative programming.

as far as i can see the test has nothing to do with imperative programming. one could devise a mental model which concurrently processed the 3 statements and still pass the test IFF you use the same model on the other questions.

I think the test should be in some meta-language pseudo-code because intelligent people good at algebra and math but have never touched a programming language in their lives doesn't have to know that in something called C programming language "=" is an assignment operation. Maybe something like.

BEGIN PROGRAM
x <- 10
y <- 20
x <- b
END PROGRAM

What are the values of x and y at the end of the PROGRAM.

People are going to lynch me for this, but those inventing pseudo-language are a bit silly. There's already a perfect language for this: COBOL

Granted, my COBOL is incredibly rusty, so I probably can't write code that compiles here, and it would be best to leave the division separators off, or, in fact, the whole data division, but:

MOVE 10 TO A
MOVE 20 TO B
MOVE B TO A

With a note that 'MOVE leaves the first value intact' if needed. (Or make it 'COPY' instead.)

In fact, this is basically what COBOL was designed for...having non-programmer readable programming language, and the reason it's such a joke is that, as this article points out, that doesn't actually work. Certain people simple cannot parse programming languages, no matter how simple the languages are.

It's not a matter of intelligence, I've known very very smart people who passed programming course by 'faking it' by copying and pasting code they knew worked, and I've known pretty dumb-in-other-respects people who could write code just fine.

'Sequence' is a big one that programmers who have not tried to teach simply won't believe, but there are people who simply do not order statements in their mind when evaluating them. They know to do it, they just look at blocks of code as a gestalt, and see a=10, and that's the right answer. This isn't due to intelligence or even knowledge level, it's something like dyslexia. Just like people with dyslexia can manually go letter by letter to get a word, they can manually run them in order, but that's very hard when loops or recursion or functions are involved.

There are, indeed, entire groups of people who *cannot* be programmers, will *never* be programmers, and really should be encouraged into some other line of work at the start of their programming education. Just like someone with dyslexia probably shouldn't be a copy-editor.

Now, if someone wants to star in their own made-for-TV movie about how they overcame hardship to do what they loved to do, that's one thing, but honestly, most of the people just heard their was a lot of money in computers, and should be firmly encouraged into law or something else where they aren't going to always be struggling just to understand what's going on.

Hrm. No preview. Well, let's hope this is right.

Let the compiler/assembler/interpreter sort it out. Everything else is subjective.

"...not a perfect divider of programming sheep from non-programming goats..."

I don't like that for some reason. I think we're programming goats, and they're non-programming sheep.

Actually goats are the more intelligent animal. Sheep are mindless. While from the perspective of archaic theology that values conformity over independence or intelligence, sheep may be preferred, the metaphor should be reversed for dividing programmers from wannabes.

I have not seen it mentioned, but the US Gov has known about this since the 60's. They still use the same test for programmers, the ADPE ( Automated Data Processing Exam ). It consists of four sections, word association, numerical sequencing, memory, and dimensional correlation.

The failure rate for the tech training was around 80-90% as well.

I think it's interesting that so many respondents seem to see the mere possibility of a test that *may* reliably indicate programming talent or the lack thereof as being somehow threatening or pejorative. Have we really come to the place where an individual is lessened somehow if they don't possess all talents? I guess I should really reexamine my value to society since I'm a pretty good programmer but I can't carry a tune to save my life?

One of the things that intrigued me about the study is that it reminded me of discussions I've had with my wife. She is a top-notch educator with 30+ years experience and she has often mentioned that she thinks our institutions of higher learning should have an effective way to detect and discourage education majors who have absolutely no talent for the vocation. Her reasoning is that it is a disservice to the college students to allow them to spend so much of their time and money pursuing something they will never excel at and that it is also a disservice to the future students who wind up in their classrooms. I think the same reasoning can be applied to CS majors who have no talent for the trade. They suffer because they don't measure up in their jobs and those of us who come after them suffer when we spend our time cleaning up their mistakes.

I think it's important to strive for excellence in all areas but there are some areas where individuals will never achieve excellence because those individuals are not well suited for that particular type of endeavor. That doesn't mean we should prohibit them from pursuing those goals if they're set on doing so but it might be helpful to them to know that it's likely to be an uphill climb; we can leave certainly leave the final decision to them.

Mr. Bauman,
I agree with what you and your wife say, however, I don't think you realize why it's not happening: Departments need numbers of students to get funding, building space, faculty, staff, etc. So they suck in every student they can and help them as much as they feel like whether they have a shot or not.

I think it's reticent of one of the biggest problems with bureaucracy: It puts money where it's needed in strange ways that have nothing to do with the the industry and everything to do with the number of people and their ability to fill out useless forms. And it rewards bad spending!

It's possible that not every school has this glaring problem and that some may actually deeply care about the students well being and education, but I'd bet most are more like this.

Also, to other commenters: You're not reading the study. Their conclusion was that the group that fairs the best is the group who finds a consistent model (whether it's C like or not) and sticks with it. They also found that not much changes after they understand the syntax.
And if you really want a familiar syntax you should use an algebraic syntax:
a=5
b=6
a=b

Unfortunately, this still doesn't make sense as algebra is formed without statements. This is more similar to formal proof language where you setup some variables early on (except you don't assign numbers, obviously).

However, back to your idea Mr. Bauman. I think the way to do this would be private industry placement exams taken between high school and your sophomore year of college (it doesn't make much difference if you take the wrong major in your freshman year, especially if you're in the right college -- as in college inside a university system).
So students could pay for exams that would say if they have aptitude for a given field. You could add a general aptitude test to give them some direction, but most students would likely be able to guess based on their personal history.

Charge $50 for a test like this and give nice results. I imagine you could get quite a few students on board! You might need to hire overpriced PhD's to "back" your tests though, in which case you can charge $150 for the exams.

My take on it is that the Teacher is inexperienced. I've not seen that sort of results in the night classes I taught for Harnell college in King City, CA and surrounding areas. At least after the first 3 or 4 years of teaching.

I believe that most "intro to computer" classes are taught by graduate students. As a group they tend to assume that Everybody already knows the basics. By the time they grasp how to teach the novice, if they ever do, they’ve moved on and a new inexperienced teacher takes their place.

Bob Durtschi

I think all the comments to the effect that "a = b is bad syntax" miss the point entirely. The test isn't about whether this represents good syntax or not - it's about whether the student will do well in CS or not. You can argue all you want about this syntax, but if this test is a good predictor, that's all that counts. If it's bad syntax, then apparently an intuitive understanding of this very bad syntax is a good predictor of success in CS - the fact that it's bad syntax is entirely off the point.

For the record, I never thought it was bad syntax. Any math person understands things like "let x = 15. Then x+5 is 20. Now let x=20. Then x+5 is 25". Nobody gets confused and says "Wait! You just said x=15 which means it can't be 20!". On the other hand "x=20+5. Then x=25. Oh, wait, now x=30." is confusing. It just depends on whether you're using "=" as a declarative statement about the permanent and necessary value of x or whether you're using it as a verb to consider x to have a value of 15 for the moment. Both are used in standard math without confusion so I never really understood people's confusion about the verb sort of usage in languages. Some languages us "let" or "set" to clarify, which I guess makes it a bit clearer that the verb form is being used, but it seems a bit unnecessary to me.

I like the guy's comments about meaninglessness. I remember trying to teach my mom a bit of programming. If I would write down "age=20" she would immediately pipe in "no, that's not my age!". I'd try to explain that it's just a name but no amount of explaining would do. Finally I'd change it to "x=20" in an attempt to divorce it from the meaning she was assigning it. At that point she'd ask what "x" was. I'd have to tell her we were using it to represent her age and we'd be back to "but I'm not 20!". She wasn't able to accept that "x" or "age" is meaningless in the context of the program and had to know what it "stood for" at which point all sorts of real life connotations would flood in making it impossible for her to understand what was going on.

This problem has puzzled me for years. I learned programming myself, with able tutoring from a friend. In turn, I've tried to teach others, some smarter than me, some not. There were some that picked up the concepts and then built on them. Others had great difficulty with the basic principles - and even when they'd gotten them, could not seem to carry forward with what they'd just learned.

I've always wondered at this difference. Looking back at my own history, I've also been curious why certain populations thrived in the school environment but never seemed to really grasp complex subjects - and vice-versa, some of the best minds felt totally out of place and suffered during their school career.

This essay really opened my eyes, and explained so much of why I felt out of place in school:
http://www.reciprocality.org/Reciprocality/r0/Day1.html
(if the comments don't allow links, google the words
mappers,packers,software and look for the link "Thinking about Thinking".

I don't attach a value judgement to mapper vs. packer, each have their place; and of course no one is all one or the other, it's a continuum. Still, it's a useful mental model when thinking about the issues this blog post has raised.

I have the advantage that I know Richard Bornat, the author of this paper. A lot of the comments on him here make completely false assumptions. In particular, he wrote his own programming text book many years ago which was quite well-known in its time, and he was an expert in functional programming, which he taught. So the idea that he's fixated on imperative programming, or just some dumb person trying to teach from a prescribed textbook is wrong, wrong, wrong.

Richard Bornat was Head of Department at the university Computer Science department where I teach when I joined it as a new lecturer (he's now moved on to another university for various reasons). He taught introductory programming, and I taught it for several years after him.

Every year we used to get into fits of depression - and I'm seeing it with the new guys who are teaching intro programming now - just WHY can't the students get it? We've tried everything - functional first, objects first, back to imperative first, use an IDE, don't use an IDE, use a language they use in industry, use a simple teaching language etc etc - and the SAME pattern still happens, a huge proportion of those on the intro programming course either fail or ought to fail (sometimes we just have to be lenient on the passing requirements since if we failed too many we might lose our jobs).

What's more, if you look at the big conferences on teaching Computer Science, you'll find people saying the same thing the world over. No-one seems to have hit on a way of teaching intro programming so that most people who take the course pick it up and end up competent in it. If there was such a way, and Bornat and me and everyone else who teaches intro programming are just bad teachers, wouldn't that way have become apparent by now?

Bornat's also a political lefty, the sort of person who naturally recoils from the idea that there are innate born abilities, with people naturally divided into categories. He's certainly not saying this because it suits his ideology. No, he's saying it because it reflects his long experience of trying to teach programming. It seems there really is a divide between those who "get" programming and those who don't.

The test he's using here is acknowledged to be really simplistic, an observation which might be taken further, but with statistics so clear it's obvious something is happening. If you read the paper, it's clear it's not a test on whether you've previously encountered