Good stuff. I'm interviewing for a developers position that I really want on Friday morning. Guess I'll have to test myself on some of these and shake off the rust...

anyone who asked me any of those questions would get a lead pipe across the head.
first of all...if your a CODE MONKEY your just DATA ENTRY
second...GOOGLE is your friend
third...what the f? are you hiring me to wrote code for the god damn space shuttle or to make some charts & graphs for business reports?

If you want a MATH MAJOR then say so.
I don't waste my time with idiot phone screens anymore
AND
I don't bother to even go for the "let me test you skills".

After 20 years of writing software I'll lead pipe you before you start

THIS BLOG IS A JOKE

ChaChaChing: After 20 years of writing software, you are a lunatic!

See, the screening process works: chachaching would be eliminated from contention without having to see him rave in person.

I understand interviewing is stressful. I've been on the receiving end of it plenty of times myself, and I am totally sympathetic. But please bear in mind we're not looking for perfection here, just familiarity. Remember the disclaimer!

--
Please understand: what I'm looking for here is a *total vacuum* in one of these areas. It's OK if they struggle a little and then figure it out. It's OK if they need some minor hints or prompting. I don't mind if they're rusty or slow. What you're looking for is candidates who are utterly clueless, or horribly confused, about the area in question.
--

Getting screened out on the phone may be painful-- nobody likes rejection-- but I assure you it's a far better alternative for everyone than somehow making it through to the hands-on test and failing in person. That is not pretty at all.

It seriously frightens me that there are programmers out there who can not solve that phone number problem in a full hour. That should take more like 2 minutes! If you can't answer that, go live in a cave.

Seriously, now I know where those code examples at the DailyWTF must come from. (Well, that and chachaching, apparently.)

Proposed new regex challenge: write a script to find where those programmers are working and fire them all before they bankrupt their companies or get somebody killed.

Let me ask you something, Jeff. The day you started work at your first professional programming job did you know all those things you highlight in this post? Because, I certainly didn't. Sometimes, it's the person's capacity/willingness to learn that's more important than the things they already know.

But what do I know, I'm just hanging on to the edge of the lower tier of the hierarchy of professional programmers. That's why I play the lottery, I guess. Programming all day for nimrods vs. sitting on the beach sipping mojitos --- I'll take the beach. Hey, a fella can dream...

How do you do #1 (have the candidate write code) over the phone? On a whiteboard in person, sure. Maybe even in some kind of IM session. But on the phone!?

The other four seem quite do-able in a conversation format.

These are very good questions. Complex enough that you have to think about them, but simple enough that you have no reason to only say "I don't know" for any of them.

From what I've found though, lots of companies seems at least equally as interested in knowledge of products (if not more interested), such as SQL Server, ASP, and .NET, than in fundamental concepts like data structures and big-O algorithmic complexity. That's why they're always full of incompetent code-monkeys.

Jeff,

Sometimes you have to understand the smartest of all people don't need to know everything. Even though some topics seem to be part of the standard set of "101 CS Questions To Ask", you don't need to know everything because it depends on what you're trying to do.

For instance, I was once asked during an interview what is an AVL Tree and how it works. I studied the specifics of AVLs 5 years ago during undergrad, and it's used to keep a tree balanced. Now, what is the exact procedure etc -- I literally don't know because I haven't tackled it in 5 years. But a quick google/wiki search reveals (that took _ONLY_ 5 seconds): http://en.wikipedia.org/wiki/AVL_tree.

Now the interviewer was not happy that I did not know the answer. I then asked him: if you're designing a multi-threaded server that needs to access busy network queues, what type of synchronization methods would you use? He couldn't answer..only to struggle with "some mutex". I was looking for Spin Locks, Reader/Writer Locks, Semaphores/Mutexes, Read-Copy-Update Locks (employed by linux kernel), etc.

Of course, I didn't want to make them feel stupid, but only wanted to prove a point.

I've been on the receiving end of 5 or 6 serious interviews for the first time over the past 3 months. I have around 3 years experience and a uni degree.

Whilst I can sympathize with the whole idea of testing and screening I don't think it should ever be the be-all-and-all of whether or not you reject someone - especially if you're looking for specialized skills that are not easy to come by (in my city, it seems good programming skills are in high demand).

The first couple of interviews I went to I did very badly in the technical test. The interviewer thought that I wouldn't be up to the standard they were after. The 3 or 4th I did much better. The 5th and 6th I blitzed and got some very nice job offers!

OK, I may have felt the pressure a bit after bombing on the first two, so I have been doing some extra study. Did my technical skills really develop so rapidly over the past 2 months, or did my interview skills develop? Did the first interviewers just miss a good opportunity... or were the later interviewers duped?

The fact is, it's easy to get good at interviews after a couple of times. You hear the same questions every time, so you can go away and cover the holes in your knowledge.

Whilst the interviewer may be thinking "I'm only looking for total vacuums", this is not the attitude I've experienced. You can see what I'm talking about at the end of Steve Yegge's post:
"What language are you most proficient with?"
"I've done mostly Java"
... Why would this mean the person would not be able to answer the data structure and HTML parsing question? The candidate is simply answering the question given to them. Sure, it may not be the best answer, but under the stress of an interview "best answers" are usually not given. I know for a fact that I would answer something along these lines but I would also be perfectly capable of solving both the data structure and HTML parsing question.

In the next question the guy forgets to answer "tree" and Steve thinks he will fail the data structure test. You can be sure though that the candidate will not forget "tree" in his next interview and the next interviewer will be duly impressed. In both cases it will be the interviewer who has really failed the test.

I think a big piece that's missing here, and from every step of most developer interviews, is questions on testing and debugging.

How do you know the code works? How do you design a unit test? What kinds of conditions do you assert in the code? How do you debug a crash? What if it's intermittent? What types of mistakes lead to intermittent bugs?

I find debugging questions quickly give me a feel for how much experience they've had actually coding. Even great programmers who write very few bugs have been at it long enough to develop good debugging skills. Noobs can often do fine on a simple coding problem but have no idea how to track down an error. (Nothing against noobs--sometimes you're hiring for a junior position. But sometimes you have to spot an expert.)

You can look probably 100% of these up on google. That would be my answer to all of these during a phone interview.

How do you think I found CodingHorror in the first place!?

I have been a programmer for years and have perfected the art of only knowing (retaining) that information which is useful at the time. I am now very good at knowing where to "look it up" when I need it.

Regarding the Phone problem:

"To me, this question indicates whether a developer will spend days doing programming work that he or she could have neatly avoided with, perhaps, a quick web search and some existing code that's already out there. "

Does an interviewer want to know how I would solve the problem, if I am able to solve the problem (in spite of not spitting out an answer within the entire hour, which assumes also, all things being equal, that I am a good "test taker"), or if I am able to effectively use Google, which proves nothing about my ability to program?

You mention out of one side of your mouth how Programmers Can't Program, yet suggest as a suitable answer for this question "I'd Google for an answer"?

Answer that way in my interview and it would be over. I dont want data entry people and effective web searchers. I want programmers who are good at what they do and are eager to learn to be better at what they do. Google teaches nothing except cutting and pasting.

I don't really think knowing a single language is a problem if the candidate can demonstrate a knowledge of the underlying concepts.

I've programmed in C++ for most of my career, but when required I can pick and program other languages with little trouble.

I've coded in Python, Perl, Php, Java and numerous custom scripting languages when required, but couldn't tell you a thing about them now and wouldn't call myself anything other than a C++ programmer.

A competent programmer should be able to be programming constructively in most other mainstream languages within a few days.

Kenneth: When you are hiring someone with no experience, you know that and the interview will be totally different. Most people are not hiring entry level people with no experience, they are hiring people that claim to have 20 years experience. These questions will filter out those that are not strong candidates for the 20 years experience they claim to have.

ChaChaChing: Another thing a phone screen can do is filter out people with personality problems. Threatening people with a lead pipe for asking questions show that you would fail this test.

People who are getting upset here: you're taking this the wrong way. Once you have to start sitting in on interviews, you'll understand the value of screening out the "No Way, No How" folks. I was never particularly hung up on The Right Answer; it's more a way to see how the candidate thinks, how they work through problems. I will admit I been in interviews where the interviewer seems more interested in showing how smart they are, but that's not the point here.

The regex one was a revelation when we started asking. I wasn't surprised that so many people had never heard of regexes (we were a Microsoft shop and thus most candidates were as well), but I was shocked by how many people insisted the problem either couldn't be solved or spent the entire answer period trying to get me to add constraints to make the problem simple.

Sorry, Jeff, but if most companies threw those questions out as part of a standard interview process, the job market for programmers would be insanely hot right now, because I'd be more than willing to bet that at least half the programmers out in the field right now wouldn't be able to pass all five areas. I know for a fact that I'd fail the Scripting/RegEx section, because A) I've had approximately zero experience with regular expressions and B) no experience whatsoever with scripting languages.

Blame it on my associates degree, but we didn't get into building class systems that modeled the whole fekkin' animal kingdom, nor did we spend the time on bitwise vs. logical ANDs. I would love to have spent time on it, but we were already trying to learn the basics, and we only had two years to do it; that's why I'm getting my SE degree now. Point being this - what Steve is looking for is someone who has (probably) several year's experience in every conceivable aspect of the programming world, which is understandable if he's looking for someone to work on the backend of Amazon.

If these are the kinds of questions that college graduates would be expected to know the answers to when they went to apply for their first job, you'd see the candidate pool for programmers/software engineers dry up pretty dang quick, before the applications even got filled out. I know that the first time someone asked me to "describe some options for implementing a priority queue," I'd politely tell them that I had no clue, and hang up the phone.

Then I'd move to the west coast and become a hippie, having my dreams crushed. :P

Aha! I knew Google's phone interview was bad. (They made me drive the call.) No fault of mine if I'm rejected by a bad process. ;)

I'm with Kashif Shaikh and others here. A lot of these questions are specific fundamentals that a quick google get you the answer.

What you need are candidates that know how to find out how to know and can grep the results quickly. People that can see what the problem is recognise what the solution may be, *understand* the why, be quick and effective in implementing the solution, and be able to explain why and stand by it (this bit catches the copy'n'pasters).

I've studied all sorts of data structures in college, written code in several languages and platforms over there years, bitwise operations in z80 assembly back in the day, etc.

And, I'd probably fail an interview such as this.

I'm working on a trading system for an investment bank, and I tell you what, the hardest part of being a developer is solving the god damn business problems.

"I'm working on a trading system for an investment bank, and I tell you what, the hardest part of being a developer is solving the god damn business problems."

Why is it our problem to solve business problems? Isn't that the job of businessmen? We're trained to be computer scientists dammit! ;)

@Jimmy:
"I'd be more than willing to bet that at least half the programmers out in the field right now wouldn't be able to pass all five areas."
Definitely. That's kind of the point, remember? FizzBuzz and all that?

"nor did we spend the time on bitwise vs. logical ANDs. I would love to have spent time on it, but we were already trying to learn the basics" --- Heh. What's more basic than the bitwise operators? Plus and minus? :)

"What Steve is looking for is someone who has (probably) several year's experience in every conceivable aspect of the programming world" --- Not at all. #1, #2, #4, and #5 are all typically covered in a 200-level (non-majors) programming course at U.S. four-year universities. #3 is easier if the interviewee has either basic Unix experience or power-user Windows experience; it's definitely answerable with the single magic word "grep". If the interviewee is so clueless that he can't utter the word "grep" (or "find" or "I'd use a regex" or even "I know there's a Java package that can do stuff like that, but I forget the name") then he doesn't get the job, duh. Remember, we're not looking for a polished answer here; we're just looking for the lack of *complete and utter cluelessness*.

"I know that the first time someone asked me to 'describe some options for implementing a priority queue,' I'd politely tell them that I had no clue, and hang up the phone." --- Exactly! That's Steve's point! The phone interview is *designed* to weed out unqualified applicants. If you can get the applicant to determine on his own that he's unqualified, you've just saved a bunch of time for the both of you.

Chachaching's comments just reinforce that if you find a candidate that refuses to prove that he can think, then you don't want that either.

Also, threatening violence during an interview probably won't land you in the "Please hire" check box on my interviews.

It should say in the summary section of your resume:

"I am a pompous asshole who can write business reports. I also can't spell or use grammar to save my life, so please don't ask me to do any of that. I also don't do math, windows, or laundry."

Heh - the phone number question is amusing, becuase the very best coders will probably answer it without having to write any actual code... just a short grep statement (or two) with a few parameters. The next best coders will do the exact same thing, but appear to be working for 2 days.

Stevey rip-off serach http://steve-yegge.blogspot.com/

I've just been through a lot of these technical interviews for .NET roles and in my experience I've not had any gruelling tests.

The general format of my (4-5) technical interviews has been:

1) Some question where I have to demonstrate a knowledge of SQL to the degree that I know GROUP BY and HAVING. (SQL)
2) What is polymorphism? (OO)
3) What is the VIEWSTATE used for? (ASP.NET)
4) Give me an idea on how you would connect to a db (ADO.NET)
5) What is the StringBuilder class used for?
6) What is reflection?
7) Demonstrate a design pattern and how you have used it.

I've been disappointed that I've not been asked how I would go about solving any problems. It seems most companies would much rather know you have had experience using the tools.

This is definately a reasonable set of questions. I agree that some of the proposed questions are difficult (I don't think I could get them all right) but the interviewer should be looking at the thought process, not the final answer. As Jeff says, if the interviewee is completely off, that should raise a red flag.

However, having said that, following the interviewee's thought process on the phone is a difficult task that requires effort on both sides. I think the way to go is by proposing ideas and giving hints to get the ball rolling.

Funny this comes up. We just ran into a problem in our office were people had to be let go because of excelling in a phone interview yet having terrible skills on site. (Maybe they were effective interviewers or googlers?)

I think the phone interview is very valid, and most of these points are good, with one caveat: interview them again.

Grab them on Day 1 (or in the face-to-face) and ask them the exact same questions (or a subset). Make sure they're still up to snuff.

Also, to those complaining about the questions, I don't think you'd not get the job if you both tried to answer and were moderately proficient in at least one area. I'd probably be weakest at the scripting bit because I hate scripting languages, but I can offer up an attempt.

It's not so much getting the answer exactly right as it is showing how you think and approach problem solving. I've worked in a few different shops that would rather have someone without a lot of knowledge but with strong problem solving skills. Granted, they're more needed for junior-mid roles than architects, but for those out of college, having the ability to solve a problem is usually more valuable than having the answer to a question memorized. In my experience, anyway.

Jeff:

I've read your blog for awhile, and usually agree with your blog, but this post has me rolling my eyes. I have been programming for over 10 years with a focus on Microsoft Desktop and Web Based applications. I've done a good deal of contracting, so I’ve had exposure to a number of software teams and I'm usually one of the most productive programmers in the group. I say this because you might consider me a “one trick pony” in that I’ve focused mostly on VB.NET since 2002, with the exception of JavaScript for web projects. Before 2002 I was paid to write in VBScript (classic ASP), VBA (Access and Excel) and classic VB (mostly desktop apps).

With all that said, I wouldn't get pass the phone interview for the following reasons:

1. The last time I wrote in C++ (no experience in C or Java), was in college (mid 90s). I could certainly read and understand most C code, e.g. when I find samples in C# on the web I normally can translate back to VB.NET without an issue. It’s absurd that you would limit your pool to just C, C++ or Java languages.

2. The Data Structure questions create the same trap. I can go into detail about the .NET data types that I've been using for the last 6 years, but I would fail at describing the differences between linked lists vs. a vectors, binary trees, etc. If the nature of the position you’re hiring for requires the programmer to code C++, C or Java, then it would make sense, but to ask a .NET programmer to speak in depth on these structures doesn’t accomplish much.

3. I would be hard pressed to recite the details of Binary Arithmetic. The only time I spent with the subject was in a Computer Science class, I’ve never revisited the subject in the last 10 years of application development.

Your practical questions related to scripting with the RegEx question seems logical. And the higher level questions related to OO design, which would be language- neutral would also be a good test to weed unsuitable candidates.

I’ve been on both sides, having hired programmers as direct reports and having been “code tested” by hiring teams. You have a valid problem -- weeding out incompetent programmers, but your solution of hiring similarly experienced individuals is an invalid solution.

The most effective software team needs a mix of talents. I’ll grant you that out of a group of a dozen programmers on a team, you need a few geeks that enjoy tweaking algorithms with Binary Arithmetic. But I’ve been in design discussions where hours are lost in debates about the efficiency of an array type, but the big picture was lost that the slight performance gain of either type was trivial.

I would add the following business solution questions to your OO and Scripting questions:
• A business requirement in the project you are working on requires a high level of secruity to secure sentive information that includes SSN, Credit Card and Medical Data that is stored on a public website. Outline and describe a solution that uses defense-in-depth and least privalage pratices that would most effectly would protect this information, focus on the software, not hardware issues to solve this issue.
• A business requirement is that an application interacts with, and displays information from a total of five online web servicse. The sources are not under the company control and the application design should accomindate for communication failures. How would you design a system that could meet these requirements?

The importance of these types of questions are two fold:
• You get a developer to reveal pratical experience, if any, in these subject mattters.
• You gain insight in their problem solving ablitiy, even if the applicant doesn’t have a partical experience in these areas

Half of the techniques in the article are useful, but the other half is very much counter productive. As another one with 20 years of experience in software business I agree somewhat with ChaChaChing that this technique will most likely get you a code monkey. All the projects that I mananged and led are successful because I put a great effort in interviewing and getting the right people.

The interview must match the nature of the project. If maths and science is important than the proposed coding and scripting test are useful otherwise they will screen off the best people. A Business nature project needs someone with good macro thinking and good business logic thinking but not someone with micro thinking and arithmatic logics.

Unfortunately we are not living in a perfect world, not many developers are good in both contexts. A good business developer is usually a very bad engineering developer and vice versa. So a generic test that tests on both skills will get you an average developer.

I read too many articles by self-aclaimed recrutment experts that have lost touch with the current generation and market. This is very much one of them.

It is now a rapid development and google generation. Hiring developers that like to write everything from scratch means you will miss your deadline. You need developers that knows how to find and reuse resources.

Also young developers are from so called Generation X. Best developers are tend to rebellous and like to "threatening people with a lead pipe". As much as I hate the new generation, I need to adjust to manage them so I can get the best out of the skill pool. I think this article will get you mostly older developers. But to get the extra cutting edge in business, you do need to take some risk to add some of the young and fast in your team.

Applicant: High order bit you say? let me think about that...

*sound of keyboard typing*

Interviewer: are you typing?

Applicant: ehhhh no.... why do you...

*mouse clicks*

Interviewer: Sounds like you're clicking your mouse... Are you googling..?!

*phone hangs up*

--------------

I worked with a guy that interviewed with Microsoft on the phone. He sat in front of his computer (at work mind you) and googled the answers. Too bad for him that "normalizing databases" is a subject too complicated to get a quick google answer.

just out of curiosity. whats the significance of 2^16?
Something related to excel? :)

I've been through quite a few of these types of questions in the last 12 months. I've been interviewing with a lot of companies and have learned to enjoy the process. The thing is I've learned a lot in the process and the questions are usually very stimulating. If I wasn't working a second job at the moment I would be out interviewing just for fun. I found the 2nd job looking for another job. The place wanted me so much that they let me work there nights and weekends. I enjoy the challenge and I like to learn what other companies are doing.

You may argue that I'm wasting a companies time. That isn't true. If the right offer came around with the right type of work I would take it.

About the questions. These are all basic areas and you could easily study up on them for an interview. There are many websites that are geared towards brain teasers and programming problems to study up on. Furthermore, knowing the information explained above can be very helpful tools for a programmer. Especially regular expressions. Whether your a Linux or a MS programmer you can use them. I know in VS you can at least use regexp to find and/or replace.

Data structure knowledge is essential if you are going to write any type of program that has some sort of time speed constraint and it handles larger sets of data.

OO Progamming questions are important to make sure the developer understands the paradigm. This is a good place to weed out people that write spaghetti code. I also believe it is important that the programmer understands the terminology. In any type of team it is important to effectively communicate.

The Bits and Bytes area can come in handy more often then you think. It really doesn't take that much to study up on this area plus it is useful for doing optimization.

Overall, I think the above topics are a strong indicator of a applicants skills. My understanding is in most interviews they aren't expecting you to get everything exactly correct and know everything. They more want to know your skill level and how you approach problems(thought process).

gulzar,

2^16 is how many bytes of memory my first computer had, after I upgraded it. A bank officer made the first student loan of his career, equal to a year's graduate stipend for me, to buy this computer. From a little-known company named Apple. Various friends learned to program on it...

To a computer, 2^16 is a nice round number.

Question: How do you look stuff up on google?
Answer: Er, hold on, let me type "how do you look stuff up on google" into google so I can find out - I never bother remembering how to do stuff.

Question: How do you solve a problem you haven't seen before?
Answer: I look it up on google beause everything has been solved before.

Question: How much drugs would I need to do in order to hire you?
Answer: Um.......

Yes, people, you know about google. And so do all the other people who're experts in looking stuff up while they're wondering why they can't get good jobs.

Also, this is a perfect set of questions for recruiters. Any company that has 90% of their work in ASP.NET and yet reasds this and thinks that they have to ask coding questions using only "C, C++ or Java" is clearly dominated by morons we don't want to work for. And anyone who claims 10 years of Java experience yet can't code a trivial string manipulation function is clearly a liar, no matter how great their ability to use google to find code may be.

And anyone who hangs up the phone when asked to write a function to reverse a string because they think it means the employer actually makes their staff write their own string reversal functions in production code is so hopelessly retarded that they're not worth hiring.

If nothing else, I don't want to work with a primadona who's so proud of their 20 years of experience that they're offended at the idea that they have to demonstrate that they can do some basic coding tasks. Even if you're all google-using geniuses, I can't imagine wanting to be in the office next to most of the people complaining here.

...these points are...pointless depending on the state of your effort.

if you want to mount an expedition
- you need an adventurer

if you want someone to lead the way
- you need a leader

if you want someone to clear the path
- you need someone with a machette

if you want someone to cart the luggage
- you need someone with cartage abilities

only in the programming field have I seen idiotic retards asking the leader to cart the luggage, hack the machette AND mount the expedition
for next to nothing

every one of the fresh meats that I have worked with always think their way is the best way - why? - because they have
NO EXPERIENCE
and
mommy, daddy and the girl they just laid last week told them are the best.

comes back to the old saying

you can be a Jack of All Trades and a Master Of None but
you can't be a Master of All Trades...

stick that in your pipe and smoke it

actually - I'd like to grab the lead pipe as well

2^16 is a reference to 16 bit OS or data structure.

someone grab bob and beat him over the head with the lead pipe.

he works in a cubicle next to us and is still looking for his god damn red stapler

hahahahahahaha

cha ching

let's all lead pipe it

Thanks Dave. 2^16 = 65536.

Just reminded me that excel has a limit of 65536 rows. Was wondering if this was due to some internal limit.

Also, ^ can be confused with XOR syntax.

Sorry Jeff, I have to side with the dissent on this one. Unless your disclaimer is really, REALLY broad, I think your questions are too "textbooky," way to bogged down in the nitty gritty.

If I can write code to handle whatever complex problem you throw at me, who cares if I can talk intelligently about a binary tree? I honestly can't, but I have college interns working for me who I'm sure could answer that in a heartbeat. Yet still I somehow manage to teach them a thing or two on a regular basis, usually on real-world stuff that these questions don't approach.

When I interview, I prefer work examples to pop quizzes. They transfer well over e-mail, I can review at my leisure, and they always reveal the coder's core ability. The only ones who get past the test are the ones who pass of other peoples work, in whole or part.

how often have you head this:

i know everything about [language here] even the arcane references to [arcane reference here] on [location in manual or on net or blog or where ever here] since i work on [reference to some project here] with [reference to some person here]

blah bla bla

sounds like the chicks I pick up every weekend at the bars in Los Angeles.

"I worked on [film name here] and met [film actor/ess here] and just loved the work of [director here] and oh what kind of car do you drive?"

does your company want someone who can solve problems
or someone who knows that tools inside out but can think their way out of a wet paper bag?

I just tell them to let me send them a project I wrote
with documentation to install, test and run
9 times out of 10 - these interviewers don't return my calls

9 times out of 10 - i don't want to work with that company if the interviewer is such an arrogant pompous ass thinking he is the only expert on the subjects.

9 times out of 10 - they are usually reading a script from some dumb ass lazy programmer who wrote the questions and

9 times out of 10 - you will be slogging on some code or maintaining some crap code for a shitty product that you will regret accepting the assignment for.

lead pipes!

Ohh, I get it.

DEC(25) = OCT(31)

In re: The programmer joke.
"why do programmers think Oct 31 and Dec 25 are the same day?"

Okay, now I feel really stupid. It took me a few minutes to catch on after overthinking it and trying to convert 1031 and 1225 into binary. Very reminiscent of several debugging efforts where I just needed to stop analyzing and re-read the dang error message right in front of my nose.

For the benefit of those of you who are thinking as slowly as I am today here's a hint "Oct" and "Dec" both have a meaning besides "October" and "December"

eyeD10T's

Hey, Bob at 3:56 PM, lighten up. Unemployment is LOW, so people aren't having a hard time finding work. Silly.

I have to partially roll my eyes at this post and also shudder to think I have forgotten many of the basics outlined in the tests. So for those of us who gave up Unix when we graduated college, we're not worth hiring? Really?

Should I be concerned that I never wrote code to shift bits? What if I cheat all the time and use regexlib.com instead of knowing reg ex's? Does that make me unclean?

Companies who use this screening tactic should let potential hires know that he/she should brush up on the CS 101 stuff.

I agree with chachaching, I would find an interview like that hostile and somewhat humiliating. Only that instead of a lead pipe on the head I would half-ass my answers the get it over as quickly as I could. If I'm lying about my capabilities in my CV you should notice on the face to face, the applicant is the one that is making the greatest effort to attend the interview anyway.
Besides the most important factor for hiring is that both parties agree on the money, if you hire chumps that's why the trial period is for.

I can't see what the fuss about this is. I'm just a high school senior and I could answer just about all these questions. Not some of the OOP definitions, and I'd have a bit of trouble with the scripting in a short time span, but easily done within an hour. I can do the regular expressions, but making it automatically go through the directory/directories would take a bit of Googleing, mainly because I don't write scripts or use Unix that often

MikeS has raised a good point. These questions don't show how to think critically, and obviously a high school senior can do the tasks prescribed.

Any candidate I interview, I ask them a basic networking question (what is the difference between UDP and TCP (TCP is reliable, connection stream oriented, and UDP is a best effort datagram protocol)? Give me an example that uses UDP (voice over IP).

This question has no merit, just want to listen to how people talk.

Then, because I am an actual developer, working on actual problems, I'll ask a problem based on what I'm trying to solve. To me solving real world problems is far more important to being an engineer, than say reversing a string, or memorizing quick sort.

And to reverse a string you call: string.reverse()

doy.

One of the hidden lessons in this entry has to be "know your limits". If an employer posed the questions here, I would certainly know it would be a bad fit.

I may have 10 years experience, but I'm a one-trick pony. (ASP.NET)

Although I would fail Steve's test inside of 5 minutes. Every single one of my interns, who are CS majors, would also die an embarrasing phone death.

I have a BA in a non-technical field. My thirst for programming books alone has served me well for the past ten years. It keeps me marketable, and bails me out of all kinds of interesting business problems, usually with six-figure price tags.

But I won't get hired at Amazon because I can't do binary in my head.

If I presented the above test to my boss, and demanded that all new hires pass it, I'd be laughed out of the office. 80/20 rule. (We'd also have no new hires!)

Occasionally I think I'm in Jeff's 20% elitist programmer group, but posts like this are quite humbling. Mercifully my salary does not depend upon this level of knowledge.

you dont have a string.reverse in java or C#. Which language are you talking about?

I think Steve's got the right idea, but the questions would definitely need to be tailored to the particulars of the job. In my line of work, regular expressions have virtually no use and most engineers would do just fine knowing C++ and only C++. I'm sure there are developers (like "Me" above) who do just fine not knowing C/C++ like the back of their hand. I'd be a little more hesitant to hire anyone unfamiliar with data structures--even though 99.9% of engineers won't need to write them, it's critical to understand their implementation so you can pick the right tool for the task (e.g., when is a vector/dynamic array a better choice than a linked list?).

I'm not ashamed to admit that I'd fail the regexes question, and hard. I've never worked with databases or web applications and have only a cursory knowledge of regexes and SQL. It doesn't make me stupid; only inexperienced in that field. If for some reason I needed to learn those things for a new task or a new job, I know I could pick it up. And more importantly, I know that I don't know enough about these things to go sending my resume to Amazon.

A lot of people are commenting "I could learn blahblah in a week if I needed to" or "I could just google blahblah".

This is simply not true for a lot of technical skills. You may be a superb Java developer with 20 years experience but the fact is you are not going to pick up ASP .NET skills in a week. There are just too many complexities specific to ASP .NET. If you're really smart you may be able to get your head round it all in 6 months. It is then a matter for the business to decide whether 6 months is too long to wait.

On the other hand, being a master of one skill to the exclusion of others, whilst not always ideal can work out for some people and can be suitable for some roles. I've got a friend who makes a very tidy sum just doing CSS. No programming skills or qualifications at all but he can do CSS like a mofo and earns more money than most people twice his age. The fact is, even the best programmers would need 3 or 4 months of solid CSS work to come close to his level of ability.

This is pretty subjective. I've encountered many smart and productive people in IT, who get things done, who would have a hard time with these questions.

One of the hardest tests I took was via email -- had to write code, compile, send them my source. Questions were such that research could pay off, but my turn-around time and resourcefulness were key.

A BIG problem, in my mind, is THE WAY IN WHICH interviews are conducted (like having 5 special questions), and the PEOPLE who conduct the interviews (i.e., not really qualified to do an interview).

I would value people who can analyze and understand a business process, and then model that in code, over anyone who can Regex.

First post .. the rant:
Are you sure you'd go over all of this on the phone? Please. How long would that take?

RE 1-trick ponies - Is working in C# ASP.NET since early days "A 1 trick pony"? As much as I'd like to dabble, I have a life. Sure I use javascript, know css, xhtml, ajax, data structures, use nAnt, nUnit, Cruise, sql, blah blah blah, etc. Oh wait, you couldn't answer the question about xyz, well I must be the idiot.

Worse typically is weeding done by HR/recruiters who only read acronyms, thus the smart person puts every possible acronym that's loosely associated with their knowledge so they don't prematurely get nixed. If they don't pretend they're gods gift to programming they get passed over. If you're humble and completely honest you lose out to the blagger who has skills but in no way would let you know they're not perfect.

What does scripting have to do with searching 50,000 web pages. I can that write a throw-away program in C# or python, what exactly am I doing wrong? You do mean recursive instead right? Unless you're a unix shop, your scripting question is crap. (grep sure, but it's just regex in a command line utility. maybe i've been programming windows too long)

Regex: I'm pretty good at it, but it sure doesn't come off the top of my head. I sit there with a builder utility and test hypothesis and write unit tests. So sue me, it works and I leverage the tool.

Miscellaneous rants:
Why do interviewers ask questions that have nothing to do with what they program in daily. Ask relevant questions.

Trick questions are useless.

For the love of ... when asking OO, DO NOT use the same old tired excuse for a question. Who the hell but a zoologist cares about this class is an animal, this class is a biped, blah blah blah. Inheritance is not the only feature of OO.

any new monkey() can figure that out that without being able to actually program in an OO related way. delete monkey;

Second post: thoughtful response:
The phone interview should be a 2-way litmus test.
a - they're not a complete idiot
b - they're what you're looking for
c - you're what they're looking for
d - tell them what the job really entails because job specs are always crap

Try to be smarter about the process.

Work through a probable problem:
Q1: You need to do x (e.g. 50,000 web page example). Tell them to make some assumptions, and if they assume too much, reign them in.
Q2: You discover per item takes 100 seconds, how long will it take, and what do you do?
Q3: You discover that performing the task takes 100% CPU as well, now what?

The answers I would look for are
a - recursion using whatever your language calls files in directories to create a work list
- use of regex or pattern matching. ask them how regex works
b - using worker threads. don't get caught up on syntax (that's what compilers are for)
- show them a few examples of code and ask which they feel is more correct.
c - here I'm thinking about farming out the work to more than 1 computer, and different computers need to cooperate. It can be as simple as a database, maybe it's more complex

Next, do some programming with them. Grab that crappy computer nobody uses in the conference room and give them some code to work on. Don't go for the super complex project, but go for something that is relevant to your organisation or what they will do.
Winforms if you do it
ASP.Net if that's your bag baby
Perl if it's what 99% of your codebase is in.
- add a feature to do X
- fix a small bug
- refactor code

OO - give them different code from above. The code should be 2 classes that do pretty similar things and there should be pretty obvious duplication. This is refactoring again, but it's also a chance to let them show you they really know OO. I'm thinking of creating a common base class, pushing functionality up or down the hierarchy, virtual functions, etc.

PS @thinker - loved your post

It's interesting that when these types of posts show up on the 'net, they get a large number of negative comments. I guess that only proves the point of asking such questions. Interviews are meant to whittle a large pool of candidates with varying skills and intellect down to a small set of people likely to be a good hire. If everyone thought the interview was easy and fair, then it wouldn't be effective in screening anyone out.

While it's not easy to come up with the right questions that test the traits of the candidates you are looking for, it's certainly a good a approach to make an interview challenging enough that you really learn something about the candidate. If all you do is ask them to talk about the projects they've already summarized on their resume, then what have you learned?

It's not coincidental that the firms I've worked at that had the best employees have also had the toughest interview process. Better to set a high bar and risk taking a while to find the right person than inviting every person who can stumble around VS in for a round of interviews.

I'm not convinced that these questions tell me much about an applicant's ability to develop an actual application people might use. Trivia about bits and bytes or data structures might be important in Yegge's line of work, but it's pretty low on the list if I was hiring someone to develop web applications (for example). Of course, web applications development is an example here; the questions should be tailored towards your line of work.

But performance on Yegge's questions don't tell me anything about how a developer would do in just about any real-life programming job. High order bit? Really? Does that factor into the way Amazon developers work, in this day and age?

When I read the comments, I get the idea that half the people think that these questions should be THE entire interview.

JHubSharp; Please don't tell me you hired people after doing just a phone screen and without any formal interview...

A phone screen is meant as a rough filter. You want to turn down a big percentage of the dirt even if that has the risk of also turning down a diamond or two. You just don't want waste the time of multiple people (HR, IT, interviewee) if a few simple question could tell you that you can't use him.

"2 days"... "short timeline"... That would be a red flag for me as interviewee, I would consider not working at your company. My reaction would be to ask why you would allow me 2 days for this task? If you allow me that amount of time, I guess you allow your current employees (possibly my future colleagues) that amount of time, which means I will be placed too close to a group of lazy morons. I would not function in that kind of environment. I would try to find out about that first from you, before answering.

Something like this I guess:

grep -l -r -P "(\([0-9]{3}\)\s|[^0-9][0-9]{3}-)[0-9]{3}-[0-9]{4}\b" .

I always find the people that post defensively on these articles kind of funny. Or the people who assume that these questions aren't elementary, and that professional experience is needed before you learn items like the ones mentioned.

I'm a computer science student, and every question above with the exception of some of the insertion complexity questions were almost easy. If you are a programmer and expect to learn this stuff 'on the job,' your education is lacking in some area. (And to anyone responding, I haven't studied most of this stuff for a couple years, it isn't fresh in my mind either.)

Jeff - I've seen you post clarifications a couple times in response to your own blog postings when people get upset at them. I understand why you do, but if your point was that the questions above are basic enough to screen potentially bad employees, I really don't think it is necessary. The people that disagree with you because of the question difficulty are exactly the same people I would screen if I were conducting the interviews.

Great blog, I enjoy your posts :)

@ David Avraamides on January 23, 2008 05:23 PM
You're missing the point. Take note of the perspective of most of the commenters. They are *interviewers* (myself included, see above), not candidates, and they are saying that these questions are not useful.

Yes, Jeff listed some tough questions, but tough questions = well-studied candidates, not necessarily good ones. Your analogy of top tier employees at companies with tough interview processes shows correlation, not causation. I'd be curious to see what kind of candidates they turned away in the process, or even the ones who avoided those companies completely.

@Frank on January 23, 2008 06:17 PM
I guess I read this post to mean more that you should pick some good technical questions suited for the position you are hiring for and challenge people to answer them. And you shouldn't expect that everyone will nail all of them - that's not the point. But if you take this approach, and apply it consistently across a large number of candidates, you'll begin to see trends that will help you make a better hiring decision.

And yes, correlation is not causation, but I still believe that having high expectations for candidates _does_ increase your chances of making a good hire - and that the downside of excluding a good candidate is more than offset by the upside of getting a better pool of quality people to choose from.

The whole coding test is fine, but coding over a phone seems kinda crazy. I'd say hold on I'll email it.

Another thing is I have built 3 hardware libraries in my time. Full of bit operations, when I was interviewed way back I could do math in Hex trivially. However it's been about 2 years since I really haven't had to do a bit operation. So unless you need a hardware guy, bit operations are not that main stream anymore these days.

I'd much rather ask about sql or network questions.

Why would someone really smart choose a staff programming job over investment banking or starting their own business?

There's a reason many programming candidates for staff programming jobs are average: the jobs suck!

@BoredGuyAtWork: As an interviewer I always ask about bits and bytes. Most don't know. It's not an issue though, you don't need to have correct answers. It's about the conversation and how you handle the situation. You can tell if a candidate feels like a failure when they don't know, or isn't phased at all when not knowing, is indifferent, doesn't care about not knowing it, is defensive not knowing it, is apologetic not knowing, is confident they could learn quickly if you just told them, etc. There are millions of possible reactions, and that's what we look at. Not at the actual answer.

If you did bit operations 2 years back I'm sure you would do fine.

Btw, why are you bored at work? Just quit today! Every minute you are bored at work is just a waste of your life. Only do what you love. Imvho.. ;)

I think if the top 20% is lamenting over the quality (or lack thereof) of the 80%-ers, I suggest they work harder to change the curriculum in both high school and college. Public school systems stink. Start at that level and rest will follow.

@Tom if you think the jobs suck that is telling a lot about you, not about the jobs. You are what makes the job.

I've never done any interviews, as I'm military and somewhat stuck in my current job, but I consider myself a fairly decent programmer for someone who is purely self-taught.

I realize there are many things I do not know. For instance, I know what bitwise shift and comparison are but that doesn't mean I can explain the importance of them. I've just never needed them.

1. I'd probably be a total vaccuum in the area of writing C, C++, or Java flawlessly. I can identify C/C++ and Java, but that doesn't mean I've ever used them in depth. I took courses in college that utilized them to some extent, but they weren't my language of choice(C# at the moment).

2. I'm mediocre on the OO concepts. I started teaching myself back in the days of GW-BASIC, and that's where I was stuck until I started making a life for myself less than 9 years ago. PHP and C# are the first time I've used OO concepts and I'm no master by any means.

3. I am not familiar enough with regex to write the expression for the phone numbers from my head, but I know where to find out and I've dealt with regex before. I could do the rest of the script easily though.

4. I have never used several data structures... even hash tables are still new to me, but I'm always up to a challenge and I learn quickly, especially if I have a chance to tinker around with existing code.

5. I can't give you textbook answers to bits, bytes, and binary mathematics, but I think I know the basics. I couldn't write a program to count the bytes in a number, but now that I've read about it here, I'm off to do some research. It just never occurred to me that something like that would be useful.

Sorry for the long comment, I'm just trying to point out how someone like me would likely fall into your "Never should have been a programmer" category, and that I think it isn't necessarily true. To echo a previous comment, it should also depend on the applicant's ability and willingness to learn.

This is mostly good advice, though the Dec/Oct thing is just an inside joke -- it doesn't demonstrate knowledge, just sense of humor. If you offered to explain to me the joke, I would think it was funny, though.

Also, I have ~5% hearing loss and sometimes in a pressure situation with a soft talker, I could run into trouble here; hopefully, people with very minor hangups people like slight hearing loss, crappy phone connections, or time-of-day constraints, would be accomodated.

Someone mentioned how being a skilled interviewee is sometimes enough to get the job done and I totally agree. I think it is in the interviewers and employers best interest to be ready to ask questions that are relevant to the current interviewee and go a little deeper than the superficial "what the diff between and abstract class and interface" type question. It's obvious that if you keep asking the same questions over and over they will find a way to the internet and the next time you ask them someone will have the perfect answer, and not because they're the best programmer who ever lived but simply because they know how to search.

My point is (and this is how I do phone screens) is start with simple questions in the 5 categories mentioned above and then dig deeper using the interviewees resume as reference. I like to ask them to model some sport that they like and relate to and I like to refer back to that example. That way the questions, while similar, are in no way exactly the same as the last interview I had.

just my 2 cents...

I think it's amusing how many people talk about having difficulty with the "scripting language" requirement.

Folks, it's one command: /bin/grep No scripting involved. Just read "man grep" and match the options up against the problem statement.

I haven't used Windows in years, but I can't believe the Windows shell doesn't have a similar command. Searching through files is one of the most basic things a computer can do.

I just went through a few months of trying to find a decent Java programmer (and I needed a real developer, not a script monkey), but I worked from the top down. I tended to ask questions such as these: what is deadlock, and what can you do to eliminate deadlock in your multithreaded programs? When would you call SwingUtilities.invokeLater(), and why? How would you localize a Java program? How does JUnit identify the testXXX() methods in your JUnit tests? Name some common refactorings. Java listeners and Swing actions are examples of what design patterns? How would you track down a memory "leak" in a Java program, and why does a Java program sometimes swell in memory when it has a garbage collector? Basically, I look to see if they can handle a commercial programming job instead of just being able to knock out a two- or three-hundred line Java demo. You'd be surprised how many programmers can't answer any of these.

I'm a sophomore CS major and I was able to answer most of these easily. I'm applying for summer internships soon, so hopefully I'll do alright.

[circa early 90s]
Interviewer: "So, are you familiar with common developer tools in a unix environment?"?
Candidate: "Oh yeah, sure am!"

Interviewer: "So you could tell me what a Makefile is?"
Candidate: "Oh yup, Makefile, make directory, I know all those!"

NEXT!

Not that the above is saying much. I also have a willingness to learn and explore. Not just rote memorization =)

For all those people saying a college grad can't be expected to know or pass these questions - I'm a not even a CS student, and *I* know these things. At the very least, I can demonstrate the basics and that I have sufficient grounding to pick up the more advanced aspects.

Why are so many of you feeling so threatened by this?

To those who disagree with Jeff a little bit, I would say Jeff just tried to find someone with global picture of the computing systems, to be more flexible to come up with different solutions for an application, in order to pick-up one solutions. He just tried to test the extension of your knowledge and skills.

Exposure to different levels and different aspects of technology surely will boot your ability to think out of square. After all, I think it is important to demonstrate your approach of solving a problem, if the interviewers really want someone with competent analysis/design skills.

The comments made me really sad. Now I can understand why the quality of software is really dropping all the time — it's because programmers consider some basic fundamental things to be specific and it gets worse.

Complexity of insertions isn't math — it's about whether one uses the right data structure. Binary arithmetics isn't about hardware or low-level stuff.

If a programmer doesn't know some of this then the probability that he/she will do something _really badly_ goes higher.

It doesn't mean that they can't do good programming sometimes; it means that every their commit should be reviewed.

I think that question about floating point arithmetics could be useful (e.g., what are possible outcomes of comparing x to sqrt(x*x)?).

I could probably fudge my way through the parts of an interview that asked these questions without embarrassing myself too much.

Good reminder that without revisiting the basics frequently, you get rusty.

I agree with the guy who said business problems are the hardest.

Most of us aren't going to go work for Google or Amazon or the next big thing. So we spend less time close to the metal, and often don't have the luxury of long stretches of uninterrupted flow, cutting trees and flipping bits like there's no tomorrow.

Doesn't mean we're dumb, I aced that shit in Uni :P Focus has just shifted.

The comments in this thread have really brightened my day. Programmers get pretty hostile when they realize that they suck.

IMHO, the better test in an interview takes the form of "here is a [very basic] API. Implement [simple algorithm using recursion] with it." It's not a phone interview question, but it's a first-live-interview question, which might be a showstopper of the entire process (no reason to waste anyone else's time if the first technical interview bombs).

The trick of course in making the API simple enough (without stripping it clean) and finding an algorithm to implement which is straightforward enough to be explained completely in a few minutes.

Bonus questions include: what would you change about the API to make it perform better? Where can additional safety be introduced?

This seems to be a key differentiator between a developer who can dive into our code base, figure out the dependencies, and innovate, and the developer who knows only basic CS 101 stuff but can't apply that to a real codebase (which is acceptable in a fresh-from-college candidate, but a major thumbs-down if the resume lists years of business experience!), and the developer who has coasted along on the coattails of others his entire career.

Why not just watch the homeboy (or homegirl) code on his own system using Yugma or similar?

@Lawrence, you got a point, I'm quite bored. However now that I think about it, it's not because of the incompetency around here, the other guys here are not incompetent, they could all answer these questions. But the general lack of soul, you know the will to just try stuff and challenge themselves everyday. I think interviewers need to worry about that above knowledge base.
Your also right about getting a new Job. I'm just wasting my time here. Gonna start job hunting today.
---
By the way I was wondering why in the world would anyone have 50.000 html pages. If you expect someone to solve the problem with scripts, shouldn't you first ask yourself why you didn't use a script language and regex to create those 50000 pages, making the whole issue of replacing a phone number a matter of updating a field in your database :)

Well Jeff it was bound to happen I finally disagree with what you say. When I first graduated school, I would have had that total vacuum in RegEx's my OO Design skills would have been poor. I could do it but I couldn't do it at the level I am at today. I think you've three very important things here.
1. The job context. I could probably figure out and test and develop you code that uses a RegEx to solve your phone book problem in maybe 5 hrs if I had to use RegEx. However writing code to drive a digtal earth reference model do you know how many places I have had a need to use RegEX ? Maybe 1 or 2, in the last 3 years. Then there is co-workers and the internet. So I think the context in some areas is pretty important.

2. Secondly Interpersonal relationships. If you're not a team player and you want to sit in a closet and code. Great I don't want you on my team, I don't care how good you are, END OF STORY! There is no test for that.

3. You're ability to solve problems. I am not interested in if you can do what's been done 100 times out there. Can you really really think critically ? I suggest the thought analysis process is the most important. I've seen developers who've had 10 + yrs who could code anything you gave them but couldn't have an original thought if they're life depended on it. Again these are old problems that have been solved 100's of times.

I think that these test degrade our past experience and references. If I have 3 or 4 + years of experience and half decent references, why assume that it's just luck that I've stuck around ? Obviously I must have done something right. If you want me to write a test to get a job then don't ask for a resume. I wouldn't waste my time on an employer that asked me to prove myself in an interview. That's what a resume and references are for.

I wouldn't go so far as to lead pipe someone but that would be the end of the interview because, I wouldn't work for anyone who felt my resume/references were good enough to talk to me but not good enough to consider hiring me until I proved my worthiness.

Good Post, interesting insight but I am in 80% disagreement with what you've said here.

Chris

Hi Jeff,

I usually agree with you on everything, but here I have to side with the mob :)

Your questions are really textbooky and I would fail at least 50% of them (as in, not having a clue without Google). I have no uni degree, six years of real world development experience, and currently being self-employed.

If you're doing a phone screen of an fresh out-of-CS-student then your questions will suffice - in checking whether he/she was paying attention in certain classes, but for any other ... geezzz...

Take a CS major, submit him/her to 10 years of real world development, idiotic requirements, even bigger idiots with money and of course users - and look at textbooky stuff slipping away ...

"Design a deck of cards that can be used for different card game applications."

This could be a black hole of design in itself. You could sit there for the whole hour coming up with useful stuff to cram in there.

"Design an OO representation to model HTML."

Correct answer: Google up the DOM. Cut, paste, done.

"Last year my team had to remove all the phone numbers from 50,000 Amazon web page templates"

The real task here is to make it so they don't have FIFTY THOUSAND frickin' templates sitting around. Who designed that monstrosity??

"What are some really common data structures, e.g. in java.util?"

Well, not being a Java programmer, I'll need Google to answer that...

"A candidate should be able to demonstrate a basic understanding of the most common data structures. More specifically, the big ones like arrays, vectors, linked lists, hashtables, trees, and graphs."

This could be an issue, particularly since till this moment, I was under the impression that a "vector" was a mathematical n-tuple representing an offset in an n-dimensional Cartesian coordinate system. This is despite the fact that I have used "vectors" in the sense the interviewer meant countless times.

"They should also know the fundamentals of 'big-O' algorithmic complexity: constant, logarithmic, linear, polynomial, exponential, and factorial. If they can't, that's a huge warning flag."

...that the candidate doesn't have a degree. Which you may or may not care about.

OMG, Steve would be also rejected by me if I saw his 'main()...' C/C++ program.
The standard says 'main' has to return an 'int'.

The phonenumber thing cracks me up, it's a very realistic "problem". I've had to solve it multiple times (for different kinds of data) I don't think it ever took me more than 15 minutes to write a usable regex for anything that can be expressed in one of a few ways and will be on one line.

I guess what I'm saying is that in a worst case scenario it'd take about 20 to 25 minutes for a real programmer to have a working tool to extract the requested information ready enough for testing by someone other than the developer that wrote it. Anything over 45 mins would be unaccaptable i think, even without regexes it's easy enough to do afterall.

It took me under two minutes to write /(\([0-9]{3}\)|[0-9]{3})(\s|\-)[0-9]{3}\-[0-9]{4}/ off the top of my head as a probable solution
(xxx) xxx-xxxx and xxx-xxx-xxxx right here in this textbox, so untested and probably with at least one typo, but given an IDE that should mean the regex is probably very doable in under 5 minutes.

The problem is simple enough to make me think its possible to do it with just sh, more, grep and some piping between them on a single command line.

Anyway, just my 2cts. good post btw.

@Lawrence,

Your post is a great reminder to read comments before writing my own. You are so right.

It seems to me that a lot of the readers of this article have got the wrong idea. It's not like if you don't know this stuff, you will be screened out because you can't program. The idea is to get a feel for the kind of person you are.

Obviously if they want someone to come in and get a specific job done then not having the skills in that particular area is a bad thing. But if they are looking for a long term employee then they are going to want someone who can learn and adapt to new stuff quickly. Having the skills and having the ingenuity to learn the skills are two different things.

I've not really bee working that long and haven't come across 'Scripting and Regular Expressions', so i have no clue about that phone thing. I could guess, and I’ve have an idea of where to start looking and that’s what I’d tell my interviewer anyway.

Besides i could always look up some comments made about a article blog about programming. :D

Atario,

It's not about java.util.*, it's about knowing basic data structuring libraries in your language of choice.

About the big-O, you probably don't have to know how to write it down. You just have to know that it if you want to look up a value in a sorted linked list, it will take twice as long for a list double the size (in average). While if you had to look up a value in a binary tree, it would only take 1 extra iteration/function call when the tree doubles in size.

Caring about big-O is not such a huge issue when there are few items. But when the amount of data starts to increase, a badly chosen data structure will slow down the application.
See also http://www.codinghorror.com/blog/archives/000957.html about the rationale of caring about big-O in sorting algorithms.

Having done quite a lot (thousands, probably) of interviews (I did it for a living for a few years before going back to uni for my CS degree), I would say that Jeff/Steve is/are generally on the right track. However, I would emphasise that you need a mix of general questions (eg questions 1 (modified to your target language set), 2, 4 and 5) and the specific (question 3).

A question requiring you to write a regular expression does NOT count as general knowledge. You should know what one is, but the specific ability to write one depends on how often you use them (I haven't written one in years, so I could no longer do it without looking up the syntax). But if you need to know them to do the job you're applying for, then it's a fair question.

It's a common trap to think that all good candidates (for any job, not just software) should be exactly like you in knowledge and outlook (although nobody could be quite as good as you, right?), so you need to guard against that. The regex and data structure questions particularly have that sort of smell about them.

Personally, I'm also inclined to ask some process oriented questions, eg patterns, unit testing/TDD, refactoring, SE methodologies etc, as that tends to give you a different perspective on the candidate than the detail stuff.

Questions about the interviewee's personal life/interests can generally wait until a face-to-face, unless you're interviewing someone a long way away, in which case you should be more thorough with your screening.

Also, the interviewer should NOT be doing most of the talking, although they should be controlling the direction of the interview (programmers in particular can tend to rabbit on a bit!). The phone interview is to find out about the interviewee, ideally quickly, to see whether they are worth meeting. The interviewer jabbering away does not contribute to that. You do need to mostly ask open questions that require an answer beyond yes or no. For example, ask the candidate to explain something, or talk about their experience with something, rather than if they've heard of it.

Also, the phone interview is good time to check out any red flags raised by the CV - long unexplained breaks, overlapping time periods, lots of very short jobs, impossible amounts of experience with a technology, and so on

@Atario:
"This could be a black hole of design in itself. You could sit there for the whole hour coming up with useful stuff to cram in there."
Which would neatly show that, in a real project with shifting requirements, you'd be incapable of solving basic problems without overcomplicating them.

"Correct answer: Google up the DOM. Cut, paste, done."
And? That's correct (I think), it solves the problem, and all you might need to do to show off is highlight the nouns you'd turn into objects.

"The real task here is to make it so they don't have FIFTY THOUSAND frickin' templates sitting around. Who designed that monstrosity??"
Converting 50,000 Amazon web page templates; a long, slow tiresome process that will cause more trouble than it solves, and will never be complete.
Writing the regex; about an hour's work, including debugging.
Pretending this problem doesn't come up all the time; Priceless.

"Well, not being a Java programmer"
What, you've never encountered or expect to encounter Java in any project? You can never be trusted to work on a single project involving Java? If you and Java come into contact, the universe would warp, and life as we know it would become lavendar-fresh bath salts?
I'm not a "C++ programmer", but my current project involves C++. Even if
"I'll need Google to answer that..."
If you need Google to answer the question "What are common data structures?" (like Lists, Linked Lists, Vectors, Maps, Hash Tables), how in the name of the Gods are you going to program if your net connection fails?
java.util was a *clue*, not a requirement.

"This could be an issue, particularly since till this moment, I was under the impression that a "vector" was a mathematical n-tuple representing an offset in an n-dimensional Cartesian coordinate system. This is despite the fact that I have used "vectors" in the sense the interviewer meant countless times."
So, if they asked you to perform a simple task at work, you'd make some clever quip about double-meanings rather than actually doing what they asked... That's actually good for an interviewer to know.

"They should also know the fundamentals of 'big-O' algorithmic complexity: constant, logarithmic, linear, polynomial, exponential, and factorial. If they can't, that's a huge warning flag."
"...that the candidate doesn't have a degree. Which you may or may not care about."
No, that they have no grasp of code efficiency. If you don't know why an exponential-time algorithm is bad news, I really don't want to use your sorting algorithm.
You don't have to know the full terminology, but words like polynomial and exponential are taught at high-school level. And appear in every text-book on the subject. And are bandied around by so-called "real programmers".

This is why they need to start asking these questions. It exposes not only any areas of ignorance, but also your whole attitude to problems and authority. Which is far more useful. You can *learn* regexes, learning how to deal with other people is harder. More useful too.

I won't say I aced the whole 5 questions, but I did the FizzBuzz in a couple of minutes. I'm not entirely happy with my solution yet. I want it to need one less if. Strangely, when I wrote it out on paper I did it with Java-like for loops and logic even though I haven't used them really since university. Perl for the win.

To all the haters, this is a phone screen. The whole hiring process is two-way, if you don't like the way the company hires people you may not want to work there. Plus, it's the type of thing _Steve Yegge_ says he wants to see. Presumably he has learned this from experience and different interviewers may look for different things. Probably most of the time most programmers don't have to care about the limitations of integers nowadays, but if you have no regard at all and you need your application to really scale really big, then you had better find out quickly. Though I might just brush up on my twos complement, haven't done that in over 10 years I think.

Someone said they failed at a couple interviews, but aced a couple of later ones and the later success was down to study and interview technique, not so much skill. Studying and wanting to learn is a skill. I haven't really had to learn too much outside my immediate field of work and I'm having trouble making myself learn more.

Well, I'm afraid to say I disagree with a lot of the detractors who've posted here. Great post Jeff.

I've done a fair bit of interviewing in my time and what I've noticed from people who've been hired is that there are a lot of people who can solve _most_ problems, but there are a few people that can solve _any_ problem, no matter how hard it is.

Example of "any" : There's a customer on the phone who's fed up because once every two months the app we sold them intermittently trashes a bit of data and causes their system to be unusable, but testing can't reproduce the problem in house. It happens.

The "average" developer will throw their arms up in despair at this and hope somebody else will come along to rescue them. Now this isn't actually a bad thing itself, if the developer is in a junior position, provided they can then learn and get better as a result of somebody higher up helping them out. (Focus on the first derivative!)

The "superstar" developer might not know exactly what the problem is, and may well not be able to reproduce it in testing, but they'll be able to look at the code, and think about the symptom, and come up with some ideas. It might take a while, but they'll never give up.

As a business, you need people who can solve all the problems that get thrown at you. It's no good having developers who can solve 95% of the problems if the last 5% means you have to loose your largest and longest standing customer.

Yes, I don't need to worry about writing my own code to reverse a string in my day job. I don't use regexs _that_ often, and I rarely need to worry about bits and bytes, and when to use a binary tree over a list. I don't deal with these things for 95% of my working time. But you know what, sometimes, you've just got to know about these things to solve a really hard problem.

And, all else being equal, people who've at least _heard_ of the sort of areas Jeff and Steve are talking about in their phone screen are more likely than not to be the sort of people who can tackle the really hard problems when the business needs them to.

From reading Steve's original post, I don't think he's even expecting competence in these areas during the phone screen. You might not know the exact regex to extract phone numbers from HTML, but you've at least _heard_ of grep, right? And you know it can be used to filter output on a file? I think that's enough to get through the phone screen.

Also, bear in mind Steve's phone questions are tailored towards a job at Amazon. If your business is 100% ASP.NET development and is never going to diverse away from that (and how sure are you that's the case), you might want to ask some different questions instead. Ask the candidate if they've heard of view state and session state. Don't necessarily worry about if they know the difference between them during the phone screen, chances are if they recognise the names they might be able to do a bit more than just drag and drop!

I'm sorry if I've antagonised anybody by saying this, or you think I'm looking down on you, but sometimes the "superstar" things are what you need to do to do your job! Maybe working as an internal dev in a bank's different and you don't get these problems - I dunno.

Rejecting a candidate is _not_ personal (or at least shouldn't be). I've interviewed for places and been rejected. It happens. Get over it.

I think that the questions with string reversals, malloc, btrees, anything related to hex -> n conversion are totally outdated and not productive, because they mesure "math" skills and not the real world knowledge

They are also favoring recent graduates without experiences (because those structural alogorythm tasks are part of exams in my country) and desfavor the experienced developers. I haven't done nothing of the above mentioned things after 1998 when I graduated and I code at least 40 hours per week.

Another thing here to be mentioned is that all of those stupid questions are very well documented on various sites so an smart candidate could memorize all those commonly asked "puzzles" and trick the intervier

To me much better approach is giving a smelly code to candidate to refactor asking them to debug something, asking some of the gotchas we have faced etc

Summarized: IMHO, this style of interviewing candidates is not acceptable and I personally feel somehow insulted when participating in that kind of interviews

Well, I'd be pretty happy with answering all those questions (although maybe a bit weak on Data Structures, and the answers might not always be 'perfect'), but I do think that those questions do cover most of the range of a general programmer. And good developers will have some holes; this is to be expected. There's no such thing as the perfect recruit. Someone who is baffled by all of the questions though, well, I'd avoid.

It's interesting the number of negative comments that seem to be along the lines of "I have no degree / a BA and I don't think these things are important". I did a degree that was more about electronics and mathematics than programming (Cybernetics). When I got hired into my first job, I knew nothing about regexs, databases, uml. So, I very deliberately went and learned these things. And they are useful, even if only used from time to time. I totally get why folks would look for these things in a hire.

That's not to say I'd only hire CS graduates. Computer Science degrees aren't perfect (the students are usually rotten or rarely brilliant at mathematics, for example), and CS isn't really about Software Engineering. But just saying "I don't know these things and I can program, therefore they're not important" sounds a bit like Pointy-Haired-Boss-ism

"I think that the questions with string reversals, malloc, btrees, anything related to hex -> n conversion are totally outdated and not productive, because they mesure "math" skills and not the real world knowledge"
Surely tree-related searching is real world knowledge. I mean, it's a core part of search-engines, isn't it?
Yes the debug approach is good as well, but isn't it hard to do down the phone?

I'm sorry but if you cannot show you know at least what the interviewer is talking about and make a stab at the answer, ten what are you doing in a programming job?

The RegEx question which so many people seem to think is irrelevant is basic pattern matching and even if you don't know regex's well (I don't) you should at least recognise that pattern matching is the answer?

The .NET people who seem to think they do not have to understand data structures... How do you choose which objects/routines to use for *efficiency* if you don't know the underlying datastructures? (Although it does explain why some .NET programs are so slow..)

The java library question as many have said should be tailored to the in-house language of choice

But as Jeff said these are not questions to be solved but to weed out the no-hopers who don't understand the question, and don't know how to find the answer...

Asking these kind of questions can be useful, but it all boils down to what is that you're after. I'd rather have someone who can fix the problems than a mathematician, I have a co-worker who'd be definitely unable to answer most of these, however, everytime we ask him to do stuff, he excels at it or knows when/where to ask if he's blocked. He's got great attitude and is very valuable for the company as such. I guess it's a matter of preference.

I agree with both parties, those who religiously claim that "if you don't know this then you don't deserve..." and then those who bash this saying stuff like "we know how to google".

I understand them both. :) Me? I'd have some trouble with some, but I definitely would be able to fix it and/or find a decent solution to all of them given a box, internet and some time. I wouldn't be afraid to ask or say I don't know this right now.

As Jeff always says: You don't want to the the smartest guy in the room. I'd love to work in a place where I could learn. We are not building a rocket :)

"It seems to me that a lot of the readers of this article have got the wrong idea. It's not like if you don't know this stuff, you will be screened out because you can't program. The idea is to get a feel for the kind of person you are."

Asking these test wil fail to get the good programmers with lots of experiences but let the good student programmers that just came from school get passed the test.

If I get a telephone interview and people ask me these questions. Than I start to have doubts about this company if it is worthwhile to work there. First of all they fail to ask the really important questions how to implement software in such a way that it is easily installable, easily configured, user friendly and when it fails, it does not detonate the computer. The questions about the OO and lists are only minor details in the grand scheme of your software product.
Experience shows that you cannot predict how a program will behave, on paper every- thing works but once you start to implmement and test then some technology that would be very logical and fast to use might nowt work. So the best thing to do is to create a framework that could switch technology or at least make it transient.

They fail to ask how I operate in groups, if I program quick and dirty or create quality code???

I have created very complex stuff that just works reliably and for many years. But I would probably fail the tests and I would probably not be interested in that company anymore if I get these type of questions.

Let me just say that as a programmer right out of college, I would have had no difficulty proving at least basic competency on those questions. They seem like an entirely reasonable set of quick weed-out problems.

He's not asking you to come up with the regular expression for the phone search off the top of your head - he's asking if you know *what* a regular expression is, and that it can be used for that sort of problem, and that in a day or two you can figure out how to write *one* into your favorite text editor and/or scripting language and figure out how to solve that problem.

After a year of experience (I know, still not much) I've found that the biggest time wastes I've experienced were projects where I or someone else didn't know the right term/concept to begin researching previous solutions. Programmers need to be more than cut and pasters - but they also shouldn't be trying to reinvent the wheel as it will often end up as a Reuleaux triangle.

Notice that he didn't say that they failed the test immediately because they only knew java, or forgot about trees - he said it was an early warning that they will most likely fail other skills questions on the test, based on his experience. I'm entirely willing to believe that a competent programmer could miss a few questions - I know I'd probably miss some of the technical OOP jargon stuff just because they didn't use those names in my classes - but I agree completely failing one area and/or failing multiple areas is a huge red flag.

One thing Steve mentioned in his original article that we haven't touched on yet is another "failure mode" where the candidate can't remember what they did in their degree two years ago.

Let's see, in my degree I remember ... C programming under Unix, shell programming, functional programming in Lisp, trees, quicksort, multithreading, PC internals, Minix, human computer interaction, and networking.

Haven't used Minix _ever_ since my degree but I can still remember it's a cut down OS and the way it allocates inodes for files does something funny when they get above a certain size.

And that was 15 years ago.

I've hardly ever touched Lisp, but every now again I write or refactor something that makes me think "oh yeah, anonymous functions in C# - they're a bit like lamda expressions."

If you know your stuff, you don't just forget these things!

From what I can tell, the goal of Steve's interview isn't to find a programmer that is 'the right fit' for the job opening that is currently posted. He's developed something that he believes will find top tier programmers. The kind of people that Joel Splosky is always looking for, or even Martin Fowler (http://martinfowler.com/bliki/PreferDesignSkills.html).

Most professional programmers aren't highly technical, top-tier individuals, just as most basket ball players aren't Michael Jordan. That's just the the facts. You can still use an 'MJ' test to hire an entire team and find out who is your MJ. Whether or not that's a good idea is totally up for debate.

Personally, I prefer the methodology laid out by YerFired above since it gets to the root of the problem and can find the senior staff you need without this kind of rigmarole.

g

This is a very interesting phone screen technique. I'm certainly interested in using parts of it myself, although I think my application of it would be much lighter.
I guess it depends on the level of the candidate that you're looking for too.

My one concern is that this just filters out those who can't remember their CS 101.
It's has the faint resemblance of a test designed by people who took CS degrees to remove people who don't have CS degress to justify the fact that they spent x years getting their CS degree as opposed to spending that time in the industry.
Both of those choices have their pros and cons and I don't think anyone can categorically state that either option results in better developers than the other. Arguably sure, but i've met idiots with degress and competant people without and visa-versa, so its not so black and white.

There are a bunch of _great_ developers who just _don't_ work in the CS mode.
It seems to me like for some people computers start with bits and bytes but to others they start with a problem of a potential client.
I like solving problems, the quicker and more efficient the better that's why I started coding. Bytes? Sure they're an important underpinning but I rarely need to use them because most of my clients want business software and I use .NET because it wraps low level things up and makes me more productive.

Before the low level community goes up in arms; what is the yard stick for a good developer? I've got software running every day across the world that helps people perform their jobs more effectively, it's clean, well architected and maintainable code. Some of my deliveries have zero bugs. But I've never written a XOR operation in my life.
So do I suck? Is everyone else writing business software in bytes or something?

I sometimes wonder if Yegge's wizards at Amazon actually failed because they wrote the core in LISP and make it so incrediably "good" that no-one could understand it. Is that good or unmaintainable?

This really rings true for me, and I'd definitely want to screen out the guys who need to Google this stuff. Sure, if you don't know what a hashtable is you can look it up on Google and find an implementation.

But if you never learned what it is, you won't understand its weakness and you'll wind up repeating the beginner mistakes. The worst part of it is the attitude that embraces ignorance. A good programmer is constantly *learning* which means adding to a body of knowledge and applying those lessons learned to make the software fundamentally better, not just perpetually patching over bugs.

"Write a function to count all the bits in an int value; e.g. the function with the signature int countBits(int x)"

This was my favorite question of the bunch.

int countBits(int x)
{
return sizeof(int) * 8;
}

Oh wait did you mean all the bits which are set to 1? ;)

Those questions aren't too hard. I was able to figure out 4 of them with a few minutes. The fibonacci sequence I did as part of brainteaser on the eulerproject. Break string into an array and, recursively pop it back into a string. OO questions I hate because its just more textbook junk. The bitwise is what threw me but I could brush up on it by the second interview.

Usually, when you are looking for work it's good to do a lot of interviews. Each one will make you a stronger candidate. I interviewed twice with the same company. By the end of the first I new I wasn't prepared and told the interviewer that. He kept my CV and called me a few months later. But by then I had found work elsewhere.

The questions shouldn't be treated as an all or none deal. If you don't know try to steer the interviewer towards your strengths. I've let a few interviews get out of hand (example: one diverged in coldfusion because they saw it listed on my CV, but I hadn't used it in years). And try to learn another language.

G'day.

I just took your test as a dare from a mate of mine.

I got 40%. Almost certainly a fail.

So given the fact that I've co-founded a software company (I'm the chief software guy) and we're turning over in excess of 80 million dollars a year, how do you validate the usefulness of this metric?

Here's a better question

You have n trees.

You need to produce a list of arrays of nodes of all combinations of the trees, where only one node can appear from each tree.

How would you approach this problem?

Nice post Jeff, spot on. I sent the URL around to my coworkers.

As Remy in "Ratatouille" might have said, anyone CAN program, but that doesn't mean that anyone SHOULD.

I think this post misses the point of the interview process. When I interview someone, I concentrate on their interpersonal skills, how they will interact with the team, and what experience they have, what their educational background is, what projects they have worked on, and what type of other programmers they have worked with and how they got along with them. As long as I am confident that they know how to use Google and can research answers then I am confident that they can do the job. As a previous poster said, I'm not looking for a math wiz or savant, I just need someone to develop decent eCommerce solutions and a web application every now and again.

I've been to interviews like this in the past and there wasn't a single question that couldn't be answered with five minutes of work on the internet.

Interesting set of links and articles to peruse. At my most recent job I ended up getting the junior position, when I had interviewed for a more senior one (but I didn't find this out till I came on... nice). Then the senior dev. turns out to be a bully, and a joke. Anyway, I do my thing, and the bottom line is, despite complaining a lot and being picky and demanding, I've just quit that job and gone on my own because the boss is incompetent (as a boss, he's really super nice as a guy) and because the co-worker is a bully.

No big deal, right? Only the catch is, I got a $15/hr RAISE, to WORK FROM HOME, on my OWN CLOCK, and I get to do the COOL PROJECTS. A simple test up-front would have shown who is who (but as this article implies, it takes management being aware of these issues for anything real to be done).

Well, after reading both this article and the linked article - I'm feeling a little less than super today :)

In any case, I would fair decent on most of the interview, fail on certain points (namely, I code in JavaScript and PHP, not a very data structured environment, likewise, some features of these languages aren't friendly to Java, C and C++) Most of my work is in scripting languages, I'm rusty with ASP classic, can muddle through Cold Fusion (non-tag based code) or grasp it (tag based CF), used to code in Java, took C and C++ in college (even Pascal and had a year of COBOL, but neither ever see the light of day on my resume). A long time ago, I needed to use Perl for stuff.

I don't think I would be totally clueless on everything, but I would provide incorrect answers to questions that, in my scope of programming, would be correct. For example, data structures, I'm familiar with the simple type (array, for example) - not vectors (as all arrays I use scale as I need them) so I wouldn't mention them as I don't use them and they wouldn't come to mind. Likewise, I create objects for complex data structures as I don't have prebuilt ones all the time for what I need.

Bits and Bytes would be fun (as I watch my binary clock and am able to read it...) but some questions I'd know how to do the work, just not necessarily how it is named - although the bitwise operations I've often glanced at but never really figured out a practical use (for what I'm working on) in order to use them. All the logical operands I use all the time.

The 50,000 page phone number lookup would have been easy, I would have mutter grep (not that I've used it, I just know about it) and regular expressions (which I would have provided the code needed for it)

OOP for JavaScript is a slight bit different than for tightly structured languages, I get about 80% of the terms listed on the other article from this blog and can provide examples of them (so, according to the article, I don't know the basics) - but I use OOP coding in my JS all the time, likewise, I'm using OOP in my PHP as well (even if I do use a framework... I do code a fair bit of the needed application on top of that, which uses OOP).

I was surprised no questions on data(base) structure, queries and such. While knowing a high degree of information on low level stuff, knowing how to access and work with databases would, I think, be key in working with large projects and (in my case, my focus is websites) ecommerce websites.

While I do think the list of questions (the concept) is a good one, the specifics of the questions aren't applicable to ever developer job. While some frown upon, look down upon, some of us 'script' coders - we are developers too and our questions should be different from the questions asked of traditional (dare I say "legacy") programming languages. Because, honestly, I see the code turned out from people attempting to use their "hard core" coding on JavaScript and, oh my word, it is the worst code I run across (for example, I have a simple date picking utility for a website form that I've seen written in 40-60k+ of code complex, all classed out and such - my code is a lean 13k, including CSS, which does more than every other similar product on the market - and don't get me started on form processing, can never understand why something so simple is so hard for Java coders to get... but I digress)

I would want to know how a coder would write efficient code (not everyone has broadband, after all, and even if they do, if every "feature" is 60k - the menu, the ajax, the date picker, the search, the CSS, the content slider, the tabs, etc you're 600k - and please, LEARN to use external JS files....)

From Jax on January 24, 2008 05:38 AM

"Some of my deliveries have zero bugs. But I've never written a XOR operation in my life. So do I suck?"

Yes, you suck -- but it's not limited to your lack of "XOR" experience. The fact that you would ever say something to the effect of "Some of my deliveries have zero bugs" indicates to me that you don't actually know much at all about delivering software. (Unless perhaps you count "Hello world!" among your 'deliveries.')

You have failed TWO of the major yard sticks for measuring developers that I use. 1) You believe that you have created non-trivial, yet "bug free" software. 2) You believe in magic.

Allow me to elaborate on #2. There are really only two types of developers: those who believe in magic and those who don't. Your assertion that you don't need to understand the underpinnings of what your software does clearly indicates that you believe in magic. Believing in magic effectively PREVENTS you from solving any problem that occurs on the other side of the magic wall between your code and the code it calls or in the process of crossing the magic wall.

I make it a personal policy never to hire developers who believe in magic, or non-trivial yet bug-free software.

Hi
I'm not sure about your type of questions. E.g. If I Have to evaluate Steve based on the answers he's provided
Look at the implementation of sumFile --> Summing int's into an int? hasn't the man heard of overflows? Not closing an IO in finally?
candidate rejected!

I usually ask questions based on the last project the candidate has worked on. If he can answer them clearly and knows what goes on in his project, he can handle mine. I normally give them plenty of rope at the start and pretend Im not going to ask much. Typically this means the candidate then states how he has taken requirements, architected, designed, developed, lead the team, tested and supported the application. After which i ask pointed questions and If i get an answer oh I didnt work on that area that much after having stated that he did, its a reject. I also always change a couple of parameters on the last project and ask now what changes in your solution?.

Its quite silly going with a fixed book set of questions. E.g. How many of us developing websites really work with bit's on a regular basis?

I'm genuinely surprised at all of the defensiveness here.

To the people saying that most programmers only have to write a couple of reports and a couple of forms: you don't need a programmer for that, just somebody with marginal Microsoft Access experience.

To the people saying that you could get the answers on Google: sure you could, but if you have to look up the word "the" in the dictionary in order to obtain the correct spelling, you're not going to make it as a writer. There are some fundamental things that you just HAVE to know if you're going to be working on shrinkwrap software or a mission-critical business system.

To the people saying that they've been successful in their careers in spite of not knowing these things: so what? Wealth and profits prove very little about actual skill and quality. Verizon and Bell make billions per year in profits. So does McDonald's.

If your idea of programming is to look up everything on Google, then you're a prime candidate for outsourcing. That's exactly what so many of the outsourced developers do: skim the first page of Google results, and if they don't get an obvious answer there, start spamming forums with inane questions. If this is your strategy and you're managing to make more money than they do, consider yourself extremely lucky. If you lack the ability to come up with *original* solutions, then you deserve to be a low-paid code monkey.

And if you have seriously never had to use a regex, a tree, a hash table, a bitwise AND, a string formatting function, a queue, or even a 2-level class hierarchy, and you're absolutely 100% sure that these things would never have come in handy, then please, for your own sake, find a new job, because your current one is turning your brain into putty.

If I weren't so sure that it would weed out ALL candidates, I'd throw a basic multi-threading question in there as well. How many non-trivial products today are single-threaded? Even web apps aren't immune; you have to know how to avoid long blocking calls on the server side.

As Barbie says, math is hard! Fortunately most of the companies I've worked for don't have anyone on staff who would understand any of those questions. The only question I could answer is #3:

3) Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages.

This is something I've worked out before and I have some code in my notes for future reference. You really need a recursive function to traverse subdirectories and ideally you should generate a report listing of the line numbers and files where phone numbers were found so you can double check your work.

I bet it would not occur to you to load the web in FrontPage and use its search and replace to do the same thing. It even supports regular expressions for find and replace.

""I'm working on a trading system for an investment bank, and I tell you what, the hardest part of being a developer is solving the god damn business problems."

Why is it our problem to solve business problems? Isn't that the job of businessmen? We're trained to be computer scientists dammit! ;)"

No, Damian nailed it. If you work in the business world, you are NOT a computer scientist. You are a business analyst with programming skills. The biggest mistake business developers make is assuming that the BA or the business got it right. They will throw competing requirements at you and not even know it. They will ask you to do things that will cost more than they are willing to pay, and its your job to put it in terms they understand and then build the system they actually want, not the one they asked for to begin with.

There are computer scientists, mathematicians and business analysts with coding skills. These are not the same thing. Know what you're hiring for.

Because computer science started in math departments, there's an implicit assumption that a math skill set and mind set is a necessary prerequisite for development.

It's *helpful* but necessary? Business analysts are more like plumbers. They have some basic skills. The put the parts together. They probably don't know the details about how pipe is made, or the soldering iron works, or the chemistry of polymer pipe. They may not care about getting a pipe with the shortest distance between two points because it doesn't *matter*.

A business analyst won't know how to optimize code either, for the same reason.

Getting the phone screen right means looking at what you need first, before you dial the number. The problem may not be them. It may be you.

Good luck with your interviewing, we went through the same thing here and tried to come up with technical questions to "screen" candidates. It never really worked out for us and our requirements were apparently at a much lower than yours (we were more interested in people who could figure out how to use common productivity tools).

I think the problem is that this approach is exactly the way a programmer would try and go about solving a problem - as if people can be treated like big algorithms. The unfortunate thing I have learned is that this is not really possible, you really need to bite the bullet and spend a lot of time interviewing people, relying on your gut feeling, and then taking a risk on someone. Taking exercises out of a computer science textbook is not really a great assessment of how people will adapt to your workplace.

ChaChaChing: After 20 years writing software, you forgot how to write proper English.

Hey Now Jeff,
another great post. Makes me think a lot about fundamentals.
Coding Horror Fan,
Catto

Jeff, you usually have good articles that hit the spot, but this one was so big "miss" you'd not been hitting barn wall even if you'd been inside of it.

A lot of people seem to think that perfect answers are needed to all these questions; note that this is a first-stage phone screen, just to decide whether or not to bring you in for interview.

So, "I think my text editor can do a recursive search for a string matching a pattern" is good enough for the regexes question (unless you claim knowledge of something like Perl on your CV); it shows that you are aware that there are tools to do the job quickly, even if you'd have to find them.

Remember that the goal is to remove the obvious losers before you bring them into the interview room, so perfect answers aren't needed. You want answers that show that the candidate is aware of the big world of computing, and that they are capable of escaping their box if they need to. Indeed, I'd be suspicious of anyone who could give me perfect answers to all 5 questions without stumbling or having to tell me what they'd research.

I dont think you'd get a code monkey for asking the guy what 2^16 is.
If he takes 3 minutes to think it through fine... but it is an extremely important number. And being able to answer questions like that would get you completely the opposite.

Some of the questions though would get you a code monkey.
But if you put it all together, you have a very well balanced developer that can play code monkey when needed to, and be creative when needed to.

"google is your friend"
Why not apply for a job there then?

hmmmm - not sure that I agree with this one either (I suppose that means this post will be removed as usuall).

I wrote some complicated stuff at University - but generally in my day to day business life, I try my hardest to make things as simple as possible.

KISS

orange, orange, orange, orange!

I just love when an interviewer uses a set of terms that are arcane, part of a general pool of terms developed internally at the particular company, or platform-specific, and expects an interviewee to understand what they're talking about without any further explanation. In one particular case I gave up answering many of their questions simply because I was sick of listening to questions using Windows-only terms that were almost as old as the OS itself, and haven't been used in most of the documentation since before I started writing software. In the end, I found that I really didn't want that particular job, and was glad that it wasn't offered to me.

The best part, though, was that I was able to recognize many of the questions as being minor adaptations of the crap floating all over the internet. The reality of the situation is that I probably would've been able to answer every single question just by reading blog posts and asking for different terms (if they could have provided them).

What I was really looking for was a job that was more focused on software development and that would understand that their particular work has a specific focus that may not be the focus I have had in my work. I would expect questions that allow for some room to figure things out and even the chance to look them up. What I usually find, though, is an escort to a room with 2 senior programmers that have been working in the same field far too long and expect everyone they work with to use the same language they do, and do most of their programming on paper or whiteboards.

I see we have a mix of opinions about these questions. Let's list some pros and cons:

PROS:
- They test the candidate on the RIGHT variety of technologies / skillsets (although I am worried about the lack of database questions), as opposed to HR's idea of "do you know language X?" with the accompanying oddball, irrelevent, googlable questions. (I was asked in a phone screen, by HR, what a transient variable is).
- They indicate the value of an education (As a person who graduated Cum Laude, I don't want to work with someone who doesn't value my education)

CONS:
- They assume that all programmers must be a jack-of-all-trades.
- It omits the most important question: "Why do you want to work for us?" This question should weed out 99% of all candidates. Some might argue that this is a question is an in-person question, but asking it in the phone interview gets the candidate thinking about the commitment involved. It also indicates how much research was done into the company before applying.

I haven't participated in interviews, but I talk frequently with someone who does. What people are really looking for is how much you want the job. I have answered all of these questions in interviews and still not gotten the offer, because I wasn't acting like I wanted the job.

From Ian on January 24, 2008 07:16 AM

"You have failed TWO of the major yard sticks for measuring developers that I use. 1) You believe that you have created non-trivial, yet "bug free" software. 2) You believe in magic."

Oh great so now i'm a liar too huh?
To account both of these deliveries were applications for mobile devices, one for stock taking the other was a major upgrade for a warehouse solution. They weren't Enterprisy but they weren't trivial.
Basically a big job for one man.
I was as surpised as you sound, I also believe there is no such thing as bug free code and the longer the phone didn't ring the more I worried, but it can happen and it DID.

I don't presume my code is error free which is why I have unit tests and also vigourously test before committing changes to my source control.

Unfortunately you just seem incorrectly assume I'm lying much like the people who incorrectly assume programmers suck if they don't know how to XOR.
Another risk in this scenario is letting your own ego get in the way of hiring a great candidate.

I'm amazed at some of the replies on here.

> 'I don't need to know about data structures, because I write .NET'
Huh?? Take a look in System.Collections - see those classes, they are data structures.
If you don't understand them then how can you choose the most appropriate one to use?

> 'I don't need to know binary because I code websites'

Uh-huh.. so if you need to query a database that uses a field containing flags then you are going to be lost?
Or if you need a function that masks together its parameters (like Open(Create|Read|Write)

> 'I don't know regex so I'd write a throw-away program'

So you have heard of regex, but rather than take an hour to go away and read about it you'd spend a couple of days writing your own broken version.
And 'I use Windows not Unix' is not an excuse - the Find option in most decent text-editors can use regex.

I do agree though that some process-oriented questions would also be good: describe unit testing; give a typical project life-cycle; etc

Sigh. For a few minutes at first, I was reading the first piece of advice as "Don't let the candidate drive to the interview." Er ok, I thought, if you think that's important...

These types of posts always seem to bring out the "best" in people.

Anytime I read a post regarding the types of questions to ask in an interview, the subsequent replies devolve into a pissing match. Almost nearly as fast as a ".NET vs Java" or "VB vs C#" flame-war of zealots.

This speaks to both the hubris of many developers and the complexities of what it means to write software. There are no "magic bullets" for interviewing because the needs of various positions differ so damn greatly.

Imagine an advertisement that was titled "Pilot Wanted". Ok, fixed wing or rotary? What type of aircraft? What a "pilot" is can mean a great many things, but any pilot should understand "altitude", right?

Steve Yegge's and Jeff's point is that there can unifying questions; so pick and choose what is applicable to you in a thoughtful manner and f*ckin' save yourself a lot of wasted interview time.

I've been on both sides of the fence, the quiz taker and the quiz maker.

As a quiz taker, the worst was a twenty question test on a "made up" programming language. The company required 19 out of 20 correct. I got 18, which wasn't enough to move on the next phase, and they didn't validate my parking either (bastards). The person who set me up on the interview said that the company did a lot with SQL, so naturally I studied SQL (silly me) before the test. Didn't help one bit.

As a quiz maker, we were interviewing for C# candidates, so I made up a little 10 question quiz. The questions were not that hard and I asked the candidates after the quiz, do you think the quiz was fair, was it too hard, too easy, that sort of thing. Most of them said it was fair and good test of basic knowledge. Basically, we wanted some programmers with *some* C# experience, who would be getting into a large C# codebase right away, not people with no C# experience, which would hurt the project.

The last question on the test was:

Write a method that adds 2 numbers and returns the result.

Usually I got something like this:

public AddNumber(int a, int b)
{
return a + b;
}

Question 10 was usually followed up with:

Do you think this method needs error handling?
What happens if I send in Int.MaxValue for parameters a and b?
And so forth. These were my probing questions.

After some probing, my final question was, how would you add 2 numbers together regardless of size?

Sometimes I got blank stares, other times I got solution proposals and so forth. But that question usually revealed alot about the candidate, and how they would go about solving a problem.

The truth is no quiz is perfect, I used quizzes as a validation that the candidate knew something about C#. They weren't the only thing we looked at. We had several candidates that did well on the quiz that we didn't hire. For the quiz, I tried to keep the questions from having to memorize anything or any "smarty pants" questions. But, I admit, this was tough to do. Quizzes in of themselves tend to put a superiorty complex on the quiz maker and an inferiority complex on the quiz taker.

There so much stuff out there, its impossible to know everything, so yes Google is your friend to get your started. I haven't worked on a Unix box in over 15 years, so Grep is beyond my sphere of infuence. But, hopefully the job description says something like "Solid background in Unix", in that case I would screen myself from the job by NOT submitting a resume. But most resume and job searches blast resumes as long as there is a keyword match.

Self correction... on previous post

public AddNumber(int a, int b)
{
return a + b;
}

public int AddNumber(int a, int b)
{
return a + b;
}

I'm glad you corrected yourself, Jon. I was about to put that code into production :P

One of the problems with phone interviews is that they don't replicate the process of sitting down and solving a problem very well. I look at programming as like constructing a building - you need tools. You bring many things with you - prior knowledge, experience, knowledge of coworkers (if you asked them a question for example), and various other "tools" such as books, reference materials, and yes, google. It's not the same as being asked a question and expected to answer right off the bat with nothing but what's in your head. It proves knowledge, not ability or talent. Any indication of ability or talent you get from the answers is just a projection of how that person sounds, not an actual test of their aptitude in those areas.

As a psych major I studied interview processes and an interview is the worst predictor of how well someone will do on a job -- think about it, you're testing someone's ability to interview well, or in this particular case, testing their ability to recall specific bits of knowledge. I think that's why some people have such a negative reaction to it -- because if you're asked to prove your knowledge, it makes you look stupid if you don't happen to have the answer.

The "If you don't know this you aren't a good programmer anyway" responses come off as elitist and condescending to me. Sure, studying for an interview proves that you're willing to put effort into something, but maybe some people don't have time to sit around and code and work on brain teasers in their spare time -- especially busy people, who have 40 hour a week jobs and families and lives. In other words... more well-rounded people.

I totally agree that for some jobs you need geeks, especially places like Google and Amazon (although I'm sure they also need people with good team skills and problem solving abilities)... some companies just don't necessarily need that; other skills are more important and should be evaluated for instead. Kudos to the person who said as much earlier, and that companies will in the end be taking a risk on someone and going with their gut. That's often more important for MOST companies, than looking for the egghead. But as always it depends on the task. Sometimes problem solving is more important, sometimes it's creative thinking or 'outside the box' approaches.

"Write a function to reverse a string.
Write function to compute Nth fibonacci number."

Just for kicks I wrote my own versions of these in C#, and then again in Scheme. It was actually a really fun exercise to do the latter! Maybe an interview question worth trying would be to have the candidate quickly learn Scheme (it takes all of 3 minutes, <a href="http://en.wikipedia.org/wiki/Scheme_%28programming_language%29">http://en.wikipedia.org/wiki/Scheme_%28programming_language%29</a>) and then write a few of those in functional/recursive style.

Anyway I never thought of the interview from the angle presented here, that of "find something they DON'T know and see how they fare," but it makes a lot of sense. I actually interviewed for a job about a year ago and I did pretty well. After being asked all sorts of cool questions about some non-typical binary tree stuff, I was asked to reverse a linked list in C. For some reason though, I completely bombed that question! The guy interviewing me obviously knew that this was something that could be found quickly through an online search and wouldn't actually stump anyone on the job. But, I screwed it up (I was *close*), and I admitted as much, so maybe that was fair enough. I just thought it was funny that I tripped over such an elementary question! (Yes, I did get the job)

Where are the jobs with interviews like this? I have had 3 programming jobs, out of my 7 total jobs in 9 years of IT work experience, and I have never encountered a phone screening, or even a skills test. Like many here, the samples provided in the linked posts seem trivial, and I have trouble (or would have, prior to reading thedailywtf) believing that someone out there who cannot pass these tests is holding a job as a programmer. I would love to go through such an interview process, if only for the novelty and fun of it.

those things don't define a good programmer - it's the research skills and coding experience that really matter, not if I can recite some BS to you - I agree with David
" It's not the same as being asked a question and expected to answer right off the bat with nothing but what's in your head. It proves knowledge, not ability or talent. "

I want ability and talent from my developers, not the memorization of how to compute Nth fibonacci number - your theory and methods are way off base - good luck in the real world

Phone interviews aren't supposed to replicate the process of sitting down and solving a problem. They're supposed to weed out the people who lack the basic fundamental skills to sit down and solve any of the important problems that your business encounters. Same goes for in-person interviews; they aren't going to immediately identify the vigorous go-getters, they're just going to eliminate the ones who are downright unpleasant.

As a Psych major, David, I'm sure you're familiar with Bloom's Taxonomy. If someone can't even demonstrate basic knowledge or comprehension of a subject, they won't be able to apply it at any job. Obviously - people can't apply knowledge that they don't have, can they?

Why is this so complicated? If you can't solve a simple problem, it's extremely unlikely that you'd be able to solve a complex one. You can't get through differential calculus if you don't know how to take a derivative. You won't make it as a statistician if you can't calculate a binomial coefficient. You can't write novels if your vocabulary is limited to 100 words, even if you've got a dictionary and a thesaurus right beside you. And indeed - you can't write a complex program if you don't know how to use a hash table, traverse a tree, or model a couple of animals and verbs as a class a hierarchy.

I truly hope that the people who consider these to questions to be "academic", "egghead", "brain teasers", etc., aren't actively employed as programmers or software engineers. The only thing worse than ignorance in this industry is militant ignorance.

Oh, and to say that interviews are the worst predictors of performance seems a bit specious, given that they're the only predictors available and are therefore also the best.

Apart from the technical questions I would also ask something about the company and the company values. I had an interview once were they asked me to arrange some company value words. It was a great way for me to know if I would like to work there or not.

Fresh out of school, I failed a bit on the technical questions (that was something like you mentioned) but got the value words pretty nice lined up. I guess I'll apply again soon ^_^

It is interesting that no one has talked about the setting of a phone interview. I rank them just above telemarketing calls. I've done lots of interviews face to face as the interviewer, but never had to do a phone interview as one (the company had buzzword jockeys for that). Yet I did a lot of them as an interviewee during a year long job search. Many of them came while I was preparing dinner for my family. One in particular came on my cell as I was coaching my son's soccer game. The interviewer refused to call back, and since I couldn't give any focus to the questions, I refused to waste his and my time and said good bye. A face to face is scheduled, and I make it a point to leave my cell phone and any other distractions behind. Phone calls are ad-hoc, and don't account for distractions. I only passed one phone interview, and that was for a job I didn't think I was qualified for. I thought I did well on several given the fact I was fielding questions from the interviewer and my kids, plus getting other calls.

> Watch out for one-trick ponies. Candidates who only know one particular language or programming environment, and protest complete ignorance of everything else, are a giant red warning flag.

And watch out for those that say they know a huge list of programming languages. They probably had contact with it in College or something, but they can't possibly know all of them.

Write a program to calculate and display the shopping days til Christmas in any way in the language that the shop uses the most...

To keep it simple, any day except Christmas is a shopping day.

It must return the correct answer.

If they can't do that then they can't handle:
dates,
leap years,
logically thinking out the problem.

Have them submit the code.

Look for:
Good usage of components (good)
Correct handling of leap years (good)
Good OOP principals (separating the interface from the function from the UI) (good)
Good re-usability (very good)
Good coding style (good)

Just a little suggestion...

The comments from this post reflect the ongoing battle between those who believe that software developers only need real world experience to do their job, and those who believe that a person needs a Computer Science or Software Engineering degree to do well.

My personal opinion is that a person who wants to be a professional software developer and not just a computer programmer will do both. He (or she) will get the CS or SE degree, and then try to work in all aspects of software development, not just implementation.

For my part, this post only emphasizes my desire to hit the Computer Science Theory books again to get a better grasp of the fundamentals.

Many people are single skilled like one-trick ponies. I strongly agree with the interview tip, but we need to understand that it is not the single skill that really is the issue, but instead is the attitude of rejecting the means of utilizing other languages as an alternative.

Colin Joss
Colin.Joss@gmail.com
East Lothian, Haddington
United Kingdom

Interviewers typically have to offer a lot of leeway for the nervousness factor, although some may have the wrong perception about the quality of programmers applying for jobs because they simply aren't able to see both sides of the table anymore. I see some of that arrogance leaking out in some of the comments -- Yes, of course there are people out there that just don't have the chops to be good programmers, but I think this is getting exaggerated a bit by looking at random experiences from interview performances. I say this having been on both sides of the table before. Believe it or not there are good programmers who don't do great with interviews in the style they are often done, and this can't be ignored as simply 'disqualifying a good candidate occasionally' -- especially when it's admitted the system is flawed AS IT IS and people get through those phone screens who don't have the skills desired.

RE: Aaron G from earlier - a better predictor of ability to learn and be a great developer over time would be an aptitude test that tests abilities seen as conducive to being a good programmer. (Problem solving... for sure, basic Math ability, yes... random code hacking on recursion and string manipulation problems, I'm not so sure. One that I took once even included analogies :)

For you to accept that interviews are the best predictor means that anyone who fails an interview must be a bad programmer -- yet in reality most programmers have flubbed interviews before, or done badly when presented with certain areas or questions they were unsure of (combined with the nervousness of being there). It seems a lot more shaky when you look at it in light of people's realistic experiences and the 'hit and miss' nature of the whole thing.

Especially considering all the people who make it past phone screens anyway (part of the original topic) - designing a better phone screen doesn't mean potshotting more random technical questions, many of which aren't related to tasks you actually will DO and work with on the job.

"Write a program to calculate and display the shopping days til Christmas in any way in the language that the shop uses the most...

...
Good OOP principals (separating the interface from the function from the UI) (good)
..." - Mac

Umm, why would this little exercise involve objects? This could be done in one line of Perl.

This post reminds me of a series of blog posts I wrote a few months back. I basically railed against everything Steve Yegge thought was good.

I still disagree with his philosophy.

http://codeeleven.blogspot.com/2007/11/want-job-reverse-string-sad-state-of.html

And a follow up:

http://codeeleven.blogspot.com/2007/11/whiteboarding-code-part-duex.html

I did a phone screen a while ago. The guy was monosyllabic, which I thought might be interview nerves, so I tried to draw him out a bit:

"What's the technology you're most enthusiastic about? What do you feel you're a real expert at?"

"Hibernate"

"Ok..." (I wrote a book on it) " so what version are you currently using?"

"Er... er... um... I dunno."

"We'll get back to you."

Google hires by the Lake Wobegon strategy (everyone they hire is above average). They are trying not to dilute the quality of their engineering pool over time.
http://googleresearch.blogspot.com/2006/03/hiring-lake-wobegon-strategy.html

Steve's questions from his Amazon screening days are designed to find people who are curious, curious enough to have gone all the way up and all the way down the abstraction ladder in computer science. If you aren't interested, are you interesting?

Also, look at where he's coming from. He's a productive Emacs and Unix hacker with some classic tools in his toolbox. He's trying to avoid people who would be profound time-wasters.

The interview questions are from the cutting-edge, research-oriented places with problems no one has ever solved before. There's no way to Google answers to them, and if you don't have the solid CS background to invent the answers, you won't be able to make yourself useful.

By the same token, if you're not trying to get a job at that kind of place, if you're not interested in the hacker culture or working with the best in the world, then you probably shouldn't worry about these kinds of questions.

And finally, for those who can't do these problems easily: even if they look stupid to you, the subjects they cover would likely broaden you as a programmer and increase your productivity.

I think that all of these questions are fair to ask a potential employee. The interviewee may not be able to answer all of the questions, but a lot can be determined of the person just by watching / listening to his or her reaction. For example:

For the telephone problem, the interviewee may not know how to use RegEx. Uh-oh... but it's possible to do this anyway... He or she could describe that they'd open each html.file and turn each line into an array of characters (ie. 'O', 'n', 'c', 'e', ' ', 'u', 'p', 'o', 'n', ' ', 'a', ' ', 't', 'i', 'm', 'e', ) and then loop through, one character at a time, seeing if it found a character that is 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9... then see if the next one does, and the next, etc, using a large string of ANDs and ORs.
This method, obviously, would be terribly inefficient, but it would show that the person was able to work around a barrier.
An undesirable candidate would not be able to work around a lack of regex knowledge.

Of course, Google solves everything, but......

It would be a bit like math tests where you show your work. Even a wrong answer still can get half credit for trying hard.

sobani:

"It's not about java.util.*, it's about knowing basic data structuring libraries in your language of choice."

In retrospect, I may have been taking the "e.g." as a direction to the interviewee and not the interviewer, as the original author may have intended. I was taking it as a literal and not a compiler directive.

"About the big-O, you probably don't have to know how to write it down. You just have to know that it if you want to look up a value in a sorted linked list, it will take twice as long for a list double the size (in average). While if you had to look up a value in a binary tree, it would only take 1 extra iteration/function call when the tree doubles in size.

Caring about big-O is not such a huge issue when there are few items. But when the amount of data starts to increase, a badly chosen data structure will slow down the application.
See also http://www.codinghorror.com/blog/archives/000957.html about the rationale of caring about big-O in sorting algorithms."

I'm not saying knowing about big-O is not important, just that knowing the notation and terminology ("logarithmic", "geometric", etc.) probably shouldn't be the real target of the interviewer.

Tom:

"Which would neatly show that, in a real project with shifting requirements, you'd be incapable of solving basic problems without overcomplicating them."

It all depends on what you're trying to accomplish, doesn't it? If you're writing a class for use in your company's lone card game, you'll want to write it differently from, say, a class for sale to card-game developers worldwide.

"And? That's correct (I think), it solves the problem, and all you might need to do to show off is highlight the nouns you'd turn into objects."

I meant that the answer would be that it has already been done -- as it says in a different spot, "don't reinvent the wheel". Especially since modeling HTML is a pretty massive undertaking to give someone in an interview question.

"Converting 50,000 Amazon web page templates; a long, slow tiresome process that will cause more trouble than it solves, and will never be complete.
Writing the regex; about an hour's work, including debugging.
Pretending this problem doesn't come up all the time; Priceless."

If you have 50,000 templates in use, you don't have templates. You have a mess. And I don't think anyone here is saying that messes don't exist. It's just that there's "doing the right thing" and theres "slap-dab it till it doesn't error out". Not that we haven't all been guilty...

"So, if they asked you to perform a simple task at work, you'd make some clever quip about double-meanings rather than actually doing what they asked... That's actually good for an interviewer to know."

Too clever by half, apparently, since it failed to get across to you that emphasis on specific terminology will mislead the interviewer to think that the interviewee doesn't know something, when, in fact, they know it by a different name. This is a problem with 99% of all computer "skills assessment" testing, by the way.

"This is why they need to start asking these questions."

My point is: they do need to start asking new questions, but they also need to be careful what they're really asking.

Wow - Ron - how amazingly patronising of you....

"And finally, for those who can't do these problems easily: even if they look stupid to you, the subjects they cover would likely broaden you as a programmer and increase your productivity."

I wouldn't be so bold to state that every good developer should be able to spit out a f*cking "Virtual Zoo program" on the end of a telephone.

and... Jon Raynor - what you wrote was a Function not a Method... In order for it to be a Method, your code needs to be inside a class

i.e.

class MathFunctions
{
public int AddNumber(int a, int b)
{
return a + b;
}
}

I would also recommend using camel casing on your Method decleration, include some comments, provide better variable names for your parameters and change the method instantiation type to static.

i.e.

/// <summary>
/// Class that contains general mathematical functions,
/// accessible to all classes (Public).
/// </summary>
/// <remarks>
/// Dated : 24th Jan 08
/// Author : First, LastName
/// AddNumber: Method that returns the result of adding two
/// integers together
/// </remarks>
public class MathFunctions
{
/// <summary>
/// Adds two integers together and returns the result
/// </summary>
/// <param name="ValueA" type="int">
/// <para>
/// a value to be added
/// </para>
/// </param>
/// <param name="ValueB" type="int">
/// <para>
/// a value to be added
/// </para>
/// </param>
/// <returns>
/// An integer value of the two parameters added.
/// Note: there is no overloading, or error trapping
/// in this method.
/// </returns>
public static int addNumber(int valueA, int valueB)
{
return valueA + valueB;
}// end of addNumber Method

}// end of MathFunctions class

Tim: It's certainly unprofessional to insist on an on-the-spot phone interview. I would expect most employers to allow candidates to schedule one within some reasonable time frame (and of course during normal business hours).

David: Funny, but I would have thought that a well-chosen and carefully-tailored subset of these softball questions *would* be an aptitude test. Or are you talking about something like the SAT, which no self-respecting developer would ever take just to land a boring job at BoringCom writing boring business software boringly?

You certainly haven't described your magical aptitude test in any reasonable detail. It seems like more of a roundabout way of saying that it's all subjective, which sounds like more defensive posturing to me. The only two things you mention either have hardly any bearing on programming ability (everybody should have basic math skills), or aren't even testable in a generic sense ("problem solving" doesn't exist in a vacuum, it exists for specific problems in specific domains, and that's exactly what we're proposing to test with these kinds of interview questions).

Nervousness isn't an excuse either. The real world is a stressful place, with unrealistic budgets and deadlines, conflicting and constantly shifting priorities, and a dozen people knocking at your door wanting their pet project done RIGHT NOW. If you can handle all that without losing concentration, you can *definitely* handle the interview; it's the reverse that's not always true, and that's why you have an in-person interview after the phone interview and usually some kind of probationary period after that.

My advice to you is stop referring to these as "random technical questions". All it does is reveal the underlying ignorance and bitterness. I don't even have a CS or CE degree - I'm an electrical engineer with far less than 20 years of industry experience, and these questions are just dead easy. I'm almost embarrassed to ask questions like this to people who claim to have been working in the industry much longer than I have, because I feel like I'm insulting their intelligence by suggesting that they might not know the answer. But the fact is, so many of them don't, and they've managed to blow through their entire career as a remora.

Oh, poor you, you flubbed an interview because you were nervous. That's life, man. Time to build up your confidence and come prepared next time.

Steve - it may need to be inside a class to be a method, but the question does not state a requirement to define the entire class. What Jon wrote is a perfectly valid method signature to stick inside any class (although it probably ought to be static). So actually, yes, he has "written a method", and you're just being a wise-ass.

Save those antics for TheDailyWTF.

@Steve: Um, wasn't that kind of presumed whatever the candidate wrote was inside of a class since he said it was in C#? The class in that instance was of no importance; the METHOD was.

Speaking of interviews, I just came up with a new concept: Have a candidate read and comment on a post just like this one.

Maybe a majority of the commentators here can program their asses off, but personality conflicts can bring a team to a grinding halt as fast as total incompetence. I would much rather take a humble-smart developer on board than a genius-asshole.

Snarky != Intelligent

I'm surprised at the level of defensiveness exhibited in the responses here. There are many different flavors of programming job so no one set of screening questions can possibly be appropriate for all jobs. I assumed that questions discussed were relevant to the types of problems being worked on at Jeff and Steve's shops. Am I out in left field for thinking that five years doing ASP.NET does not in itself qualify you to walk right into a job doing Windows desktop apps using C/C++?

I thought the question listed here were dead easy. On the other hand if you asked me about the first three normal forms for databases I'd be sunk. That doesn't mean I'm an idiot, it doesn't mean I'm a bad person. It does however mean that it would be a waste of time interviewing me for a serious database position. I've done database work in the dim past, and I don't doubt I could pick it all up again, but why would you do that when you can get someone who matches all my other stellar attributes, but also has the pertinent knowledge about databases already at their fingertips.

The most frustrating experience I ever had on the hiring side of the desk was at a small consulting firm in 1994. We all knew VB, I knew C as well, and for a variety of reasons we wanted to find an additional person who knew C++. Despite clearly requesting a strong C++ background in our ads, candidate after candidate would show up who thought that knowing C++ meant writing C code but using the C++ operators instead of printf and scanf . Inevitably they would plead that they were fast learners, and they could quickly pick up the rest of C++. We could only sigh and point out that we were clever fellows too and could do the same, but we were really looking for someone who already knew the language, if only so they could teach the rest of us. That experience is why I think these screening questions are useful.

Puh-leez this article is a joke. What planet are you on and when are visiting days?

I like the idea of asking indirect questions about tools and practices. IHMO a good programmer will have reasoned opinions about the tools he uses, the style in which he codes, and the process by which he codes. That can't be the only thing you quiz, but you can learn a lot by asking "Tabs or spaces?" and listening to the justification.

been on both sides of the phone. Basically, you have a staff position to fill, a contractor is too expensive, management wants to hire Steve Wozniak at the guy on the corner holding the "Going Out of Business" sign's salary.

You have approx. 200 applicants for the position (once you've weeded through the kids of people who attend church with your managers - if you wanted a Facebook page or someone to type your questions into Google all day, hoo-yah, but you're just looking for somone to mentor), all of whom are (according to their CV) dynamic proven self-starters with long tails of success at jobs working at the print shop and university computer lab.

Pop quiz - what do you do? what do you do?

Keywords! Everyone comes up with some questions they think are interesting - after 8-10 years of practical software development experience.

Then you surf the web and come up with a few more, collectively cooing "ooh, that's a good one" because the idealistic side of you wants someone bright-eyed and dedicated to lapping at the oozing pores of your wisdom, not considering that pretty much everyone coming out of college has just spent 4+ years alternating between learning how to have adult relationships, trying to cope with the expectations of their parents and working meaningless jobs while getting the entire collected history of humanity jacked into their heads Matrix-style.

So, you need to weed. Most of it is personality - you want someone who's not an ass, someone who can look at a set of poorly understood facts in miniscule timelines and solve the problem at hand without creating too many additional ones. It would also be nice for them to admit when they are wrong and show you when you are wrong without turning every discussion into a deathmatch.

Plus, it would be cool if they liked Golden Tee and anime and debating who would win a battle between the Death Star and Battlestar Galactica.

So anyway, now you have three goals in your little "interview this person" user story:

1. Conceal the fact that your system is a 2001 value proposition with a tree-ring like series of patches, fixes, stuff a contractor dude did during a 30 day contract, good ideas unrealized, bad ideas overmaintained, things people forgot to maintain and semi-tested code layered on top;

2. This person will not in fact be doing the job that's advertised, but maintenance and support work until s/he proves themself capable of wearing the big-boy pants and not ruining another one of your weekends with an untested patch;

3. Gently ease them into the idea that this is a job, which means you lose the right to expect justice when you cash your first paycheck. There's a great group of people, who are smart and dedicated and often leave you dumbfounded with respect for their intelligence and professionalism. You'd really like to bring someone new in and teach them some good things they could build a career on, but always remember that this isn't really a family and if the stock price tanks or some VP needs to have an accomplishment on his PMI you can and will be out.

So, they come in. It's cool when they dress up (at least to me) - not like we all hang out wearing ascots and drinking martinis but it shows someone is trying to tell you they take the job seriously and they want you to take them seriously. If I want to spend time chatting with a guy in Dockers and a muscle t-shirt I can play GRAW 2 with my sister's idiot boyfriend.

Of course, you're in the middle of a fix because Mr. Dipsy J. Doodle once again checked experimental code into production (and he's in a different reporting org so you can't properly beyatch-slap him, just write a five-paragraph email both managers will delete at the first detection of obtuse techie-speak acronyms like CVS and FUBAR) and you didn't have time to prep the great story about how this is a fan-fricking-tastic place to work.

You'd really like to ask "Hey, I have this process that keeps terminating on Wednesdays when the outside temperature goes over 60 degrees - it's a J-21 implementation of the RBVA-16 use case that tries to implement a singleton strategy pattern factory because that was what the architect we brought in to catapult a design specification at us had read about on the plane ride here" but you'd spend seven hours explaining the business problem.

So you have keywords. Because it's objective and easy and you have to get the pool down somehow so you can get back to work and fix the build in time to have dinner with your wife before the 10 o'clock news.

The respondents fall into three classes:

1. People who try to rephrase the first Google answer that came up;
2. People who sit blankly and say "um..I don't know."
3. People who say "I haven't worked with that, but I would try Google."

So...who would you hire? Is this where I reveal my secret sauce, my magical ingredient that lets me pick out the cream of the crop so we can all be entrepreneural and start our own blogs about developing turnkey Joomla solutions for non-profits saving babies in Afghanistan?

No. It's not. Because again I have no idea if this guy is a twit or a blowhard or what. I know s/he's inexperienced. So I ask, how would you find out? Have you ever dealt with problems like this before?

And then they fall into three subgroups:

1. People who are curious about everything;
2. People who are curious about what they think they are paid to do.
3. People who are basically not curious, either because they are obtuse or they think they have all of the answers worth knowing.

I prefer 1 but I have been through enough to know that I need 1 and 2. I do not want 3. If I have too many 1s, we have a great time and loads of fun and no work gets done. If I have too many 2s, there are a ton of related problems in Bugzilla you have to threaten people with termination/death to find root cause and take CA on. If you have too many 3s...well...that doesn't last.

So the tech questions are a gateway to the thinking process. And finding out if you have any chemistry with the person, which in the end is what hiring is all about.

and all of you outside-the-boxers who want to be snitty about corporate software - I'm sure your new site kicks ass, whether for the Taco Hut or that new social networking board that you and your 20 friends use came up in 5 days with free tools, but I notice you don't trust your investments, airline reservations and bank balances to the hipster doofuses of the world. And trust me, the affiliators and link managers of the world laugh like hyenas at most of your sites coming up in the affiliate request lists.

When it comes to money, or anything meaningful, you want Gollum in the cave and you want someone to be beating him with a whip while he minds your gold. That's the consistent side of the business.

@collator, I think you are exactly right. A lot of the "screening" questions seemed designed to select those who know the same trivia as the interviewer.

You want to hire people that have a spark of intelligence in their eyes, a curious mind. And if they fit in personality wise, to make the team greater than the sum of it's parts, bonus.

@collator - nicely put...

I enjoyed that, thanks :)

Albeit cynical, its true.

I would happily replace all of these tests with a test on coupling and cohesion. If someone doesn't understand the importance of these two concepts then they have had a deprived education. That is what destroys 99% of projects.

Also, what about a list for hiring managers? They can really suck too.

@YerFired:

You said,

Q3: You discover that performing the task takes 100% CPU as well, now what?
The answers I would look for are:
c - here I'm thinking about farming out the work to more than 1 computer, and different computers need to cooperate. It can be as simple as a database, maybe it's more complex

That's an interesting thing to look for. I would have looked for something like this:

"First, I'd want to know why it was taking 100% of the CPU's resources. Then I'd address that. If it's addressable by hardware or software, we could fix that. Either way, we profile the application to determine which parts are consuming the most processor cycles and optimize the heck out of them. If that still doesn't work, then we look at the webfarm solution."

Can the developer see the BIG picture? Is there a problem with the server hardware? Does the problem occur in both the test and production environments? If it doesn't, there's something different between them, and it needs to be resolved. If it's not a server configuration issue, then you profile the application. No line of business application should consume 100% of CPU cycles. HUGE RED FLAG.

Aaron G sounds like StrCpy() guy to me. This was the guy that decided that the best way to determine if ANYONE knows how to program is to check their knowledge of Strcpy:

http://davidavraamides.net/blog/2005/05/07/strcpy/

After all, ANYONE who knew how to program would have read this specific book and studied this specific function and be able to regurgitate it 5 different ways!!! Right? This guy is so convinced that everyone is thinking exactly like him, he isn't able to step outside his own shoes and look at it from the standpoint of a younger person. Similar to you I think, especially with the mocking responses you had to anyone who legitimately questions the validity of certain questions.

Look, you've probably been frustrated by clueless coworkers like most of us, but what I think you fail to see is that THESE INTERVIEWS DO NOT WEED OUT ALL OF THOSE PEOPLE - simply the less clever ones. Believe it or not, you're going to be working with people whose ability and knowledge frustrates you on some level. Trying to design a gauntlet that will make every person hired perfect is not possible - and yes, everything is subjective, but you want to have a reasonable test. You cannot tell me that anyone who knows how to program knows recursion - what about 20 year veteran COBOL programmers who have designed massive systems that didn't use recursion? The real question is, what does the ability to program entail? It's a worthy question and should be considered when designing good phone screens.

"A lot of people seem to think that perfect answers are needed to all these questions; note that this is a first-stage phone screen, just to decide whether or not to bring you in for interview."

You are right about the fact that it is not about the correct answers but how people respond. But if you notice in this thread, the response will be that you will lose good programmers if you even try it on them. In theory it should work, but in praktice, if someone asks me these stupid questions, then the first thing that comes to mind is that the people in that company are stupid moron's so I won't even bother to procede with the interview because working there will limit my skills in a some kind in mind-killing boring serial work.

Good developers do not think inside the box. They dare to go outside the box too, and try to experiment. Otherwise you just create another clone of a program that has been created a zillion times.

You need innovation. I always use Google since I am creating never before used softwaretechnolgy and maybe someone out there has some good idea that I just take and make it even better.

"We all knew VB, I knew C as well, and for a variety of reasons we wanted to find an additional person who knew C++. Despite clearly requesting a strong C++ background in our ads, candidate after candidate would show up who thought that knowing C++ meant writing C code but using the C++ operators instead of printf and scanf ."

I had the opposite experience, they asked a C++ programmer, but they thought that C++ meant USING the classes from MFC while programming in C. They actually forced me to do it their way since they did not understand what I was doing. Inheritance? Polymorfisme? Private?

> I dont think you'd get a code monkey for asking the guy what 2^16 is.

18.

I have seen a NASA UAV collision avoidance algorithm coded in C which used ^ for exponentiation, though that was in a research paper rather than production code.

Anybody else taken a crack at LittleRascal's problem? Good problem, harder than it looks, here's my (probably inelegant) solution in Scheme. I went for stack-intensive rather than having to do any copying/flattening of the data structures, etc.

I'm assuming your trees are binary, for simplicity, and defining a tree as a list with (data left right).

(define global-combos '())

(define left
(lambda (tree)
(car (cdr tree))))

(define right
(lambda (tree)
(car (cdr (cdr tree)))))

(define data
(lambda (tree)
(car tree)))

(define solve
(lambda (tree rest-of-trees current-combo)
(if (null? rest-of-trees)
(begin
(set! global-combos (cons (cons (data tree) current-combo) global-combos))
(if (not (null? (left tree)))
(solve (left tree) rest-of-trees current-combo))
(if (not (null? (right tree)))
(solve (right tree) rest-of-trees current-combo)))
(begin
(solve (car rest-of-trees) (cdr rest-of-trees) (cons (data tree) current-combo))
(if (not (null? (left tree)))
(solve (left tree) rest-of-trees current-combo))
(if (not (null? (right tree)))
(solve (right tree) rest-of-trees current-combo))))))

Then you do such as:

(define trees
'(
(4 (2 (1 () ()) (3 () ())) (5 () ()))
(b (a () ()) (c () ()))
(Z (X () (Y () ())) ())
))

> (solve (car trees) (cdr trees) '())
> global-combos
((y c 5) (x c 5) (z c 5) (y a 5) (x a 5) (z a 5) (y b 5)
(x b 5) (z b 5) (y c 3) (x c 3) (z c 3) (y a 3) (x a 3)
(z a 3) (y b 3) (x b 3) (z b 3) (y c 1) (x c 1) (z c 1)
(y a 1) (x a 1) (z a 1) (y b 1) (x b 1) (z b 1) (y c 2)
(x c 2) (z c 2) (y a 2) (x a 2) (z a 2) (y b 2) (x b 2)
(z b 2) (y c 4) (x c 4) (z c 4) (y a 4) (x a 4) (z a 4)
(y b 4) (x b 4) (z b 4))

This took me a few hours of muddling, and were I posed this question in an interview I likely would have managed to eke out a few things about "recursion" and "closures" and some graphs on a piece of paper, and probably the start of some incorrect but close solution.

Were you expecting people to solve this in the space of an interview, or was it intended as a thought process thing?

Delmania
You make a good point though I don't agree with your personal opinion as it stands. You state "He (or she) will get the CS or SE degree"
and speaking for myself the answer is not necessarily. However I do read a lot about Computer Science that interests me , I just don't feel the need to validate the learning with a degree.(Similar to not going in for a JEE certification for example). I do agree that some people look for theoretical answers and some for real world experience(I'm with the latter) and this post seems to be with the former.

Aaron G: I would have thought scheduling a phone interview was professional as well. But looking over the 21 interviews I did that year, there were only 5 by phone, not counting recruiters. I think one or two of those were scheduled. To be fair, other than the soccer example I gave, the rest asked if it was a good time to call.

Is that the spark fun cell phone ? Very cool.

Let me tell you the "Dec 31 vs. Oct 25" joke is just not 'funny'. Ask your mother, she will explain why. That kind of joke falls within a 'geek' culture that separates programmers from the rest of society... I don't find that healthy.

Been a coder for 20 years, working with programmers for a good while, I encourage my (often younger) colleagues to "not-be-a-freak"... For the sake of our mental sanity!

Now shut down your freakin' computer, get out & have some beers, and talk about non-computer stuff, ideally with people not interested in computers at all. In other words, get out of the box! That improves your thinking skills, the one-and-only to be asked for at any job interview. Alas, I am yelling that to myself!

I doubt more than 10% of our development staff, myself included, could actually deliver the goods on such a phone screen. If we were lucky you would be able to tell from how we handled ourselves that we were good candidates, but the whole premise here was to get past those gut feelings to an empirical test.

I give it a C-.

Jeff, I will bend the context of your article a bit to pretend that the definition for Q3 was simply "3) Scripting and regexes. The candidate has to describe how to find the phone numbers in 50,000 HTML pages" with no further definition supplied (a better test, I think), as I want to make a point about the assumptions that lie behind the questions (and the possible answers)...

Candidate 1: That's trivial! I can do that in 2 minutes! (Failed)

Candidate 2: I've never had the need to use regexes before, but, hey, that's what books are for. But first, I have the following questions.... (Passed)

Why? Because Candidate 1 will give the appearance of being brilliant and producing quick solutions, but his solutions will probably be rushed, half-assed and ill-conceived.

Candidate 2 asked the following questions during the interview, thereby demonstrating that she understands real world problems and client needs. She is a capable solutions developer, not merely a programmer with a collection of theoretical tools and no practical understanding as to how they might be used...

* How many of the telephone numbers are badly-formed, broken over multiple lines, or protected from harvesting by Java script or embedded as graphics files?

* Are any of the pages Unicode and/or in other languages?

* Once we determine the proportion of telephone numbers that cannot be extracted by regexes, what is an acceptable loss? Are we willing to lose all or them, none, or somewhere in between?

Now, drawing liberally from one of the comments above, the difference between the guy who took days/weeks and the guy who took 2 minutes is that the guy who took longer recovered 99% of the telephone numbers and the other recovered 25% (only properly formed numbers located in the USA stored in plain text).

If this were a privacy issue, for example, "we are required by law to not allow telephone numbers to be published or people can sue us into oblivion (or we can end up in jail)", the yield is important. And, as we are bending the context for the sake of argument, we also assume Amazon, being an international company, wants international solutions to international problems and does not want programmers who are thoughtlessly US-centric (a plague infecting many international websites).

Now, if neither guy asked any questions about the scope of the problem (and it's goals) before starting, both are idiots. One may have produced an inadequate solution that comes back to bite the company 3 years later with a massive lawsuit, the other may have wasted time and money unnecessarily by over scoping the problem.

Before anyone says, the scope of the problem is defined by the assumption that regexes are to be used: it is very common for employers to be determined to use Product/Method A to solve the problem (they only have a hammer and all the world is a nail) and it is up to you, as a seasoned professional, to advise them that Product/Method A may not be up to the job and Product/Method B should be used instead and explain why, clearly weighing up the pros and cons.

Sadly, you could eliminate a lot of candidates by asking: Do you pass parameters, or use global variables?; Draw me a data model for a database that holds questionnaires and their results; Do you know what a For-Do loop is and why you would use it?; What is a class?; What does 'event driven' mean and how does it affect the way you design your code?

I have met a disturbing number of in-house developers that would fail these questions.

"Watch out for one-trick ponies. Candidates who only know one particular language or programming environment, and protest complete ignorance of everything else, are a giant red warning flag."

The counter to this is obvious: those who know many are experts in none. Sometimes you want experts, not dabblers.

Just get by the "FizzBuzz" question in the phone screen and I'm willing to talk in person. My most recent problem is the human factor of hiring freaks with "insane" personalities ticks that don't emerge until three months after you hire them. Yikes! How do you interview for that?

Umm if the scripting question is soo easy to answer (find phone numbers in 50,000 pages of HTML) could someone explain to me how you would do this. Not that I am going to an interview anytime soon but I feel slow considering many thought it was easy.

billybob,

I don't know about other people.. I think that kind of tasks are not too difficult. It is more to rigorous instead. I don't see a method other than looking at it one by one (by script, of course). The way to speed this up may be with multiple "crawler" or faster connections.

"Umm if the scripting question is soo easy to answer (find phone numbers in 50,000 pages of HTML) could someone explain to me how you would do this"

Use a script (Perl being the obvious choice) that loops through the files, reads in the contents, use pattern matching (regex) to find matching text. Anyone familiar with Perl would consider it trivial.

I noticed a lack of questions about Software Engineering practices like requirements specifications, validation, etc.

Steve,

I did massive page searchings last year. It was around 30,000 pages to look for (not 50,000) and definitely not on Perl. I have to leave the 50,000 challenge to Perl experts.

Jeffry,

I think it is because most comment came from non corporate programmers.

"I want programmers who are good at what they do and are eager to learn to be better at what they do. Google teaches nothing except cutting and pasting."

And being so narrow minded teaches nothing but ignorance. My answer to that is, I would never work for someone like you, you settle for mediocrity and I'm above that.

"good programmers write good code; great programmers steal great code. "

"good programmers write good code; great programmers steal great code. "

Then who wrote the great code?

These are good questions - so long as the interviewer is REALLY just looking for thought process. It doesn't matter that the candidate knows about grep, it matters that the candidate asks about the tools or environment.

When I interview I tell the candidates to have google ready and I give them a moment to try to google things. To me, it shows more about the candidates' ability if they can come up with a solution having never seen the facts than one who had the solution memorized.

I hate to say it, but if someone makes me write some sample code outside the job interview, I will most likely pass. I don't have time to write some stupid sample code for you. There are a lot of other companies out there that are probably just as good as yours and they won't ask me to spend two hours writing a crappy little application so you can see if I can code. Ask me on the interview or don't waste my time.

I generally agree with everything posted here though. I think it's ok to have to google for exact implementation. I don't remember exactly how a radix sort works but I know that it's faster than quicksort for integers in most cases. I also know that a heap works well for priority queues, and I could generally describe what a heap is and why it's useful for that. Could I implement one over the phone? probably not, since I haven't had to do that since college, but it's not like I couldn't figure it out if I needed to. If I'm interviewing someone, that's enough for me in most cases, particularly over the phone. Obviously if I am hiring someone for a particular position that requires a high degree of computer science knowledge, I'd ask more complex questions and expect better answers.

The largest hole that I've personally seen in the process is the incompetent pre-screening that seems to be done by 1) HR people with no concept of either tech vocab or ontology, 2) automated systems parsing for acronyms, or 3) Possibly monkeys throwing darts at a stack of resumes.

I had a cold call phone interview (they called me) from a company in the Chicago area which shall remain nameless. Apparently they got a hold of my resume somehow. After 10 minutes of harassing me about Extreme Programming (which I'd had no experience with) the interviewer got really snotty with me, explaining that there was no place for me there since I didn't work in Extreme Programming. She acted as if I were wasting HER time, despite the fact that SHE called me out of the blue. I asked her where on my resume that it indicated that I was an XP expert, or had ever worked in an XP environment. Well, she got ruder still, and that basically ended the interview.

I later realized the problem. Under my skill set description, I had listed XP as one of the operating systems that I was familiar with. Obviously, whatever mechanism they were using to screen resumes for phone interviews was seeing XP and did not have the ability to put that acronym into context.

In another situation I had an HR person tell me that they needed a person to do RDBMs's. Though when I asked him what that was, he didn't have a clue.

In yet another situation an HR person told me they were looking for a developer with 10 years .net experience. This was in 2003.

Another time, I had an HR person tell me that they needed a developer to do XML, XSL and other EXI technologies. EXI was a proprietary application used by only one company on earth, the company I was currently working for. It was listed on my resume, so he thought he'd just make up a definition to go with it as if it were an open standard or other language.

If companies want to improve there phone screens, start at the front. Don't hire HR folks that can't turn on their computer and don't understand the context in which they are hiring.

As someone already pointed out, it's much more important to be able to quickly adapt, it's not about how much specific coding in any language you can retain, "do you know this and that" - yeah, sure, if I can have access to all my normal references: my previous code I have written and then google, then I can do pretty much everything.

The idea that you need to know how to get a phone number from 50 000 files in an hour without any references to be a good developer is just plain silly. If you are hiring people based on that I feel sorry for you, your company will have hard time adapting when the "new thing" hits and changes everything.

First of all, I don't like this inquisition style of interviews.

I'm doing interviews on a very regular basis.
And for me it's pretty simple.

Junior - all I want to know is:
a) can you say "I don't know ... help me"?

Medior
the same as junior + "Yes. I'm able to understand partial problems and write specification for them"

Senior
I can do the rest

...I don't care what people can or can't remember. All I want to know is how they think, what are the reactions on unpredictable situations and whether they can work in team.

What's so important on writing huge regexes from scratch ... or finding bugs in printed code?

simon

After I finished your test, I'd ask when do I take the personality test, the IQ test, the drug test and the physical...

I would love to have had an interview involving those questions! Never happened though...

Jimmy, and other "i have a degree and even I can't answer all those" chaps: I have no degree and no formal education in programming at all and I can. I think one of the schools this process is designed to filter is the school that thinks academic certification is strongly correlated to true programming ability.

Hi Jeff,

Am pretty sure the Original Joke is:
Why do Programmers get Halloween and Christmas confused ?
dec(25) = oct(31)
:p

Anyways, I would fair pretty well at this interview ... What money would you be expecting in the US for these skills? I'm in NZ and the money ain't good :p

:)

Funny quote: As Steve says, "Computers don't have ten fingers, they have one. So people need to know this stuff."

If he is saying humans have ten fingers and work in base ten, then a computer should have two fingers, not one, in the analogy.

It's sad that his point about learning tricks in other bases is screwed up in the quote he wanted to make memorable :)

OMFG, that's scary. Some of those questions are easy, but I have no idea what a vector is, for instance. I agree with the first commenter about the difference between a programmer and a math major (well, I'm in college now and with the shipload of math they're making me take, I guess I'll know this stuff eventually, though I'm at a loss at how it'll be useful for general-purpose programming). Also, I had to look up 2^16; I've seen the number before (it's in all sorts of programming guides), but I've never seen where it would be useful information really.

> If he is saying humans have ten fingers and work in base ten, then a computer should have two fingers, not one, in the analogy.

That depends if you consider "zero" a finger or not, I guess.

Briefly, the -E flag in the post is redundent to the -a flag. Rsync will likely fail to get crucial operating system files backed up and restored, especially while the OS is live.
Jeff, you intuitively went for a much wiser strategy. There are linux equivalents, but the method given is not one of them.

crap, that was meant for your backup post

Sorry to burst your buble, but this kind of interview is why so many companies are full of mediocres, morons and imbeciles. Any monkey on drugs can rebember a few technical solutions. If your company is structured where there is a clear distintion of who is a n analyst, designer and coder, then with this questions you are hiring coders that will putcode before anything else and are incapable of thinking, just on following code specifications written by thinkers.

Last company I worked was full of this kind of people, I call them "Russian Scientist", the ones that will write a solution because is theirs, instead of google one, or buy one cuz is someone else's.

Once we changed our interview process to get the thinking man, the one that understand there is more than one way to skin a cat and that after all we are solving an information problem, not trying to probe 1+1 = 2, the pipeline of issues really cleared out and we had more time to do more coding tht meant something to the bottom line of he company.

I have never been one to ask low–level questions of an Interviewee. What I usually look for is:

What experience do they have? What types of projects have they worked on? What type of problems have they solved, and how? What environment are they comfortable working in? Will they fit in with our team? Do they exhibit passion for their trade? Do they keep abreast of the latest technologies?

I usually ask a few technology or framework specific questions, however nothing so low–level.

Personally, if I were on an interview and were asked the types of questions you have listed here, I would be deeply offended. That would be a deal–breaker for me. I would leave the interview right there; short of telling the Interviewer to K.M.A. You may think this is a somewhat prima donna reaction. However, I did not spend 14 years building up my reputation, project portfolio, and satisfied client list to be treated like a recent college graduate.

Of course, in reality, I do not have traditional interviews anyway. It's usually a business lunch meeting.

stumbled through, read the article, wanted to say great job, it was a good read, and I'm sure to save it. thanks again!

Appreciate it. Shaken how I have forgotten my fundamentals. Agreed we don't work on stuff that involves shifting bits. Also agreed that we need to have a strong fundamental base and cannot justify forgetting them, even when you are only studying newer technologies every day. And also agreed that it is impossible to do so and it would take a constant super human effort to do so.

But hey... who said engineering was an easy 9-5 job.

messin' up is fine... not making an effort, not accepting the screw up or trying to justify your failure is what is unacceptable.

Kudos Jeff...!!! I love this one...

Been interviewing software engineers for 16 years. My experience as an employer has led me recently to the same strategy as Jeff. I now believe and test for:
1. Coding skills with simple loops and a bit of algorithm
- its amazing how many "programmers" can't code even this well
2. Some scripting
- anything really in any script language
3. Design
- User interface or API or OO, anything
4. A passion about something related to work
- a project, a technology, hardware, anything

In my experience, only someone who has all 4 is a really useful developer as opposed to a coder or an architect or an "information guy".

I had a phone interview with Disney, and I missed nothing - they mostly asked the same questions they had already asked me in written form before the phone screen! But after the interview I never heard a word from them - and they ignored all my emails & voice mails. What a bunch of douches.

These are smart advices on how to conduct an interview over the phone but I disagree with all the questioning, it is a real torture for the candidate.
I have 7 years experience, and from time to time, I forget things.
Ask me about data structures now, I would be less knowledgeable than i was 3 years ago for instance.

I refuse an interview where i know i will have to answer 50 questions about this and that.
I do not know if you guys do realize that we have left high school ages ago, and it is a real pity and a misery to have to study again, and prepare answers.

What is this exactly ? Do we really need to be super geeks and super intelligent to get a job now ?
I understand why there are so few women in IT when i see how boring the recruitment process can be.

i totally agree with chachaching.
This blog is a joke and i really do not thank you for giving such advices to conduct an interview.
IT interviews are some of the most boring in the market due to geek questions like these.

It is a real torture for the candidate.

I've only been programming professionally for about two years, almost exclusively with .NET.

My first interview was for a junior role, so the questions were more in broad scope with no real coding exercises. I had to take two written tests. One allowed Google, the other didn't. Of course they were watching me from another computer, so it was more of a "will he follow the rules" type of exam.

I've gone through a few phone screenings and they typically ask questions about, JOINS in SQL, what is polymorphism, function overriding/overloading, what is AJAX, etc...

I still find it intriguing that I've never had to provide a code sample. Perhaps it's because the positions I've accepted are not senior in nature and they want to know if I have the potential to do it, and not so much the exact knowledge at this moment of doing it.

I think it all depends. Some people can ascertain from just speaking to someone if they have a clue what they are talking about. Others want to see you actually do it.

I wouldn't be offended if at an interview they asked me to code some of the examples above. I might kind of laugh and say is this COMP 200 Intro to Programming, but I would gladly do it.

Also I almost refuse to believe that someone with over one year of experience cannot code a FizzBuzz example.

I was a music major in college. The only computer class I took had us "turning on the computer" as the first days lesson (easy A class).

I've been indie developing for 1 year, and taught myself everything. Even I can answer half those questions without a second thought. Reversing a string? Are you kidding me? I've written my own scripting for my games because I dislike Lua (even if it did point me in the right direction). Maybe I don't know some theory stuff a CS major would know but it doesn't change the fact that when you get down to it a real PROGRAMMER lives eats and breathes this stuff. A real programmer might blow an interview or two, but when they encounter something they don't know......they will learn it. Plug in those holes, not because they want the job, but because they want to be better.

This isn't to say that there should be a one size fits all interview question. It's stupid for a .Net developer to ask the same questions as a developer who uses Cpp except for general theory to see if they know how to program. At least you guys can get the phone interview. With no CS degree and no "work" experience I can't even get that.

Wow, I have to say a lot of that sounds very very familiar. I think that your company contacted me for recruitment. I must say that after three phone interviews and technical screens I decided I wasn't interested.

I can understand the desire to try and determine something of a candidates technical knowledge before bringing them in for interviews. But I was being asked to program, over the phone, very detailed and complex problems. Perhaps since the person doing the phone screen had spent the last 3 or 4 years working on that problem it was easy. But it was a completely inappropriate level of complexity for a phone interview. Also compounded by the problem that I was not being recruited for a programmer job.

I think that I have to agree with chachaching. This may get you a certain type of person and if that is the type of person that you want to recruit then well I guess it works for you. I want to recruit smart, thoughtful people, who have a broad and diverse background, and know how to research and solve problems. I really care nothing for the ability to regurgitate some meaningless program code at the drop of the hat.

I've been asked these sort of questions. If they're prefixed by "I've read your application and I'm sorry to have to ask these but we have to treat all applicant identically" then I'll humour them. Please do excuse me if I take a moment to remember what a Fibonacci number is, as they are not something I've ever had cause to actually code here in the Real World.

But if the employer's representative hasn't looked at the 40KLOC off FOSS software I mentioned in my application, then it's time close the discussion, as I've just learned all I need to know about the prospective manager's own skills and willingness to work hard.

"...are you hiring me to wrote code..."

Just hope they aren't hired you to teaching an English class.

Being indignant works better if you can put together an intelligent sentence slick...

I also have an interview coming up and I studied these questions, most of which were very easy (phew). I appreciate you posting this up for those of us who don't have 20 years of experience (or a lead pipe). Any intel on what employers may be looking for is extremely valuable information. Keep up the good posts!

Great post Jeff. It took me more than an half an hour to read all comments.

I did like it for my job interview.Before I don't understand the screen interview.It is mean for my IT professional or technological in Interviews.

I did like it for my job interview.Before I don't understand the screen interview.It is mean for my IT professional or technological in Interviews.

I certainly understand where the idea comes from in regard to this line of questioning for an interview, but I belong to another school of thought entirely in regard to "getting it done" as a programmer. I've been programming since I was 12, I would be some what annoyed by that line of questioning and not be interested at all in working for people who thought being so linear was so important.

The mindset revealed by your questions is that only linear thinkers matter, those who are centered around facts are more valuable than those who are focused on process. The questions reminded me of the busy work they gave me in grammar school, just to hammer in multiplication tables, repeat, repeat, etc.

I would never waste my brain cells memorizing all that stuff, it's dumb really, if I needed it I'd look it up, if not I'd keep it out of mind as it should be. Of course, all the linear thinkers love that stuff and they assign a belittled lable to those who don't belong to their club, the posts from that school of thought takes on a snobbish allure here. I understand why they think it's important and it's a self fulfilling vicious cycle that other members of the linear thinkers club only want to hire and value other members. It's also the flaw of being a linear thinker, you don't understand any other way of thinking.

That's not to say knowing this stuff isn't valuable, it is to a particular type of thinker. The problem is that those type of thinkers are stuck thinking in a linear, one size fits all manner - thus the trap this sort of thinking creates. The problem with programming companies and IT in general is the over abundance of this one line way of looking at things.

Someone who doesn't approach problems in this fashion is not "dumb" or unvaliable to a company and yes even as a programmer. Programming is about solving problems, how someone approaches problems when they really are problems is what makes a good programmer. The flaw with these type of busy work questions is that a programmer doesn't create needless problems for themselves, they avoid them. When a needful solution is prevented by a real problem that can't be avoided, then that's when a programmer's skill jump into action and finds the best short cut possible.

One does that by understanding the fundementals and using whatever means necessary to solve problems within the context they arise.

Being a spherical thinker and not a linear one, when I don't need to be that is, I get why certain people would put out this kind of litmus test out there, but the flaw is that not everyone thinks like that and being a linear thinker would limit you to thinking that yes they do. Someone, like myself, wouldn't be interested in doing a bunch of busy work for someone trying to se how many hoops I'd jump throw to prove to them how badly I wanted to work for them. I'd see the interviewer as flawed in his approach and limited in their concept of what makes a good programmer. The flaw is that this line of reasoning is so limited in scope.

Now of course the linears would brand me "not worthy" and would never question that their questions were off not me, but that's what the limits of being a linear thinker consists of.

I've intereviewed for these kinds of jobs and this kind of mindset has come up almost always, and I've never wanted to work for any of them because of how they approach things. I've found out that my best bet as a programmer is working for myself because these IT companies just don't "get it" when it comes to solving problems. Now that might sound like sour grapes but there's a reason IT companies are always "looking for people". If they'd stop hiring people who waste their time knowing things that don't really matter and were more process oriented problem solvers, they'd get people who had broader forms of thinking. that would translate into better solutions and not just a code monkey's code monkey with a code monkey stamp of approval.

That guy's post, chaching, was actually right on - he said it in a brusque fashion, but I thought he was being humorous whihc was in sharp contrast to the stodgy approach presented in the post. He was right, someone who ws that "stuck" on factoids needs a figurative lead pipe to the head, as to say a reality check. The approach in the psot was someoting out of some geek wish list, someone who lives in a vacuum seeking validation by evaluating others on linear thining prowess, as if that was all that mattered or was the only way to approach problems. It's funny how the linears took such offense to chaching's post, again they can only see things one way - that's the problem with having problem solvers that all linear thinkers. Yes, their focused thought process gives them on edge as far as mental reach - into the depths were others wouldn't bother. But what linears fail to realize, because they are linears, is that they gain that edge at the cost of breath. Haivng great depth and limited breath is a liablity in many circumstances, especially as a problem solver. Again programmers are problems solvers not code monkeys just creating code for the sheer joy of creating code. A problem solver dedicates their brain cells to what they need to know at any given moment, the right code at the right time is a tool to get to a solution. Showing off how much code you can rattle off from the top of your head in spontaneous spurts over the course of a phone interview is the sort of bragging rights a linear thinker would deem important. I understand linear thinking and thinkers so I get that, what tells you something is that it doesn't work that way the other way around and that is the fundemental flaw of linear approaches to things.

@Raf: Well I guess I must be "linear" then, because they all sound like perfectly reasonable questions to me.

If you are hiring a software developer then what exactly is wrong with asking them to develop some software?

Regardless of whether they think linearly, spherically or in hypercubes I would still expect them to be able to outline a quick solution to "Print out the grade-school multiplication table up to 12x12.", do some OO design and describe the properties of a data structure. That is after all, the sort of tasks involved in the job they are applying for. They seem entirely relevant to me.

But I'll bite: What questions would you ask a "spherical thinker" then?

If I'm asked these questions in an hour long *phone screen*, then I take it as "weeding out the idiots" and answer them. Got some similar ones in the phone screen for the job I have now, and let me tell you, the day to day stuff here is as interesting as any of you could imagine.

However, if I got them in the day long real interview? I'd be looking really carefully at exactly what they'd expect me to be doing. A job I was happy to be turned down for asked for the parameter list for a particular Java Swing widget - now, I used to work for Javasoft, but even when I was in the middle of implementing Mac Swing I could not have done that off the top of my head; that's what code completion is for.

XX)
#!/usr/bin/perl
while(<*.html>)
{
my $filename = $_;
my @data = <$filename>;
# Loop once through with simple search
while(@data)
{
if(/\(?(\d\d\d)\)?[ -]?(\d\d\d)-?(\d\d\d\d)/)
{
push (@files,$filename);
next;
}
}
# None found, strip html
$text = "";
$text .= $_ while(@data);
$text =~ s#<[^>]+>##gxs;

# Strip line breaks
$text =~ s#\n|\r##gxs;

# Check for occurrence.
if($text =~ /\(?(\d\d\d)\)?[ -]?(\d\d\d)-?(\d\d\d\d)/)
{
push (@files,$filename);
next;
}
}
# Print out result
print join('\n',@files);

There you go, finds phone numbers in that format, even from between html (if it's proper, ofc) and through line breaks. Coding time: 5-10 minutes. Sure, there's a lot of code out there to do that, but you really want just a random unix geek who can do perl/python.

Still, I'm guessing management would freak out "There's no way you got that out in less than an hour, do it again in some other way we can understand." *sigh*

It amazes me that so many people think that any of Jeff's questions are esoteric. I don't expect a candidate to have depth in all those areas, but they should know a little bit about them. We've used similar technical interview questions and have had excellent results in our hiring process.

I'd probably never apply at a company with such ridiculous criteria.

1) The code samples are utterly meaningless. Fibonacci sequence? C'mon. What indication will that possibly have of a candidate's ability to enter into an actual large-scale (or even small-scale). Very few programming jobs actually involve purely algorithmic solutions, mostly they involve understanding system architecture and library interfacing.

2) Phone screening. Fine with that. But remember it's a two-way street. If I'm applying for a job, I'm also screening the employer. Asking me to implement a silly textbook algorithm is a sure-fire way to get me to move on to someone who actually knows what they want.

3) Hands-on test face-to-face. I'm not quite sure what this exactly entails, but if it's what it sounds like, you'll certainly screen yourself out of a lot of good programmers this way. Personally I cannot think properly with someone staring over my shoulder. Secondly, as someone else mentioned, Google is your friend. Any sufficiently non-trivial task (otherwise known as any task worth testing on) should require some research (a very important part of any programmer's job - and one that many people fail at).

Seriously, with this type of screening process, I expect what you'll get is people who are good at taking tests (i.e. fresh college grads) and filter out people who have actually worked on large-scale software systems (but have long forgotten how to implement something like the Sieve of Eratosthenes because it simply never comes up in real programming work).

If all you're screening is fresh-faced kids, this process might help to some degree (you'll at least see who was paying attention in class). If you're screening veterans with a decade or two of programming experience this process is fairly worthless and worse, insulting to the applicant. If insulting the applicant isn't a major concern for you, then all I can say is you deserve what you get.

I'd like to add that I think it's important to have a mix of mentalities in any shop. In general you'll find that people who take a large view of software (i.e. from an engineering/architectural perspective) tend to not be the best at detail-oriented or drudge work (e.g. user interface code). On the other side, code-monkeys (as they are so affectionately known) churn out this stuff like it's high-school poetry but tend to create unmaintainable messes if they are allowed to "architect" an entire system.

Personally I'm the architectural type and know my weaknesses. I can help your average code-monkey turn 100 lines of code into 10 but have little interest in doing his job (even though, on paper, I'm more qualified). If you hire a bunch of people like me, your software will be beautiful under the hood, but look like crap from the outside. If you hire a bunch of code-monkeys, you'll have a flashy interface built on a code base that's unmaintainable. You need both types, and your screening/hiring process needs to reflect this.

The fundamental problem with creating formal processes for dealing with the vagaries of human thinking and behaviour is common across many fields: they tend to be too rigid and too narrow, and worst of all, usually become a substitute for actual thinking and responsibility for decisions. Guidelines are always helpful for helping us organize our thoughts, but if you aren't willing to step outside those guidelines and take personal responsibility for doing so, then I'd suggest you first hire (or promote) someone who is.