I <3 Steve McConnell*
Coding Horror
programming and human factors
by Jeff Atwood

May 16, 2006

Localhost HTTP debugging with Fiddler

I've had great success using ethernet sniffers (such as Etherdetect, or Ethereal) to troubleshoot communication problems. Installing a sniffer, even after installing the required WinPcap packet capture library, doesn't require a reboot. I frequently use sniffers to troubleshoot servers and desktops alike. Ethernet sniffers should be a standard tool in your development troubleshooting toolkit, too.

However, Windows ethernet sniffers do have one significant limitation: they can't sniff localhost traffic. Localhost packets don't pass through the regular network stack, so they're invisible to an ethernet sniffer.

What's a poor developer to do? The only recourse is a local HTTP proxy, such as Fiddler:

A screenshot of Fiddler

Fiddler has some special integration with IE that makes it particularly easy to use, but it can be used with Firefox as well.

I had some erratic results under IE7, but Fiddler basically works as advertised. There's tons of supporting documentation on how to use it, including two MSDN articles. I'm using Fiddler as a localhost sniffer that's limited to the HTTP protocol, but it does have some capabilities beyond what you'd see in a sniffer. For example, with Fiddler you can set breakpoints and tamper with the HTTP data before it is sent or received.

On the whole, I'd prefer to stick with a sniffer for localhost debugging. But a HTTP proxy like Fiddler is a reasonable workaround.

Posted by Jeff Atwood    View blog reactions

 

« The Long, Dismal History of Software Project Failure Snappy Answers to Stupid Programming Questions »

 

Comments

Of course, there are similar "sniffer-like" apps available as plugins for specific browsers:

ieHttpHeaders
http://www.blunck.info/iehttpheaders.html

TamperIE
http://www.bayden.com/Other/

FireBug
http://www.joehewitt.com/software/firebug/

Personally, I prefer to do sniffing at the ethernet level because it's the most flexible, and it's not tied to any particular protocol (HTTP), or application (web browser).

Jeff Atwood on May 17, 2006 10:48 AM

Live HTTP Headers is a really good firefox plugin that does the same sort of thing:

http://livehttpheaders.mozdev.org/

Tim Howland on May 17, 2006 12:01 PM

> Localhost packets don't pass through the
> regular network stack, so they're invisible
> to an ethernet sniffer.
> -- Jeff Atwood

This isn't exactly true.

Localhost packets do "pass through [much of] the regular network stack". Localhost packets just don't pass through any physical NIC or NIC driver layer (& possibly the network interface layer).

And since many Ethernet sniffers only record packets that pass through the NIC or NIC driver layer, localhost packets are not recorded.

Ian Johns on May 17, 2006 12:50 PM

> And since many Ethernet sniffers only record packets that pass through the NIC or NIC driver layer, localhost packets are not recorded.

Thanks, but are there any sniffers that record packets "below" the NIC or NIC driver level, then?

Jeff Atwood on May 17, 2006 01:11 PM

Check HTTP Analyzer: http://www.ieinspector.com/httpanalyzer/

It can capture any process's HTTP/HTTPS traffic (even to localhost) in current user session without the need to configure a proxy.

Stefan Schultze on May 17, 2006 01:15 PM

I suppose you could always try mapping localhost to an external interface in hosts, or just using an external proxy (and using the external interface IP), but if the app explicitly binds to only the localhost, you'll find that most anything you try is going to be a problem.

Pop over the sysinternals, and try TCPView and TDIMon (in the networking section). They *MIGHT* (heavy grain of salt here) do what you need, but you'll definitely have to write an app to translate their logs into something more useful for parsing.

Xepol on May 17, 2006 02:17 PM

Best opensource proxy / reverse proxy I have used is webscarab, just so feature rich:

http://www.owasp.org/software/webscarab.html

jimbo on May 17, 2006 03:34 PM

A sniffer can capture traffic, but it's a little harder to actually change it (inbound or outbound). It's also not as useful when the content is compressed or encrypted :) But if you're looking at packet-level stuff it can't beat.

c on May 17, 2006 07:46 PM

A proxy isn't the only option...there's several for localhost. The easiest is tcptrace (http://www.pocketsoap.com) that's a port forwarder...like localhost:81->localhost:80

Another is a tool like TamperIE that lets you modify the POSTs and GETs just before they hit WinInet.

Scott Hanselman on May 17, 2006 10:44 PM

Actually, there are several solutions to sniff a local (as in the same machine) connection:
1.You could run either server/service or client/consumer in a VM with virtual NATed or switched network (not bridged). I use VMware so my experience is with this one. Then you sniff packets.
2.You install two more NICs in you machine (PCI or USB), give them valid IPs and connect them with a crossover cable (one to the other). Then bind the server/service to one of those IPs and the client/cosumer to the other. Then you sniff packets.
3.You do number 2 but additionaly, if you just have to involve localhost (127.0.0.1), you route packets that have localhost as destination through one of these NICs. You'll have to fiddle with the routing table and I'm not sure about the exact command sequence (I have done this, but only with Linux). Then you sniff packets addressed or coming from localhost because they will pas through the complete IP stack (redirected to a physical NIC as they are).

gd on May 18, 2006 01:59 AM

I forgot to mention that I use a similar techniques to test serial communication.

gd on May 18, 2006 02:31 AM

I use TamperData for FireFox, very nice too.
http://tamperdata.mozdev.org/

blah on May 18, 2006 06:07 AM

I haven't found a reliable way to sniff PPP traffic on WinXP systems.

For example, my connection is PPPoA, and as such, appears to Windows as a PPP connection. Ethereal (mostly Winpcap's fault) refuses to see the traffic passing through it

ASk on May 23, 2006 03:36 AM

I found tcptrace very useful in my case. Thanks to Scott Hanselman for mentioning it.

I have a proprietary web application that I administer my web applications with. I can stop and start them using this admin application. The admin application talks (HTTP post to localhost:1085) to a deamon running on the same box and tells it to stop/start a particular application.

So I launched the admin application, stopped the daemon (to free up port 1085), launched tcptrace and told it to listen on 1085, clicked in the admin to stop my app and tcptrace grabbed me the HTTP post request that stops my app :)

Now I run a job that updates a db and then restarts my app to pickup the changes.

i5mast on November 13, 2006 01:28 PM

Just to reinforce what Stefan Schultze said about HTTP Analyzer.
It's a good HTTP sniffer for development.

> Check HTTP Analyzer: http://www.ieinspector.com/httpanalyzer/

HTTP Analyzer captures HTTP/HTTPS traffic (localhost) from the various processes.

Tiago on June 1, 2007 04:30 AM

I had a local proxy running ( WebCleaner ) to do some testing, and had no luck capturing anything with Wireshark, since it was all localhost traffic. Tried IEinspector and it worked great -- its a phenomenal tool! Thanks Stefan!

Asif on January 9, 2008 07:14 AM







(hear it spoken)


(no HTML)




Content (c) 2008 Jeff Atwood. Logo image used with permission of the author. (c) 1993 Steven C. McConnell. All Rights Reserved.